[jira] [Comment Edited] (BEAM-6740) Combine.globally translation is never called

["Etienne Chauchot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16776705#comment-16776705 ] 

Etienne Chauchot edited comment on BEAM-6740 at 2/25/19 10:04 AM:
------------------------------------------------------------------

problem is that :

{code}

PTransformTranslation.urnForTransformOrNull(transform) \{code}

returns

{code}

null

{code}

with

{code}

transform == Combine.globally(Integer)

{code}


was (Author: echauchot):
problem is that : \{code} PTransformTranslation.urnForTransformOrNull(transform) \{code} returns \{code} null \{code} with \{code} transform == Combine.globally(Integer) \{code}

> Combine.globally translation is never called
> --------------------------------------------
>
>                 Key: BEAM-6740
>                 URL: https://issues.apache.org/jira/browse/BEAM-6740
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-spark
>            Reporter: Etienne Chauchot
>            Priority: Major
>
> SDK translates Combine.Globally as a composite transform composed of:
>  * Map that assigns Void keys
>  * Combine.PerKey
> As Combine.Perkey uses a spark GBK inside it, the runner adds its own translation of Combine.Globally to avoid less performant GBK. This translation should be called in place of entering the composite transform translation.A pipeline like this: 
> {code}
> PCollection<Integer> input = pipeline.apply(Create.of(1, 2, 3, 4, 5, 6, 7, 8, 9, 10));
> input.apply(
>  Combine.globally(new IntegerCombineFn()));
> {code}
> is translated as the above composite.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)