You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Sam Whittle (JIRA)" <ji...@apache.org> on 2018/06/09 00:14:00 UTC

[jira] [Commented] (BEAM-4525) Allow combiner lifting with AfterProcessingTime trigger in Dataflow Runner

    [ https://issues.apache.org/jira/browse/BEAM-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506725#comment-16506725 ] 

Sam Whittle commented on BEAM-4525:
-----------------------------------

A work around in the meantime is to use FixedWindows or to manually combine elements in the DoFn before GroupByKey instead of relying on combiner lifting.

> Allow combiner lifting with AfterProcessingTime trigger in Dataflow Runner
> --------------------------------------------------------------------------
>
>                 Key: BEAM-4525
>                 URL: https://issues.apache.org/jira/browse/BEAM-4525
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-dataflow
>            Reporter: Sam Whittle
>            Assignee: Sam Whittle
>            Priority: Minor
>
> This seems safe with AfterProcessingTime since otherwise assigning to panes is based upon system timing.  A valid output without combiner lifting would be for all the elements to arrive in a single batch with the same timestamp so allowing combiner lifting seems equivalent and more efficient.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)