[jira] [Commented] (BEAM-1641) Support synchronized processing time in Flink runner

["Jingsong Lee (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15992110#comment-15992110 ] 

Jingsong Lee commented on BEAM-1641:
------------------------------------

There are some differences between the processing of event time and synchronised processing time in {{DirectRunner}}. The Source just emit the {{BoundedWindow.TIMESTAMP_MAX_VALUE}} as the synchronizedProcessingTime, and the downStream use {{min(clock.now(), synchronizedProcessingInputWatermark.get())}} to generate synchronizedProcessingTime.
But I think from the fundamental point of view, ingestion time and synchronized processing time have produced almost the same effect. So I think we can use ingestion time and let Flink track ingestion and event time at the same time.

> Support synchronized processing time in Flink runner
> ----------------------------------------------------
>
>                 Key: BEAM-1641
>                 URL: https://issues.apache.org/jira/browse/BEAM-1641
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-flink
>            Reporter: Kenneth Knowles
>            Assignee: Aljoscha Krettek
>            Priority: Blocker
>             Fix For: First stable release
>
>
> The "continuation trigger" for a processing time trigger is a synchronized processing time trigger. Today, this throws an exception in the FlinkRunner.
> The supports the following:
>  - GBK1
>  - GBK2
> When GBK1 fires due to processing time past the first element in the pane and that element arrives at GBK2, it will wait until all the other upstream keys have also processed and emitted corresponding data.
> Sorry for the terseness of explanation - writing quickly so I don't forget.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)