You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/10/02 18:31:26 UTC
[jira] [Commented] (FLINK-2802) Watermark triggered operators
cannot progress with cyclic flows
[ https://issues.apache.org/jira/browse/FLINK-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941348#comment-14941348 ]
ASF GitHub Bot commented on FLINK-2802:
---------------------------------------
GitHub user gyfora opened a pull request:
https://github.com/apache/flink/pull/1216
[FLINK-2802] [streaming] Remove cyclic watermark dependencies for iterations
This PR contains a simple change so that if timestamps are enabled, iteration sources automatically emit a `Long.MAX_VALUE` watermark.
While this will not help making event/processing time windows consistent across cycles (which is inherently problematic due to the cyclic time dependency), at least these programs will not deadlock.
I still need to add a test to validate the behaviour.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gyfora/flink watermark
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1216.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1216
----
commit 3671742d99fe9afdb71a00ed16f5ec47f099b0bb
Author: Gyula Fora <gy...@apache.org>
Date: 2015-10-02T16:26:32Z
[FLINK-2802] [streaming] Remove cyclic watermark dependencies for iterations
----
> Watermark triggered operators cannot progress with cyclic flows
> ---------------------------------------------------------------
>
> Key: FLINK-2802
> URL: https://issues.apache.org/jira/browse/FLINK-2802
> Project: Flink
> Issue Type: Bug
> Components: Streaming
> Reporter: Gyula Fora
> Priority: Blocker
>
> The problem is that we can easily create a cyclic watermark (time) dependency in the stream graph which will result in a deadlock for watermark triggered operators such as the `WindowOperator`.
> A solution to this could be to emit a Long.MAX_VALUE watermark from the iteration sources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)