You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by aljoscha <gi...@git.apache.org> on 2017/07/12 13:43:09 UTC
[GitHub] beam pull request #3550: [BEAM-2571] CombineTest.testSlidingWindowsCombineWi...
GitHub user aljoscha opened a pull request:
https://github.com/apache/beam/pull/3550
[BEAM-2571] CombineTest.testSlidingWindowsCombineWithContext
R: @kennknowles @JingsongLi
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aljoscha/beam jira-2571-fix-flink-watermark-issues
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/3550.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3550
----
commit dee393b2a4f287509a0af5b44841216804cae3b2
Author: Aljoscha Krettek <al...@gmail.com>
Date: 2017-07-12T12:35:58Z
[BEAM-2571] Add check for output watermark in DoFnOperator on close()
commit 39e47d973a243dfdf154b3be3e234e7bc598d4c5
Author: Aljoscha Krettek <al...@gmail.com>
Date: 2017-07-12T12:39:58Z
[BEAM-2571] Clarify pushedback variable name in DoFnOperator
commit 31791beca9d58a24cf0d851d137a1c1526221c32
Author: Aljoscha Krettek <al...@gmail.com>
Date: 2017-07-12T12:42:37Z
[BEAM-2571] Change DoFnOperator to use Long.MAX_VALUE as max watermark
This is in line with what Flink does and what BoundesSourceWrapper and
UnboundedSourceWrapper do.
commit d0aee3d08e44de6587daa209592b766aced810bf
Author: Aljoscha Krettek <al...@gmail.com>
Date: 2017-07-12T13:38:06Z
[BEAM-2571] Respect watermark contract in Flink DoFnOperator
In Flink, a watermark T specifies that there will be no elements with a
timestamp <= T in the future. In Beam, a watermark T specifies that
there will not be element with a timestamp < T in the future. This leads
to problems when the watermark is exactly "on the timer timestamp", most
prominently, this happened with Triggers, where Flink would fire the
Trigger too early and the Trigger would determine (based on the
watermark) that it is not yet time to fire the window while Flink
thought it was time.
This also adds a test that specifially tests the edge case.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] beam pull request #3550: [BEAM-2571] CombineTest.testSlidingWindowsCombineWi...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha closed the pull request at:
https://github.com/apache/beam/pull/3550
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---