You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/01/05 13:59:00 UTC
[jira] [Commented] (BEAM-2140) Fix SplittableDoFn ValidatesRunner
tests in FlinkRunner
[ https://issues.apache.org/jira/browse/BEAM-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16313138#comment-16313138 ]
ASF GitHub Bot commented on BEAM-2140:
--------------------------------------
aljoscha opened a new pull request #4348: [BEAM-2140] Fix SplittableDoFn ValidatesRunner tests in Flink Runner
URL: https://github.com/apache/beam/pull/4348
I managed to sneak a change into Flink 1.4.0 that allows blocking on pending process-time timers. Now we can finally fix those tests for the Flink Runner. 😃
Most of these changes are pretty straightforward, the only controversial change is using the output WM instead of the input WM for determining lateness in `StateFulDoFnRunner`. This is there to make sure that timers (which are elements to the `ProcessFn`) are not dropped when the input watermark goes to `+Inf`. I tried changing `FlinkTimerInternals.currentInputWatermarkTime()` to be held back by the watermark hold but that doesn't work because windows and stuff will then never fire.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Fix SplittableDoFn ValidatesRunner tests in FlinkRunner
> -------------------------------------------------------
>
> Key: BEAM-2140
> URL: https://issues.apache.org/jira/browse/BEAM-2140
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Reporter: Aljoscha Krettek
> Assignee: Aljoscha Krettek
>
> As discovered as part of BEAM-1763, there is a failing SDF test. We disabled the tests to unblock the open PR for BEAM-1763.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)