You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Kenneth Knowles (Jira)" <ji...@apache.org> on 2021/05/04 16:20:00 UTC
[jira] [Commented] (BEAM-9557) Error setting processing time timers
near end-of-window
[ https://issues.apache.org/jira/browse/BEAM-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17339112#comment-17339112 ]
Kenneth Knowles commented on BEAM-9557:
---------------------------------------
Just dropping in here that I have had this discussion a million times.
Cleanup timers need to fire according to the _combined_ watermark of the main input and pending timers. Timers create something like a "self-loop" where the output timestamps hold that self-loop back. It is fundamentally incorrect to fire a cleanup timer based on the main element-only watermark.
> Error setting processing time timers near end-of-window
> -------------------------------------------------------
>
> Key: BEAM-9557
> URL: https://issues.apache.org/jira/browse/BEAM-9557
> Project: Beam
> Issue Type: Bug
> Components: runner-core
> Reporter: Steve Niemitz
> Assignee: Reuven Lax
> Priority: P1
> Fix For: 2.20.0
>
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> Previously, it was possible to set a processing time timer past the end of a window, and it would simply not fire.
> However, now, this results in an error:
> {code:java}
> java.lang.IllegalArgumentException: Attempted to set event time timer that outputs for 2020-03-19T18:01:35.000Z but that is after the expiration of window 2020-03-19T17:59:59.999Z
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument(Preconditions.java:440)
> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$TimerInternalsTimer.setAndVerifyOutputTimestamp(SimpleDoFnRunner.java:1011)
> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$TimerInternalsTimer.setRelative(SimpleDoFnRunner.java:934)
> <snip>.processElement(???.scala:187)
> {code}
>
> I think the regression was introduced in commit a005fd765a762183ca88df90f261f6d4a20cf3e0. Also notice that the error message is wrong, it says that "event time timer" but the timer is in the processing time domain.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)