You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Jin Shen (JIRA)" <ji...@apache.org> on 2019/04/18 17:05:00 UTC
[jira] [Updated] (BEAM-7105) AfterProcessingTime trigger not
reliable; only fires on double numbers' event
[ https://issues.apache.org/jira/browse/BEAM-7105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jin Shen updated BEAM-7105:
---------------------------
Description:
we use following statement in sql to configure the window trigger:
sometimes it doesn't fire but the data is recorded in the window. and event come afterwards, the trigger fires with all the data together.
I uploaded a video of screenshot on google drive. It will show the behavior of this.
[https://drive.google.com/file/d/1wOo2s4ifAdwdAglbzOb0jEYZszsQPFjx/view?usp=sharing]
{code:java}
org.apache.beam.sdk.transforms.windowing.Window.configure()
.triggering(org.apache.beam.sdk.transforms.windowing.AfterWatermark.pastEndOfWindow()
.withEarlyFirings(org.apache.beam.sdk.transforms.windowing.AfterProcessingTime.pastFirstElementInPane().plusDelayOf(org.joda.time.Duration.millis(0)))
.withLateFirings(org.apache.beam.sdk.transforms.windowing.AfterProcessingTime.pastFirstElementInPane().plusDelayOf(org.joda.time.Duration.millis(20000))))
.withAllowedLateness(org.joda.time.Duration.millis(3600000), org.apache.beam.sdk.transforms.windowing.Window.ClosingBehavior.valueOf("FIRE_IF_NON_EMPTY"))
.withOnTimeBehavior(org.apache.beam.sdk.transforms.windowing.Window.OnTimeBehavior.valueOf("FIRE_IF_NON_EMPTY")).accumulatingFiredPanes()
{code}
was:
we use following statement in sql to configure the window trigger:
sometimes it doesn't fire but the data is recorded in the window. and event come afterwards, the trigger fires with all the data together.
I uploaded a video of screenshot on google drive. It will show the behavior of this.
[https://drive.google.com/file/d/1wOo2s4ifAdwdAglbzOb0jEYZszsQPFjx/view?usp=sharing]
{code:sql}
String.format( "T(org.apache.beam.sdk.transforms.windowing.Window).configure()" + ".triggering(T(org.apache.beam.sdk.transforms.windowing.AfterWatermark).pastEndOfWindow()" + ".withEarlyFirings(T(org.apache.beam.sdk.transforms.windowing.AfterProcessingTime).pastFirstElementInPane().plusDelayOf(T(org.joda.time.Duration).millis(0)))" + ".withLateFirings(T(org.apache.beam.sdk.transforms.windowing.AfterProcessingTime).pastFirstElementInPane().plusDelayOf(T(org.joda.time.Duration).millis(20000))))" + ".withAllowedLateness(T(org.joda.time.Duration).millis(3600000), T(org.apache.beam.sdk.transforms.windowing.Window.ClosingBehavior).valueOf(\"FIRE_IF_NON_EMPTY\"))" + ".withOnTimeBehavior(T(org.apache.beam.sdk.transforms.windowing.Window.OnTimeBehavior).valueOf(\"FIRE_IF_NON_EMPTY\"))" + ".accumulatingFiredPanes()",
{code}
> AfterProcessingTime trigger not reliable; only fires on double numbers' event
> -----------------------------------------------------------------------------
>
> Key: BEAM-7105
> URL: https://issues.apache.org/jira/browse/BEAM-7105
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Affects Versions: 2.6.0
> Reporter: Jin Shen
> Priority: Major
>
> we use following statement in sql to configure the window trigger:
> sometimes it doesn't fire but the data is recorded in the window. and event come afterwards, the trigger fires with all the data together.
> I uploaded a video of screenshot on google drive. It will show the behavior of this.
> [https://drive.google.com/file/d/1wOo2s4ifAdwdAglbzOb0jEYZszsQPFjx/view?usp=sharing]
>
> {code:java}
> org.apache.beam.sdk.transforms.windowing.Window.configure()
> .triggering(org.apache.beam.sdk.transforms.windowing.AfterWatermark.pastEndOfWindow()
> .withEarlyFirings(org.apache.beam.sdk.transforms.windowing.AfterProcessingTime.pastFirstElementInPane().plusDelayOf(org.joda.time.Duration.millis(0)))
> .withLateFirings(org.apache.beam.sdk.transforms.windowing.AfterProcessingTime.pastFirstElementInPane().plusDelayOf(org.joda.time.Duration.millis(20000))))
> .withAllowedLateness(org.joda.time.Duration.millis(3600000), org.apache.beam.sdk.transforms.windowing.Window.ClosingBehavior.valueOf("FIRE_IF_NON_EMPTY"))
> .withOnTimeBehavior(org.apache.beam.sdk.transforms.windowing.Window.OnTimeBehavior.valueOf("FIRE_IF_NON_EMPTY")).accumulatingFiredPanes()
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)