clear method on Window Trigger

[Soheil Pourbafrani <so...@gmail.com>]

Hi,

Can someone elaborate on when the clear method on class Trigger will be
called and what is the duty of that? Also, I don't know what is the benefit
of FIRE_AND_PURGE against FIRE and it's use case.

For example, in a scenario, if we have a count of 3 Window that also will
trigger after a timeout if we FIRE window on timeout and after a while, a
new data will be added to the data, will window be triggered for all data
or not just for new late data? I guess in such such case we should use the
FIRE_AND_PURGE.

Re: clear method on Window Trigger

[vino yang <ya...@gmail.com>]

Hi Soheil,

Did you read the documentation about Flink Window/Trigger [1]?

FIRE_AND_PURGE usually used to implement the count window. Flink provide a
PurgingTrigger as a warper for other trigger to make those triggers can be
purge. One of this class use case is count window[2][3].

About your example, you mean the window will triggered by count and time?
if it is triggered by count condition we could use FIRE_AND_PURGE.

[1]:
https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/stream/operators/windows.html#fire-and-purge
[2]:
https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/datastream/DataStream.java#L765
[3]:
https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/datastream/KeyedStream.java#L628

Thanks, vino.

2018-07-17 22:44 GMT+08:00 Soheil Pourbafrani <so...@gmail.com>:

> Hi,
>
> Can someone elaborate on when the clear method on class Trigger will be
> called and what is the duty of that? Also, I don't know what is the benefit
> of FIRE_AND_PURGE against FIRE and it's use case.
>
> For example, in a scenario, if we have a count of 3 Window that also will
> trigger after a timeout if we FIRE window on timeout and after a while, a
> new data will be added to the data, will window be triggered for all data
> or not just for new late data? I guess in such such case we should use the
> FIRE_AND_PURGE.
>
>

Re: clear method on Window Trigger

[Hequn Cheng <ch...@gmail.com>]

Hi Soheil,

The clear() method performs any action needed upon removal of the
corresponding window. This is called when a window is purged. The
differences between FIRE and FIRE_AND_PURGE is FIRE only trigger the
computation while FIRE_AND_PURGE trigger the computation and clear the
elements in the window afterwards.

As for your example, if you use FIRE, when the new data comes, the window
is triggered for all data. Besides, some window functions can incrementally
aggregate the elements for each window as they arrive, for example
AggregateFunction and FoldFunction.

More details can be found here[1].

Best, Hequn

[1]
https://ci.apache.org/projects/flink/flink-docs-master/dev/stream/operators/windows.html#windows



On Tue, Jul 17, 2018 at 10:44 PM, Soheil Pourbafrani <so...@gmail.com>
wrote:

> Hi,
>
> Can someone elaborate on when the clear method on class Trigger will be
> called and what is the duty of that? Also, I don't know what is the benefit
> of FIRE_AND_PURGE against FIRE and it's use case.
>
> For example, in a scenario, if we have a count of 3 Window that also will
> trigger after a timeout if we FIRE window on timeout and after a while, a
> new data will be added to the data, will window be triggered for all data
> or not just for new late data? I guess in such such case we should use the
> FIRE_AND_PURGE.
>
>