[jira] [Commented] (FLINK-12855) Stagger TumblingProcessingTimeWindow processing to distribute workload

["Jark Wu (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-12855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134826#comment-17134826 ] 

Jark Wu commented on FLINK-12855:
---------------------------------

I like this feature! I'm thinking that is this borrowed some ideas from Amazon Kinesis [Stagger Windows|https://docs.aws.amazon.com/kinesisanalytics/latest/dev/stagger-window-concepts.html]? Did you consider to make it as a new window type instead of a parameter of TUMBLE window? 

I have this question, because Tumbling window in some places called Fixed window (e.g. Apache Beam),  the window start and end should be fixed? A new type of window for this may help to distinguish the use cases. 

Btw, I'm looking forward to also support event time windows for this. 

> Stagger TumblingProcessingTimeWindow processing to distribute workload
> ----------------------------------------------------------------------
>
>                 Key: FLINK-12855
>                 URL: https://issues.apache.org/jira/browse/FLINK-12855
>             Project: Flink
>          Issue Type: New Feature
>          Components: API / DataStream
>            Reporter: Teng Hu
>            Assignee: Teng Hu
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.12.0
>
>         Attachments: stagger_window.png, stagger_window_delay.png, stagger_window_throughput.png
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Flink natively triggers all panes belonging to same window at the same time. In other words, all panes are aligned and their triggers all fire simultaneously, causing the thundering herd effect.
> This new feature provides the option that panes could be staggered across partitioned streams, so that their workloads are distributed.
> Attachment: proof of concept working



--
This message was sent by Atlassian Jira
(v8.3.4#803005)