You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Navneeth Krishnan <re...@gmail.com> on 2021/01/04 00:05:17 UTC

Tumbling Time Window

Hello All,

First of all Happy New Year!! Thanks for the excellent community support.

I have a job which requires a 2 seconds tumbling time window per key, For
each user we wait for 2 seconds to collect enough data and proceed to
 further processing. My question is should I use the regular DSL windowing
or write a custom process function which does the windowing. I have heard
that the DSL window has more overhead versus the custom window function.

What do you guys suggest and can someone provide an example of custom
window function per key. Also given the window time is very less (2 secs)
would there be more overhead in firing so many timers for each key?

Thanks!

Regards,
Navneeth

Re: Tumbling Time Window

Posted by David Anderson <da...@alpinegizmo.com>.
For straightforward tumbling windows, the regular DSL windowing performs
noticeably better than a custom process function because it takes advantage
of an internal API to avoid some serialization overhead.

There's a simple example of a ProcessWindowFunction in [1], and an example
of using a KeyedProcessFunction to do windowing in [2].

Best,
David

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.12/learn-flink/streaming_analytics.html#window-functions
[2]
https://ci.apache.org/projects/flink/flink-docs-release-1.12/learn-flink/event_driven.html#example


On Mon, Jan 4, 2021 at 1:05 AM Navneeth Krishnan <re...@gmail.com>
wrote:

> Hello All,
>
> First of all Happy New Year!! Thanks for the excellent community support.
>
> I have a job which requires a 2 seconds tumbling time window per key, For
> each user we wait for 2 seconds to collect enough data and proceed to
>  further processing. My question is should I use the regular DSL windowing
> or write a custom process function which does the windowing. I have heard
> that the DSL window has more overhead versus the custom window function.
>
> What do you guys suggest and can someone provide an example of custom
> window function per key. Also given the window time is very less (2 secs)
> would there be more overhead in firing so many timers for each key?
>
> Thanks!
>
> Regards,
> Navneeth
>

Re: Tumbling Time Window

Posted by Yun Gao <yu...@aliyun.com>.
Hi Navneeth

    For me I think you may start with using the window function and an example for the custom window function could be found in [1]. From the description I think it should be a standard Tumbling window, if implementing with the customized process function, it would end up have a similar functionality with the current window operator. 


Best,
 Yun


 [1] https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/stream/operators/windows.html#processwindowfunction

------------------------------------------------------------------
Sender:Navneeth Krishnan<re...@gmail.com>
Date:2021/01/04 08:05:17
Recipient:user<us...@flink.apache.org>
Theme:Tumbling Time Window

Hello All,

First of all Happy New Year!! Thanks for the excellent community support.

I have a job which requires a 2 seconds tumbling time window per key, For each user we wait for 2 seconds to collect enough data and proceed to
 further processing. My question is should I use the regular DSL windowing or write a custom process function which does the windowing. I have heard that the DSL window has more overhead versus the custom window function.

What do you guys suggest and can someone provide an example of custom window function per key. Also given the window time is very less (2 secs) would there be more overhead in firing so many timers for each key?

Thanks!

Regards,
Navneeth