Tumbling Count and Time-Based WindowedBolt

[Cody Lee <co...@wellaware.us>]

We noticed some behavior where we configured a BaseWindowedBolt as a tumbling window with both Count and Duration based policies and expected a Count + Time-Based tumbling window, but realized this only gives a Count base tumbling window.  This seems like it could be a useful feature to prevent events from potentially sitting in a count policy for an indeterminate amount of time before the max pending count is reached.  [feature request if this doesn’t already exist (couldn’t find it in issues.apache.org after some searching)]

It seems like one would be able to implement a Tumbling Count and Time Based Window with the BaseWindowedBolt configuring it as a non-tumbling window with Duration and Count settings.  And utilizing the “New” list from the window only, or is this assumption wrong?

Cody

Re: Tumbling Count and Time-Based WindowedBolt

[Arun Mahadevan <ar...@apache.org>]

Hi Cody,

Tumbling window can be either count or time based, not both. 

I think what you want is a count based window with a time based sliding interval. The window will activate every sliding interval and will give you the last “count” events. You can use the “getNew” if you want only the new events since the last slide.

- Arun


From:  Cody Lee <co...@wellaware.us>
Reply-To:  "user@storm.apache.org" <us...@storm.apache.org>
Date:  Tuesday, June 7, 2016 at 3:16 AM
To:  "user@storm.apache.org" <us...@storm.apache.org>
Subject:  Tumbling Count and Time-Based WindowedBolt

We noticed some behavior where we configured a BaseWindowedBolt as a tumbling window with both Count and Duration based policies and expected a Count + Time-Based tumbling window, but realized this only gives a Count base tumbling window.  This seems like it could be a useful feature to prevent events from potentially sitting in a count policy for an indeterminate amount of time before the max pending count is reached.  [feature request if this doesn’t already exist (couldn’t find it in issues.apache.org after some searching)]

 

It seems like one would be able to implement a Tumbling Count and Time Based Window with the BaseWindowedBolt configuring it as a non-tumbling window with Duration and Count settings.  And utilizing the “New” list from the window only, or is this assumption wrong? 

 

Cody