You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Aljoscha Krettek <al...@apache.org> on 2018/01/02 13:45:49 UTC

Re: Dynamic SessionWindow gaps

Hi,

This sounds good! Could you please open a Jira Issue for this and then open a PR against that Jira issue?

Btw, I would suggest to implement this as a new type of assigner, something like DynamicSessionWindows.

Best,
Aljoscha

> On 29. Dec 2017, at 20:54, Dyana Rose <dy...@salecycle.com> wrote:
> 
> I have a use case for non-static Session Window gaps.
> 
> For example, given a stream of IoT events, each device type could have a
> different gap, and that gap could change while sessions are in flight.
> 
> I didn't want to have to run a stream processor for each potential gap
> length, not to mention the headache of dealing with changing gaps, so I've
> implemented a version of SessionWindows that has one major change; in the
> assignWindows method it passes the element to a method to extract the
> correct sessionTimeout. (current Flink method for reference:
> https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/assigners/EventTimeSessionWindows.java#L59
> )
> 
> Preliminary tests show this working as required and I can't be the only
> person with this type of use case for session windows.
> 
> Will an issue and PR to add this functionality to the SessionWindow classes
> be welcome?
> 
> Dyana


Re: Dynamic SessionWindow gaps

Posted by "dyana.rose@salecycle.com" <dy...@salecycle.com>.
I've put up some implementation information and a link to the working branch for comment while I get the documentation written up: 

https://issues.apache.org/jira/browse/FLINK-8384

Thanks,
Dyana

On 2018-01-02 13:54, jincheng sun <su...@gmail.com> wrote: 
> Hi, Dyana dynamic gap is very useful feature, Agree with the suggestion of
> Aljoscha, looking forward the PR.
> 
> Best, Jincheng
> 
> 2018-01-02 21:45 GMT+08:00 Aljoscha Krettek <al...@apache.org>:
> 
> > Hi,
> >
> > This sounds good! Could you please open a Jira Issue for this and then
> > open a PR against that Jira issue?
> >
> > Btw, I would suggest to implement this as a new type of assigner,
> > something like DynamicSessionWindows.
> >
> > Best,
> > Aljoscha
> >
> > > On 29. Dec 2017, at 20:54, Dyana Rose <dy...@salecycle.com> wrote:
> > >
> > > I have a use case for non-static Session Window gaps.
> > >
> > > For example, given a stream of IoT events, each device type could have a
> > > different gap, and that gap could change while sessions are in flight.
> > >
> > > I didn't want to have to run a stream processor for each potential gap
> > > length, not to mention the headache of dealing with changing gaps, so
> > I've
> > > implemented a version of SessionWindows that has one major change; in the
> > > assignWindows method it passes the element to a method to extract the
> > > correct sessionTimeout. (current Flink method for reference:
> > > https://github.com/apache/flink/blob/master/flink-
> > streaming-java/src/main/java/org/apache/flink/streaming/
> > api/windowing/assigners/EventTimeSessionWindows.java#L59
> > > )
> > >
> > > Preliminary tests show this working as required and I can't be the only
> > > person with this type of use case for session windows.
> > >
> > > Will an issue and PR to add this functionality to the SessionWindow
> > classes
> > > be welcome?
> > >
> > > Dyana
> >
> >
> 

Re: Dynamic SessionWindow gaps

Posted by jincheng sun <su...@gmail.com>.
Hi, Dyana dynamic gap is very useful feature, Agree with the suggestion of
Aljoscha, looking forward the PR.

Best, Jincheng

2018-01-02 21:45 GMT+08:00 Aljoscha Krettek <al...@apache.org>:

> Hi,
>
> This sounds good! Could you please open a Jira Issue for this and then
> open a PR against that Jira issue?
>
> Btw, I would suggest to implement this as a new type of assigner,
> something like DynamicSessionWindows.
>
> Best,
> Aljoscha
>
> > On 29. Dec 2017, at 20:54, Dyana Rose <dy...@salecycle.com> wrote:
> >
> > I have a use case for non-static Session Window gaps.
> >
> > For example, given a stream of IoT events, each device type could have a
> > different gap, and that gap could change while sessions are in flight.
> >
> > I didn't want to have to run a stream processor for each potential gap
> > length, not to mention the headache of dealing with changing gaps, so
> I've
> > implemented a version of SessionWindows that has one major change; in the
> > assignWindows method it passes the element to a method to extract the
> > correct sessionTimeout. (current Flink method for reference:
> > https://github.com/apache/flink/blob/master/flink-
> streaming-java/src/main/java/org/apache/flink/streaming/
> api/windowing/assigners/EventTimeSessionWindows.java#L59
> > )
> >
> > Preliminary tests show this working as required and I can't be the only
> > person with this type of use case for session windows.
> >
> > Will an issue and PR to add this functionality to the SessionWindow
> classes
> > be welcome?
> >
> > Dyana
>
>