You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Sujit Sakre <su...@northgateps.com> on 2017/01/02 11:11:41 UTC
Hi, There is possibly an issue with EventTimeSessionWindows where a
gap is specified for considering items in the same session. Here the logic
is, if two adjacent items have a difference in event timestamps of more than
the gap then the items are considered to be in separate session. The issue
is, what happens if the gap between streaming records is never (or for a very
long time) less than the session gap. This is likely to lead to a race
condition. Is this a bug? How do we deal with this to process windows in
finite time? Please could you suggest. Thanks.
Hi,
We are using Flink 1.1.4 version.
There is possibly an issue with EventTimeSessionWindows where a gap is
specified for considering items in the same session. Here the logic is, if
two adjacent items have a difference in event timestamps of more than the
gap then the items are considered to be in separate session. The issue is,
what happens if the gap between streaming records is *never* (or for a
very long time) less than the session gap. This is likely to lead to a race
condition.
Is this a bug? How do we deal with this to process windows in finite time?
Please could you suggest.
Thanks.
*Sujit Sakre*
Senior Technical Architect
Tel: +91 22 6660 6600
Ext:
247
Direct: 6740 5247
Mobile: +91 98672 01204
www.rave-tech.com
Follow us on: Twitter <https://twitter.com/Rave_Tech> / LinkedIn
<https://in.linkedin.com/in/ravetechnologies> / YouTube
<https://www.youtube.com/channel/UCTaO1am-cm4FqnQCGdB6ExA>
Rave Technologies – A Northgate Public Services Company
<https://www.google.co.in/maps/place/Rave+Technologies/@19.0058078,72.823516,17z/data=!3m1!4b1!4m5!3m4!1s0x3bae17fcde71c3b9:0x1e2a8c0c4a075145!8m2!3d19.0058078!4d72.8257047>
Please consider the environment before printing this email
--
This email is sent on behalf of Northgate Public Services (UK) Limited and
its associated companies including Rave Technologies (India) Pvt Limited
(together "Northgate Public Services") and is strictly confidential and
intended solely for the addressee(s).
If you are not the intended recipient of this email you must: (i) not
disclose, copy or distribute its contents to any other person nor use its
contents in any way or you may be acting unlawfully; (ii) contact
Northgate Public Services immediately on +44(0)1908 264500 quoting the name
of the sender and the addressee then delete it from your system.
Northgate Public Services has taken reasonable precautions to ensure that
no viruses are contained in this email, but does not accept any
responsibility once this email has been transmitted. You should scan
attachments (if any) for viruses.
Northgate Public Services (UK) Limited, registered in England and Wales
under number 00968498 with a registered address of Peoplebuilding 2,
Peoplebuilding Estate, Maylands Avenue, Hemel Hempstead, Hertfordshire, HP2
4NN. Rave Technologies (India) Pvt Limited, registered in India under
number 117068 with a registered address of 2nd Floor, Ballard House, Adi
Marzban Marg, Ballard Estate, Mumbai, Maharashtra, India, 400001.
Re: Hi, There is possibly an issue with EventTimeSessionWindows where
a gap is specified for considering items in the same session. Here the logic
is, if two adjacent items have a difference in event timestamps of more than
the gap then the items are considered to be in separate session. The issue
is, what happens if the gap between streaming records is never (or for a very
long time) less than the session gap. This is likely to lead to a race
condition. Is this a bug? How do we deal with this to process windows in
finite time? Please could you suggest. Thanks.
Posted by Jamie Grier <ja...@data-artisans.com>.
If there is never a gap between elements larger than the session gap -- the
window never ending would be the correct behavior. So, if this is the case
with some data stream I would not suggest to use session windows at all --
or I would use a smaller session gap.
Another alternative would be to use Session Windows along with a
user-defined trigger that fires periodically whether the session has ended
or not. For example, if in the normal case the session window logic works
well but sometimes you want to force an eval in case a "natural" session is
too long.
-Jamie
On Mon, Jan 2, 2017 at 3:11 AM, Sujit Sakre <su...@northgateps.com>
wrote:
> Hi,
>
> We are using Flink 1.1.4 version.
>
>
> There is possibly an issue with EventTimeSessionWindows where a gap is
> specified for considering items in the same session. Here the logic is, if
> two adjacent items have a difference in event timestamps of more than the
> gap then the items are considered to be in separate session. The issue is,
> what happens if the gap between streaming records is *never* (or for a
> very long time) less than the session gap. This is likely to lead to a race
> condition.
>
> Is this a bug? How do we deal with this to process windows in finite time?
>
> Please could you suggest.
>
> Thanks.
>
>
> *Sujit Sakre*
>
> Senior Technical Architect
> Tel: +91 22 6660 6600
> Ext:
> 247
> Direct: 6740 5247
>
> Mobile: +91 98672 01204
>
> www.rave-tech.com
>
>
>
> Follow us on: Twitter <https://twitter.com/Rave_Tech> / LinkedIn
> <https://in.linkedin.com/in/ravetechnologies> / YouTube
> <https://www.youtube.com/channel/UCTaO1am-cm4FqnQCGdB6ExA>
>
>
>
> Rave Technologies – A Northgate Public Services Company
> <https://www.google.co.in/maps/place/Rave+Technologies/@19.0058078,72.823516,17z/data=!3m1!4b1!4m5!3m4!1s0x3bae17fcde71c3b9:0x1e2a8c0c4a075145!8m2!3d19.0058078!4d72.8257047>
>
>
>
> Please consider the environment before printing this email
>
> This email is sent on behalf of Northgate Public Services (UK) Limited and
> its associated companies including Rave Technologies (India) Pvt Limited
> (together "Northgate Public Services") and is strictly confidential and
> intended solely for the addressee(s).
> If you are not the intended recipient of this email you must: (i) not
> disclose, copy or distribute its contents to any other person nor use its
> contents in any way or you may be acting unlawfully; (ii) contact
> Northgate Public Services immediately on +44(0)1908 264500
> <+44%201908%20264500> quoting the name of the sender and the addressee
> then delete it from your system.
> Northgate Public Services has taken reasonable precautions to ensure that
> no viruses are contained in this email, but does not accept any
> responsibility once this email has been transmitted. You should scan
> attachments (if any) for viruses.
>
> Northgate Public Services (UK) Limited, registered in England and Wales
> under number 00968498 with a registered address of Peoplebuilding 2,
> Peoplebuilding Estate, Maylands Avenue, Hemel Hempstead, Hertfordshire, HP2
> 4NN. Rave Technologies (India) Pvt Limited, registered in India under
> number 117068 with a registered address of 2nd Floor, Ballard House, Adi
> Marzban Marg, Ballard Estate, Mumbai, Maharashtra, India, 400001.
>
--
Jamie Grier
data Artisans, Director of Applications Engineering
@jamiegrier <https://twitter.com/jamiegrier>
jamie@data-artisans.com