You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Nico <ni...@gmail.com> on 2017/03/15 17:57:55 UTC

Late Events with BoundedOutOfOrdernessTimestampExtractor and allowed lateness

Hi,

I struggle a bit to understand the difference between
BoundedOutOfOrdernessTimestampExtractor and the allowed lateness function
of a window...

As I understand it, when I use BoundedOutOfOrdernessTimestampExtractor the
watermark is lagging behind the real event time of the stream with
maxOutOfOrderness.
So for the window trigger late events won't be late as the watermark is
lagging.

In Contrast, allowed lateness will wait after the watermark passed. So
using allowed lateness a window could fire twice - after the watermark and
the arrival of late events.

Is that the only difference? What would happen if I use both? Do I have a
double delay?

Does a best practice exist when to BoundedOutOfOrdernessTimestampExtractor
or allowed latesness?

Best regards,
Nico

Re: Late Events with BoundedOutOfOrdernessTimestampExtractor and allowed lateness

Posted by Nico <ni...@gmail.com>.

Hi Yassine,

I forgot to say thank you for poiting to the post. It was really useful.

Best,
Nico :)

2017-03-15 20:19 GMT+01:00 Yassine MARZOUGUI <y....@mindlytix.com>:

> Hi Nico,
>
> You might check Fabian's answer on a similar question I posted previousely
> on the mailing list, it can be helpful :
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
> BoundedOutOfOrdernessTimestampExtractor-and-allowedlateness-td9583.html
>
> Best,
> Yassine
>
> On Mar 15, 2017 18:58, "Nico" <ni...@gmail.com> wrote:
>
>> Hi,
>>
>> I struggle a bit to understand the difference between
>> BoundedOutOfOrdernessTimestampExtractor and the allowed lateness
>> function of a window...
>>
>> As I understand it, when I use BoundedOutOfOrdernessTimestampExtractor
>> the watermark is lagging behind the real event time of the stream with maxOutOfOrderness.
>> So for the window trigger late events won't be late as the watermark is
>> lagging.
>>
>> In Contrast, allowed lateness will wait after the watermark passed. So
>> using allowed lateness a window could fire twice - after the watermark and
>> the arrival of late events.
>>
>> Is that the only difference? What would happen if I use both? Do I have a
>> double delay?
>>
>> Does a best practice exist when to BoundedOutOfOrdernessTimestampExtractor
>> or allowed latesness?
>>
>> Best regards,
>> Nico
>>
>

Re: Late Events with BoundedOutOfOrdernessTimestampExtractor and allowed lateness

Posted by Yassine MARZOUGUI <y....@mindlytix.com>.

Hi Nico,

You might check Fabian's answer on a similar question I posted previousely
on the mailing list, it can be helpful :
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/BoundedOutOfOrdernessTimestampExtractor-and-allowedlateness-td9583.html

Best,
Yassine

On Mar 15, 2017 18:58, "Nico" <ni...@gmail.com> wrote:

> Hi,
>
> I struggle a bit to understand the difference between
> BoundedOutOfOrdernessTimestampExtractor and the allowed lateness function
> of a window...
>
> As I understand it, when I use BoundedOutOfOrdernessTimestampExtractor
> the watermark is lagging behind the real event time of the stream with maxOutOfOrderness.
> So for the window trigger late events won't be late as the watermark is
> lagging.
>
> In Contrast, allowed lateness will wait after the watermark passed. So
> using allowed lateness a window could fire twice - after the watermark and
> the arrival of late events.
>
> Is that the only difference? What would happen if I use both? Do I have a
> double delay?
>
> Does a best practice exist when to BoundedOutOfOrdernessTimestampExtractor
> or allowed latesness?
>
> Best regards,
> Nico
>