You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Juho Autio <ju...@rovio.com> on 2018/04/11 09:12:11 UTC

Allowed lateness + side output late data, what's included?

If I use a non-zero value for allowedLateness and also sideOutputLateData,
does the late data output contain also the events that were triggered in
the bounds of allowed lateness? By looking at the docs I can't be sure
which way it is.

Code example:

.timeWindow(Time.days(1))
.allowedLateness(Time.minutes(1))
.sideOutputLateData(lateDataTag)

Also I know that data that arrives more than 1 minute late is written to
the late data side output.

If some data arrives 0-1 minute late, I know that the time window triggers
again with that data added. Is that data also written to the late data side
output or not?

Thanks!

Re: Allowed lateness + side output late data, what's included?

Posted by Juho Autio <ju...@rovio.com>.

Thanks!

On Wed, Apr 11, 2018 at 12:59 PM, Chesnay Schepler <ch...@apache.org>
wrote:

> Data that arrives within the allowed lateness should not be written to the
> side output.
>
>
> On 11.04.2018 11:12, Juho Autio wrote:
>
> If I use a non-zero value for allowedLateness and also sideOutputLateData,
> does the late data output contain also the events that were triggered in
> the bounds of allowed lateness? By looking at the docs I can't be sure
> which way it is.
>
> Code example:
>
> .timeWindow(Time.days(1))
> .allowedLateness(Time.minutes(1))
> .sideOutputLateData(lateDataTag)
>
> Also I know that data that arrives more than 1 minute late is written to
> the late data side output.
>
> If some data arrives 0-1 minute late, I know that the time window triggers
> again with that data added. Is that data also written to the late data side
> output or not?
>
> Thanks!
>
>
>

Re: Allowed lateness + side output late data, what's included?

Posted by Chesnay Schepler <ch...@apache.org>.

Data that arrives within the allowed lateness should not be written to 
the side output.

On 11.04.2018 11:12, Juho Autio wrote:
> If I use a non-zero value for allowedLateness and also 
> sideOutputLateData, does the late data output contain also the events 
> that were triggered in the bounds of allowed lateness? By looking at 
> the docs I can't be sure which way it is.
>
> Code example:
>
> .timeWindow(Time.days(1))
> .allowedLateness(Time.minutes(1))
> .sideOutputLateData(lateDataTag)
>
> Also I know that data that arrives more than 1 minute late is written 
> to the late data side output.
>
> If some data arrives 0-1 minute late, I know that the time window 
> triggers again with that data added. Is that data also written to the 
> late data side output or not?
>
> Thanks!