You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Michał Stępień <ms...@freeportmetrics.com> on 2018/01/12 11:57:49 UTC

Ordering in ProcessFunction after emitting multiple events from WindowFunction

Hello,

I have dev related question.

How will Flink behave when processing events in WindowFunciton (version 1.3
or 1.4 including ProcessWindowFuncton) produces multiple events? Does Flink
guarantee that ProcessFunction will process them in order after window
outputs them in order into Collector for given key? Moreover, I understand
that Flink won't process next event in ProcessFunction as long as it won't
finish previous one for given key.

Code:

env.addSource(eventSource)
    .assignTimestampsAndWatermarks(timestampsAndWatermarksGenerator)
    .keyBy(FLINK_GROUPING_FIELD_NAME)

.window(TumblingEventTimeWindows.of(Time.milliseconds((WINDOW_DURATION_MILLIS))))
    .apply(windowFunction)
    .keyBy(FLINK_GROUPING_FIELD_NAME)
    .process(processFunction)
    .addSink(sinkFunction);

-- 
Pozdrawiam/Regards,
Michał

*Michał Stępień*
Developer
*freeportmetrics.com <http://freeportmetrics.com/>*  |  +48 22 253 25 13
<%2B48%20660%20507%20111>

Re: Ordering in ProcessFunction after emitting multiple events from WindowFunction

Posted by Aljoscha Krettek <al...@apache.org>.
Hi,

I think if you're keying by the same key the order should be preserved. If not, then events from different upstream operations can arrive in arbitrary order.

Best,
Aljoscha

> On 12. Jan 2018, at 12:57, Michał Stępień <ms...@freeportmetrics.com> wrote:
> 
> Hello,
> 
> I have dev related question.
> 
> How will Flink behave when processing events in WindowFunciton (version 1.3
> or 1.4 including ProcessWindowFuncton) produces multiple events? Does Flink
> guarantee that ProcessFunction will process them in order after window
> outputs them in order into Collector for given key? Moreover, I understand
> that Flink won't process next event in ProcessFunction as long as it won't
> finish previous one for given key.
> 
> Code:
> 
> env.addSource(eventSource)
>    .assignTimestampsAndWatermarks(timestampsAndWatermarksGenerator)
>    .keyBy(FLINK_GROUPING_FIELD_NAME)
> 
> .window(TumblingEventTimeWindows.of(Time.milliseconds((WINDOW_DURATION_MILLIS))))
>    .apply(windowFunction)
>    .keyBy(FLINK_GROUPING_FIELD_NAME)
>    .process(processFunction)
>    .addSink(sinkFunction);
> 
> -- 
> Pozdrawiam/Regards,
> Michał
> 
> *Michał Stępień*
> Developer
> *freeportmetrics.com <http://freeportmetrics.com/>*  |  +48 22 253 25 13
> <%2B48%20660%20507%20111>