Order of Execution in KeyedBroadcastProcessFunction

[Anil K <se...@gmail.com>]

Hi,

I am new to flink. I am trying to write a job that updates the Keyed State
when a Broadcast Message is received in KeyedBroadcastProcessFunction.
I was wondering will the *ctx.applyToKeyedState* in the
processBroadCastElement will get completed before further messages are
processed in the *processElement* function?
Can someone explain or point me to some docs on how the synchronisation of
Broadcasted and non Broadcasted messages works
in KeyedBroadcastProcessFunction?

Kind regards
Anil

Re: Order of Execution in KeyedBroadcastProcessFunction

[liu ron <ro...@gmail.com>]

Hi,

The KeyedBroadcastProcessFunction#processBroadcastElement is called in
outer CoBroadcastWithKeyedOperator[1], CoBroadcastWithKeyedOperator is a
two-input operator. As David says, this operator is run in a single thread,
so the processBroadcastElement
method will run to completion before any further messages are processed in
the processElement method.

[1]
https://github.com/apache/flink/blob/e7eeea033a68e1ff6bf82132b5a59eb0a5a2d0ed/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/co/CoBroadcastWithKeyedOperator.java#L134

Best,
Ron

David Anderson <da...@apache.org> 于2023年9月11日周一 13:11写道：

> In Flink, all user functions, including KeyedBroadcastProcessFunction,
> are (effectively) single threaded, so the processBroadcastElement
> method will run to completion before any further messages are
> processed in the processElement method. (I said "effectively" because
> in the case of processing time timers, Flink takes care to synchronize
> the onTimer callback for you.)
>
> On Thu, Sep 7, 2023 at 6:32 AM Anil K <se...@gmail.com> wrote:
> >
> > Hi,
> >
> > I am new to flink. I am trying to write a job that updates the Keyed
> State when a Broadcast Message is received in KeyedBroadcastProcessFunction.
> > I was wondering will the ctx.applyToKeyedState in the
> processBroadCastElement will get completed before further messages are
> processed in the processElement function?
> > Can someone explain or point me to some docs on how the synchronisation
> of Broadcasted and non Broadcasted messages works in
> KeyedBroadcastProcessFunction?
> >
> > Kind regards
> > Anil
>

Re: Order of Execution in KeyedBroadcastProcessFunction

[David Anderson <da...@apache.org>]

In Flink, all user functions, including KeyedBroadcastProcessFunction,
are (effectively) single threaded, so the processBroadcastElement
method will run to completion before any further messages are
processed in the processElement method. (I said "effectively" because
in the case of processing time timers, Flink takes care to synchronize
the onTimer callback for you.)

On Thu, Sep 7, 2023 at 6:32 AM Anil K <se...@gmail.com> wrote:
>
> Hi,
>
> I am new to flink. I am trying to write a job that updates the Keyed State when a Broadcast Message is received in KeyedBroadcastProcessFunction.
> I was wondering will the ctx.applyToKeyedState in the processBroadCastElement will get completed before further messages are processed in the processElement function?
> Can someone explain or point me to some docs on how the synchronisation of Broadcasted and non Broadcasted messages works in KeyedBroadcastProcessFunction?
>
> Kind regards
> Anil