You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by "Aggarwal, Ajay" <Aj...@netapp.com> on 2019/03/04 21:36:35 UTC

Broadcast state with WindowedStream

Is it possible to use broadcast state with windowing? My job looks like below

                                inputStream
                                .keyBy("some-key")
                                .window(TumblingEventTimeWindows.of(Time.seconds(Properties.WINDOW_SIZE)))
                                .process(new MyProcessWindowFunction());

I wanted to introduce broadcast state that MyProcessWindowFunction can make use of.  The example here (https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_state.html) covers connecting a Keyed and non-Keyed stream with the BroadcastStream. But in my case I need to connect WindowedStream with BroadcastStream, which isn’t seem to be supported.

Ajay

Re: Broadcast state with WindowedStream

Posted by "Aggarwal, Ajay" <Aj...@netapp.com>.

Still looking for ideas as to how I can use broadcast state in my use case.

From: "Aggarwal, Ajay" <Aj...@netapp.com>
Date: Monday, March 4, 2019 at 4:52 PM
To: "user@flink.apache.org" <us...@flink.apache.org>
Subject: Re: Broadcast state with WindowedStream

It sort of makes sense that broadcast state is not available with WindowedStream. But if I need some dynamic global state in MyProcessWindowFunction  what are my options?

Ajay

From: "Aggarwal, Ajay" <Aj...@netapp.com>
Date: Monday, March 4, 2019 at 4:36 PM
To: "user@flink.apache.org" <us...@flink.apache.org>
Subject: Broadcast state with WindowedStream

Is it possible to use broadcast state with windowing? My job looks like below

                                inputStream
                                .keyBy("some-key")
                                .window(TumblingEventTimeWindows.of(Time.seconds(Properties.WINDOW_SIZE)))
                                .process(new MyProcessWindowFunction());

I wanted to introduce broadcast state that MyProcessWindowFunction can make use of.  The example here (https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_state.html) covers connecting a Keyed and non-Keyed stream with the BroadcastStream. But in my case I need to connect WindowedStream with BroadcastStream, which isn’t seem to be supported.

Ajay

Re: Broadcast state with WindowedStream

Posted by "Aggarwal, Ajay" <Aj...@netapp.com>.

It sort of makes sense that broadcast state is not available with WindowedStream. But if I need some dynamic global state in MyProcessWindowFunction  what are my options?

Ajay

From: "Aggarwal, Ajay" <Aj...@netapp.com>
Date: Monday, March 4, 2019 at 4:36 PM
To: "user@flink.apache.org" <us...@flink.apache.org>
Subject: Broadcast state with WindowedStream

Is it possible to use broadcast state with windowing? My job looks like below

                                inputStream
                                .keyBy("some-key")
                                .window(TumblingEventTimeWindows.of(Time.seconds(Properties.WINDOW_SIZE)))
                                .process(new MyProcessWindowFunction());

I wanted to introduce broadcast state that MyProcessWindowFunction can make use of.  The example here (https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_state.html) covers connecting a Keyed and non-Keyed stream with the BroadcastStream. But in my case I need to connect WindowedStream with BroadcastStream, which isn’t seem to be supported.

Ajay