You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Jaswin Shah <ja...@outlook.com> on 2020/06/14 10:57:34 UTC
Shared state between two process functions
Hi,
Is it possible to create the shared state(MapState) between two different keyedProcessFunction? If it's possible, how can we do that in flink?
Thanks,
Jaswin
Re: Shared state between two process functions
Posted by Congxian Qiu <qc...@gmail.com>.
Hi
Maybe you can take a look at broadcast state[1]
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/stream/state/broadcast_state.html
Best,
Congxian
Robert Metzger <rm...@apache.org> 于2020年6月16日周二 上午2:18写道：
> Thanks for sharing some details on the use case: Are you able to move the
> common computation into one operator that runs before the ProcessFunctions,
> and you are sending the results there?
> You can build quite advanced dataflow graphs with Flink to model your
> problem.
>
> On Mon, Jun 15, 2020 at 9:01 AM Jaswin Shah <ja...@outlook.com>
> wrote:
>
>> Basically, I have multiple processafunctions and they are doing
>> some.computations based on some historical results and the historical
>> events and results are common across the process functions due to which I
>> have a lot of redundant processing in many process functions so, I have
>> been thinking of some shared state accessible between multiple
>> keyedprocessfunctions.
>>
>> Get Outlook for Android <https://aka.ms/ghei36>
>>
>> ------------------------------
>> *From:* Yun Gao <yu...@aliyun.com>
>> *Sent:* Monday, June 15, 2020 8:27:38 AM
>> *To:* Jaswin Shah <ja...@outlook.com>; user@flink.apache.org <
>> user@flink.apache.org>
>> *Subject:* Re: Shared state between two process functions
>>
>> Hi Jaswin,
>>
>> Currently the state belongs to single operators, thus it should be not
>> possible to share states between different operators. Could you also share
>> the original problem want to solve by sharing states ?
>>
>> Best,
>> Yun
>>
>>
>> ------------------Original Mail ------------------
>> *Sender:*Jaswin Shah <ja...@outlook.com>
>> *Send Date:*Sun Jun 14 18:57:54 2020
>> *Recipients:*user@flink.apache.org <us...@flink.apache.org>
>> *Subject:*Shared state between two process functions
>>
>> Hi,
>>
>> Is it possible to create the shared state(MapState) between two different
>> keyedProcessFunction? If it's possible, how can we do that in flink?
>>
>> Thanks,
>> Jaswin
>>
>>
Re: Shared state between two process functions
Posted by Robert Metzger <rm...@apache.org>.
Thanks for sharing some details on the use case: Are you able to move the
common computation into one operator that runs before the ProcessFunctions,
and you are sending the results there?
You can build quite advanced dataflow graphs with Flink to model your
problem.
On Mon, Jun 15, 2020 at 9:01 AM Jaswin Shah <ja...@outlook.com> wrote:
> Basically, I have multiple processafunctions and they are doing
> some.computations based on some historical results and the historical
> events and results are common across the process functions due to which I
> have a lot of redundant processing in many process functions so, I have
> been thinking of some shared state accessible between multiple
> keyedprocessfunctions.
>
> Get Outlook for Android <https://aka.ms/ghei36>
>
> ------------------------------
> *From:* Yun Gao <yu...@aliyun.com>
> *Sent:* Monday, June 15, 2020 8:27:38 AM
> *To:* Jaswin Shah <ja...@outlook.com>; user@flink.apache.org <
> user@flink.apache.org>
> *Subject:* Re: Shared state between two process functions
>
> Hi Jaswin,
>
> Currently the state belongs to single operators, thus it should be not
> possible to share states between different operators. Could you also share
> the original problem want to solve by sharing states ?
>
> Best,
> Yun
>
>
> ------------------Original Mail ------------------
> *Sender:*Jaswin Shah <ja...@outlook.com>
> *Send Date:*Sun Jun 14 18:57:54 2020
> *Recipients:*user@flink.apache.org <us...@flink.apache.org>
> *Subject:*Shared state between two process functions
>
> Hi,
>
> Is it possible to create the shared state(MapState) between two different
> keyedProcessFunction? If it's possible, how can we do that in flink?
>
> Thanks,
> Jaswin
>
>
Re: Shared state between two process functions
Posted by Jaswin Shah <ja...@outlook.com>.
Basically, I have multiple processafunctions and they are doing some.computations based on some historical results and the historical events and results are common across the process functions due to which I have a lot of redundant processing in many process functions so, I have been thinking of some shared state accessible between multiple keyedprocessfunctions.
Get Outlook for Android<https://aka.ms/ghei36>
________________________________
From: Yun Gao <yu...@aliyun.com>
Sent: Monday, June 15, 2020 8:27:38 AM
To: Jaswin Shah <ja...@outlook.com>; user@flink.apache.org <us...@flink.apache.org>
Subject: Re: Shared state between two process functions
Hi Jaswin,
Currently the state belongs to single operators, thus it should be not possible to share states between different operators. Could you also share the original problem want to solve by sharing states ?
Best,
Yun
------------------Original Mail ------------------
Sender:Jaswin Shah <ja...@outlook.com>
Send Date:Sun Jun 14 18:57:54 2020
Recipients:user@flink.apache.org <us...@flink.apache.org>
Subject:Shared state between two process functions
Hi,
Is it possible to create the shared state(MapState) between two different keyedProcessFunction? If it's possible, how can we do that in flink?
Thanks,
Jaswin
Re: Shared state between two process functions
Posted by Yun Gao <yu...@aliyun.com>.
Hi Jaswin,
Currently the state belongs to single operators, thus it should be not possible to share states between different operators. Could you also share the original problem want to solve by sharing states ?
Best,
Yun
------------------Original Mail ------------------
Sender:Jaswin Shah <ja...@outlook.com>
Send Date:Sun Jun 14 18:57:54 2020
Recipients:user@flink.apache.org <us...@flink.apache.org>
Subject:Shared state between two process functions
Hi,
Is it possible to create the shared state(MapState) between two different keyedProcessFunction? If it's possible, how can we do that in flink?
Thanks,
Jaswin