You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by bastien dine <ba...@gmail.com> on 2019/05/14 07:55:36 UTC
State migration into multiple operators
Hello,
I would like to have some advices about splitting an operator with a state
into multiple operators.
The new operators would have state containing pieces of information of the
initial state
We will "split" the state
For exemple, I have operator (process) with uid A, with a state containing
field1, field2
I would like to split it into two operator B & C with, respectively, state
field1 and state field2
How can I split my state upon multiple operators ?
Best Regards,
Bastien
------------------
Bastien DINE
Data Architect / Software Engineer / Sysadmin
bastiendine.io
Re: State migration into multiple operators
Posted by "Tzu-Li (Gordon) Tai" <tz...@apache.org>.
Hi,
Just to add to what Piotr already mentioned:
The community is working on adding support for this directly in Flink.
You can follow the efforts here:
https://issues.apache.org/jira/browse/FLINK-12047.
Cheers,
Gordon
On Tue, May 14, 2019 at 11:39 AM Piotr Nowojski <pi...@ververica.com> wrote:
> Hi,
>
> Currently there is no native Flink support for modifying the state in a
> such manner. However there is an on-going effort [1] and a third party project
> [2] to address exactly this. Both allows you you to read savepoint,
> modify it and write back the new modified savepoint from which you can
> restore.
>
> [1] https://github.com/sjwiesman/flink/commits/savepoint-connector
> [2] https://github.com/king/bravo
>
> You can take a look at them.
>
> Piotrek
>
> On 14 May 2019, at 09:55, bastien dine <ba...@gmail.com> wrote:
>
> Hello,
>
> I would like to have some advices about splitting an operator with a state
> into multiple operators.
> The new operators would have state containing pieces of information of the
> initial state
> We will "split" the state
>
> For exemple, I have operator (process) with uid A, with a state containing
> field1, field2
> I would like to split it into two operator B & C with, respectively, state
> field1 and state field2
>
> How can I split my state upon multiple operators ?
>
> Best Regards,
> Bastien
>
> ------------------
>
> Bastien DINE
> Data Architect / Software Engineer / Sysadmin
> bastiendine.io
>
>
>
Re: State migration into multiple operators
Posted by Piotr Nowojski <pi...@ververica.com>.
Hi,
Currently there is no native Flink support for modifying the state in a such manner. However there is an on-going effort [1] and a third party project [2] to address exactly this. Both allows you you to read savepoint, modify it and write back the new modified savepoint from which you can restore.
[1] https://github.com/sjwiesman/flink/commits/savepoint-connector <https://github.com/sjwiesman/flink/commits/savepoint-connector>
[2] https://github.com/king/bravo <https://github.com/king/bravo>
You can take a look at them.
Piotrek
> On 14 May 2019, at 09:55, bastien dine <ba...@gmail.com> wrote:
>
> Hello,
>
> I would like to have some advices about splitting an operator with a state into multiple operators.
> The new operators would have state containing pieces of information of the initial state
> We will "split" the state
>
> For exemple, I have operator (process) with uid A, with a state containing field1, field2
> I would like to split it into two operator B & C with, respectively, state field1 and state field2
>
> How can I split my state upon multiple operators ?
>
> Best Regards,
> Bastien
>
> ------------------
>
> Bastien DINE
> Data Architect / Software Engineer / Sysadmin
> bastiendine.io <http://bastiendine.io/>