You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by 大森林 <ap...@foxmail.com> on 2020/10/07 04:24:00 UTC
why we need keyed state and operate state when we already have checkpoint?
Could you tell me:
why we need keyed state and operator state when we already have checkpoint?
when a running jar crash,we can resume from the checkpoint automatically/manually.
So why did we still need keyed state and operator state.
Thanks
回复： why we need keyed state and operate state when we already have checkpoint?
Posted by 大森林 <ap...@foxmail.com>.
So
state:
store the result of some operator(such as keyby,map)
Checkpoint:
store the last result when the program is running OK.
Am I right?
Thanks for your help~!
------------------ 原始邮件 ------------------
发件人: "Congxian Qiu" <qcx978132955@gmail.com>;
发送时间: 2020年10月13日(星期二) 中午1:32
收件人: "大森林"<appleyuchi@foxmail.com>;
抄送: "Arvid Heise"<arvid@ververica.com>;"Shengkai Fang"<fskmine@gmail.com>;"user"<user@flink.apache.org>;
主题: Re: why we need keyed state and operate state when we already have checkpoint?
Hi As others said, state is different as checkpoint. a checkpoint is just a **snapshot** of the state, and you can restore from the previous checkpoint if the job crashed.
state is for stateful computation, and checkpoint is for fault-tolerant[1]
The state keeps the information you'll need in the future. Take wordcount as an example, the count of the word depends on the total count of the word we have seen, we need to keep the "total count of the word have seen before" somewhere, in Flink you can keep it in the state.
checkpoint/savepoint contains the **snapshot** of all the state, if there is not state, then the checkpoint will be *empty*, you can restore from it, but the content is empty.
PS: maybe you don't create state explicit, but there contain some states in Flink(such as WindowOperator)
[1] https://ci.apache.org/projects/flink/flink-docs-release-1.11/concepts/stateful-stream-processing.html
Best,
Congxian
大森林 <appleyuchi@foxmail.com> 于2020年10月12日周一 下午9:26写道：
Thanks for your replies.
When I use no state-relevant code in my program,the checkingpoint can be saved and resumed.❶
So then why we need Keyed State/Operator State/Stateful Function?❷
"the operators are reset to the time of the respective checkpoint."
We already have met the requirement:"resume from checkpoint(last state of each operator which store the result)"❶,
why we still need ❷?
Thanks for your help~!
------------------ 原始邮件 ------------------
发件人: "Arvid Heise" <arvid@ververica.com>;
发送时间: 2020年10月12日(星期一) 下午2:53
收件人: "大森林"<appleyuchi@foxmail.com>;
抄送: "Shengkai Fang"<fskmine@gmail.com>;"user"<user@flink.apache.org>;
主题: Re: why we need keyed state and operate state when we already have checkpoint?
Hi 大森林,
You can always resume from checkpoints independent of the usage of keyed or non-keyed state of operators.
1 checkpoint contains the state of all operators at a given point in time. Each operator may have keyed state, raw state, or non-keyed state.
As long as you are not changing the operators (too much) before restarting, you can always restart.
During (automatic) restart of a Flink application, the state of a given checkpoint is restored to the operators, such that it looks like the operator never failed. However, the operators are reset to the time of the respective checkpoint.
I have no clue what you mean with "previous variable temporary result".
On Wed, Oct 7, 2020 at 9:13 AM 大森林 <appleyuchi@foxmail.com> wrote:
Thanks for your replies,I have some understandings.
There are two cases.
1. if I use no keyed state in program,when it's killed,I can only resume from previous result
1. if I use keyed state in program,when it's killed,I can resume from previous result and previous variable temporary result.
Am I right?
Thanks for your guide.
------------------ 原始邮件 ------------------
发件人: "Arvid Heise" <arvid@ververica.com>;
发送时间: 2020年10月7日(星期三) 下午2:25
收件人: "大森林"<appleyuchi@foxmail.com>;
抄送: "Shengkai Fang"<fskmine@gmail.com>;"user"<user@flink.apache.org>;
主题: Re: why we need keyed state and operate state when we already have checkpoint?
I think there is some misunderstanding here: a checkpoint IS (a snapshot of) the keyed state and operator state (among a few more things). [1]
[1] https://ci.apache.org/projects/flink/flink-docs-release-1.11/learn-flink/fault_tolerance.html#definitions
On Wed, Oct 7, 2020 at 6:51 AM 大森林 <appleyuchi@foxmail.com> wrote:
when the job is killed,state is also misssing.
so why we need keyed state?Is keyed state useful when we try to resuming the killed job?
------------------ 原始邮件 ------------------
发件人: "Shengkai Fang" <fskmine@gmail.com>;
发送时间: 2020年10月7日(星期三) 中午12:43
收件人: "大森林"<appleyuchi@foxmail.com>;
抄送: "user"<user@flink.apache.org>;
主题: Re: why we need keyed state and operate state when we already have checkpoint?
The checkpoint is a snapshot for the job and we can resume the job if the job is killed unexpectedly. The state is another thing to memorize the intermediate result of calculation. I don't think the checkpoint can replace state.
大森林 <appleyuchi@foxmail.com> 于2020年10月7日周三 下午12:26写道：
Could you tell me:
why we need keyed state and operator state when we already have checkpoint?
when a running jar crash,we can resume from the checkpoint automatically/manually.
So why did we still need keyed state and operator state.
Thanks
--
Arvid Heise | Senior Java Developer
Follow us @VervericaData
--
Join Flink Forward - The Apache Flink Conference
Stream Processing | Event Driven | Real Time
--
Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
--
Arvid Heise | Senior Java Developer
Follow us @VervericaData
--
Join Flink Forward - The Apache Flink Conference
Stream Processing | Event Driven | Real Time
--
Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
Re: why we need keyed state and operate state when we already have checkpoint?
Posted by Congxian Qiu <qc...@gmail.com>.
Hi
As others said, state is different as checkpoint. a checkpoint is just
a **snapshot** of the state, and you can restore from the previous
checkpoint if the job crashed.
state is for stateful computation, and checkpoint is for
fault-tolerant[1]
The state keeps the information you'll need in the future. Take
wordcount as an example, the count of the word depends on the total count
of the word we have seen, we need to keep the "total count of the word have
seen before" somewhere, in Flink you can keep it in the state.
checkpoint/savepoint contains the **snapshot** of all the state, if
there is not state, then the checkpoint will be *empty*, you can restore
from it, but the content is empty.
PS: maybe you don't create state explicit, but there contain some
states in Flink(such as WindowOperator)
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.11/concepts/stateful-stream-processing.html
Best,
Congxian
大森林 <ap...@foxmail.com> 于2020年10月12日周一 下午9:26写道：
> Thanks for your replies.
> When I use no state-relevant code in my program,the checkingpoint can be
> saved and resumed.❶
>
> So then why we need *Keyed State/Operator State/Stateful Function*?❷
> *"the operators are reset to the time of the respective checkpoint."*
> We already have met the requirement:*"resume from checkpoint(last state
> of each operator which store the result)"*❶,
> why we still need ❷?
> Thanks for your help~!
>
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Arvid Heise" <ar...@ververica.com>;
> *发送时间:* 2020年10月12日(星期一) 下午2:53
> *收件人:* "大森林"<ap...@foxmail.com>;
> *抄送:* "Shengkai Fang"<fs...@flink.apache.org>;
> *主题:* Re: why we need keyed state and operate state when we already have
> checkpoint?
>
> Hi 大森林,
>
> You can always resume from checkpoints independent of the usage of keyed
> or non-keyed state of operators.
> 1 checkpoint contains the state of all operators at a given point in time.
> Each operator may have keyed state, raw state, or non-keyed state.
> As long as you are not changing the operators (too much) before
> restarting, you can always restart.
>
> During (automatic) restart of a Flink application, the state of a given
> checkpoint is restored to the operators, such that it looks like the
> operator never failed. However, the operators are reset to the time of the
> respective checkpoint.
>
> I have no clue what you mean with "previous variable temporary result".
>
> On Wed, Oct 7, 2020 at 9:13 AM 大森林 <ap...@foxmail.com> wrote:
>
>> Thanks for your replies,I have some understandings.
>>
>> There are two cases.
>> 1. if I use no keyed state in program,when it's killed,I can only resume
>> from previous result
>> 1. if I use keyed state in program,when it's killed,I can
>> resume from previous result and previous variable temporary result.
>>
>> Am I right?
>> Thanks for your guide.
>>
>>
>> ------------------ 原始邮件 ------------------
>> *发件人:* "Arvid Heise" <ar...@ververica.com>;
>> *发送时间:* 2020年10月7日(星期三) 下午2:25
>> *收件人:* "大森林"<ap...@foxmail.com>;
>> *抄送:* "Shengkai Fang"<fs...@flink.apache.org>;
>> *主题:* Re: why we need keyed state and operate state when we already have
>> checkpoint?
>>
>> I think there is some misunderstanding here: a checkpoint IS (a snapshot
>> of) the keyed state and operator state (among a few more things). [1]
>>
>> [1]
>> https://ci.apache.org/projects/flink/flink-docs-release-1.11/learn-flink/fault_tolerance.html#definitions
>>
>> On Wed, Oct 7, 2020 at 6:51 AM 大森林 <ap...@foxmail.com> wrote:
>>
>>> when the job is killed,state is also misssing.
>>> so why we need keyed state?Is keyed state useful when we try to resuming
>>> the killed job?
>>>
>>>
>>> ------------------ 原始邮件 ------------------
>>> *发件人:* "Shengkai Fang" <fs...@gmail.com>;
>>> *发送时间:* 2020年10月7日(星期三) 中午12:43
>>> *收件人:* "大森林"<ap...@foxmail.com>;
>>> *抄送:* "user"<us...@flink.apache.org>;
>>> *主题:* Re: why we need keyed state and operate state when we already
>>> have checkpoint?
>>>
>>> The checkpoint is a snapshot for the job and we can resume the job if
>>> the job is killed unexpectedly. The state is another thing to memorize the
>>> intermediate result of calculation. I don't think the checkpoint can
>>> replace state.
>>>
>>> 大森林 <ap...@foxmail.com> 于2020年10月7日周三 下午12:26写道：
>>>
>>>> Could you tell me:
>>>>
>>>> why we need keyed state and operator state when we already have
>>>> checkpoint?
>>>>
>>>> when a running jar crash,we can resume from the checkpoint
>>>> automatically/manually.
>>>> So why did we still need keyed state and operator state.
>>>>
>>>> Thanks
>>>>
>>>
>>
>> --
>>
>> Arvid Heise | Senior Java Developer
>>
>> <https://www.ververica.com/>
>>
>> Follow us @VervericaData
>>
>> --
>>
>> Join Flink Forward <https://flink-forward.org/> - The Apache Flink
>> Conference
>>
>> Stream Processing | Event Driven | Real Time
>>
>> --
>>
>> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
>>
>> --
>> Ververica GmbH
>> Registered at Amtsgericht Charlottenburg: HRB 158244 B
>> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
>> (Toni) Cheng
>>
>
>
> --
>
> Arvid Heise | Senior Java Developer
>
> <https://www.ververica.com/>
>
> Follow us @VervericaData
>
> --
>
> Join Flink Forward <https://flink-forward.org/> - The Apache Flink
> Conference
>
> Stream Processing | Event Driven | Real Time
>
> --
>
> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
>
> --
> Ververica GmbH
> Registered at Amtsgericht Charlottenburg: HRB 158244 B
> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
> (Toni) Cheng
>
回复： why we need keyed state and operate state when we already have checkpoint?
Posted by 大森林 <ap...@foxmail.com>.
Thanks for your replies.
When I use no state-relevant code in my program,the checkingpoint can be saved and resumed.❶
So then why we need Keyed State/Operator State/Stateful Function?❷
"the operators are reset to the time of the respective checkpoint."
We already have met the requirement:"resume from checkpoint(last state of each operator which store the result)"❶,
why we still need ❷?
Thanks for your help~!
------------------ 原始邮件 ------------------
发件人: "Arvid Heise" <arvid@ververica.com>;
发送时间: 2020年10月12日(星期一) 下午2:53
收件人: "大森林"<appleyuchi@foxmail.com>;
抄送: "Shengkai Fang"<fskmine@gmail.com>;"user"<user@flink.apache.org>;
主题: Re: why we need keyed state and operate state when we already have checkpoint?
Hi 大森林,
You can always resume from checkpoints independent of the usage of keyed or non-keyed state of operators.
1 checkpoint contains the state of all operators at a given point in time. Each operator may have keyed state, raw state, or non-keyed state.
As long as you are not changing the operators (too much) before restarting, you can always restart.
During (automatic) restart of a Flink application, the state of a given checkpoint is restored to the operators, such that it looks like the operator never failed. However, the operators are reset to the time of the respective checkpoint.
I have no clue what you mean with "previous variable temporary result".
On Wed, Oct 7, 2020 at 9:13 AM 大森林 <appleyuchi@foxmail.com> wrote:
Thanks for your replies,I have some understandings.
There are two cases.
1. if I use no keyed state in program,when it's killed,I can only resume from previous result
1. if I use keyed state in program,when it's killed,I can resume from previous result and previous variable temporary result.
Am I right?
Thanks for your guide.
------------------ 原始邮件 ------------------
发件人: "Arvid Heise" <arvid@ververica.com>;
发送时间: 2020年10月7日(星期三) 下午2:25
收件人: "大森林"<appleyuchi@foxmail.com>;
抄送: "Shengkai Fang"<fskmine@gmail.com>;"user"<user@flink.apache.org>;
主题: Re: why we need keyed state and operate state when we already have checkpoint?
I think there is some misunderstanding here: a checkpoint IS (a snapshot of) the keyed state and operator state (among a few more things). [1]
[1] https://ci.apache.org/projects/flink/flink-docs-release-1.11/learn-flink/fault_tolerance.html#definitions
On Wed, Oct 7, 2020 at 6:51 AM 大森林 <appleyuchi@foxmail.com> wrote:
when the job is killed,state is also misssing.
so why we need keyed state?Is keyed state useful when we try to resuming the killed job?
------------------ 原始邮件 ------------------
发件人: "Shengkai Fang" <fskmine@gmail.com>;
发送时间: 2020年10月7日(星期三) 中午12:43
收件人: "大森林"<appleyuchi@foxmail.com>;
抄送: "user"<user@flink.apache.org>;
主题: Re: why we need keyed state and operate state when we already have checkpoint?
The checkpoint is a snapshot for the job and we can resume the job if the job is killed unexpectedly. The state is another thing to memorize the intermediate result of calculation. I don't think the checkpoint can replace state.
大森林 <appleyuchi@foxmail.com> 于2020年10月7日周三 下午12:26写道：
Could you tell me:
why we need keyed state and operator state when we already have checkpoint?
when a running jar crash,we can resume from the checkpoint automatically/manually.
So why did we still need keyed state and operator state.
Thanks
--
Arvid Heise | Senior Java Developer
Follow us @VervericaData
--
Join Flink Forward - The Apache Flink Conference
Stream Processing | Event Driven | Real Time
--
Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
--
Arvid Heise | Senior Java Developer
Follow us @VervericaData
--
Join Flink Forward - The Apache Flink Conference
Stream Processing | Event Driven | Real Time
--
Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
Re: why we need keyed state and operate state when we already have checkpoint?
Posted by Arvid Heise <ar...@ververica.com>.
Hi 大森林,
You can always resume from checkpoints independent of the usage of keyed or
non-keyed state of operators.
1 checkpoint contains the state of all operators at a given point in time.
Each operator may have keyed state, raw state, or non-keyed state.
As long as you are not changing the operators (too much) before restarting,
you can always restart.
During (automatic) restart of a Flink application, the state of a given
checkpoint is restored to the operators, such that it looks like the
operator never failed. However, the operators are reset to the time of the
respective checkpoint.
I have no clue what you mean with "previous variable temporary result".
On Wed, Oct 7, 2020 at 9:13 AM 大森林 <ap...@foxmail.com> wrote:
> Thanks for your replies,I have some understandings.
>
> There are two cases.
> 1. if I use no keyed state in program,when it's killed,I can only resume
> from previous result
> 1. if I use keyed state in program,when it's killed,I can
> resume from previous result and previous variable temporary result.
>
> Am I right?
> Thanks for your guide.
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Arvid Heise" <ar...@ververica.com>;
> *发送时间:* 2020年10月7日(星期三) 下午2:25
> *收件人:* "大森林"<ap...@foxmail.com>;
> *抄送:* "Shengkai Fang"<fs...@flink.apache.org>;
> *主题:* Re: why we need keyed state and operate state when we already have
> checkpoint?
>
> I think there is some misunderstanding here: a checkpoint IS (a snapshot
> of) the keyed state and operator state (among a few more things). [1]
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-release-1.11/learn-flink/fault_tolerance.html#definitions
>
> On Wed, Oct 7, 2020 at 6:51 AM 大森林 <ap...@foxmail.com> wrote:
>
>> when the job is killed,state is also misssing.
>> so why we need keyed state?Is keyed state useful when we try to resuming
>> the killed job?
>>
>>
>> ------------------ 原始邮件 ------------------
>> *发件人:* "Shengkai Fang" <fs...@gmail.com>;
>> *发送时间:* 2020年10月7日(星期三) 中午12:43
>> *收件人:* "大森林"<ap...@foxmail.com>;
>> *抄送:* "user"<us...@flink.apache.org>;
>> *主题:* Re: why we need keyed state and operate state when we already have
>> checkpoint?
>>
>> The checkpoint is a snapshot for the job and we can resume the job if the
>> job is killed unexpectedly. The state is another thing to memorize the
>> intermediate result of calculation. I don't think the checkpoint can
>> replace state.
>>
>> 大森林 <ap...@foxmail.com> 于2020年10月7日周三 下午12:26写道：
>>
>>> Could you tell me:
>>>
>>> why we need keyed state and operator state when we already have
>>> checkpoint?
>>>
>>> when a running jar crash,we can resume from the checkpoint
>>> automatically/manually.
>>> So why did we still need keyed state and operator state.
>>>
>>> Thanks
>>>
>>
>
> --
>
> Arvid Heise | Senior Java Developer
>
> <https://www.ververica.com/>
>
> Follow us @VervericaData
>
> --
>
> Join Flink Forward <https://flink-forward.org/> - The Apache Flink
> Conference
>
> Stream Processing | Event Driven | Real Time
>
> --
>
> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
>
> --
> Ververica GmbH
> Registered at Amtsgericht Charlottenburg: HRB 158244 B
> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
> (Toni) Cheng
>
--
Arvid Heise | Senior Java Developer
<https://www.ververica.com/>
Follow us @VervericaData
--
Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference
Stream Processing | Event Driven | Real Time
--
Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
(Toni) Cheng
回复： why we need keyed state and operate state when we already have checkpoint?
Posted by 大森林 <ap...@foxmail.com>.
Thanks for your replies,I have some understandings.
There are two cases.
1. if I use no keyed state in program,when it's killed,I can only resume from previous result
1. if I use keyed state in program,when it's killed,I can resume from previous result and previous variable temporary result.
Am I right?
Thanks for your guide.
------------------ 原始邮件 ------------------
发件人: "Arvid Heise" <arvid@ververica.com>;
发送时间: 2020年10月7日(星期三) 下午2:25
收件人: "大森林"<appleyuchi@foxmail.com>;
抄送: "Shengkai Fang"<fskmine@gmail.com>;"user"<user@flink.apache.org>;
主题: Re: why we need keyed state and operate state when we already have checkpoint?
I think there is some misunderstanding here: a checkpoint IS (a snapshot of) the keyed state and operator state (among a few more things). [1]
[1] https://ci.apache.org/projects/flink/flink-docs-release-1.11/learn-flink/fault_tolerance.html#definitions
On Wed, Oct 7, 2020 at 6:51 AM 大森林 <appleyuchi@foxmail.com> wrote:
when the job is killed,state is also misssing.
so why we need keyed state?Is keyed state useful when we try to resuming the killed job?
------------------ 原始邮件 ------------------
发件人: "Shengkai Fang" <fskmine@gmail.com>;
发送时间: 2020年10月7日(星期三) 中午12:43
收件人: "大森林"<appleyuchi@foxmail.com>;
抄送: "user"<user@flink.apache.org>;
主题: Re: why we need keyed state and operate state when we already have checkpoint?
The checkpoint is a snapshot for the job and we can resume the job if the job is killed unexpectedly. The state is another thing to memorize the intermediate result of calculation. I don't think the checkpoint can replace state.
大森林 <appleyuchi@foxmail.com> 于2020年10月7日周三 下午12:26写道：
Could you tell me:
why we need keyed state and operator state when we already have checkpoint?
when a running jar crash,we can resume from the checkpoint automatically/manually.
So why did we still need keyed state and operator state.
Thanks
--
Arvid Heise | Senior Java Developer
Follow us @VervericaData
--
Join Flink Forward - The Apache Flink Conference
Stream Processing | Event Driven | Real Time
--
Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
Re: why we need keyed state and operate state when we already have checkpoint?
Posted by Arvid Heise <ar...@ververica.com>.
I think there is some misunderstanding here: a checkpoint IS (a snapshot
of) the keyed state and operator state (among a few more things). [1]
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.11/learn-flink/fault_tolerance.html#definitions
On Wed, Oct 7, 2020 at 6:51 AM 大森林 <ap...@foxmail.com> wrote:
> when the job is killed,state is also misssing.
> so why we need keyed state?Is keyed state useful when we try to resuming
> the killed job?
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Shengkai Fang" <fs...@gmail.com>;
> *发送时间:* 2020年10月7日(星期三) 中午12:43
> *收件人:* "大森林"<ap...@foxmail.com>;
> *抄送:* "user"<us...@flink.apache.org>;
> *主题:* Re: why we need keyed state and operate state when we already have
> checkpoint?
>
> The checkpoint is a snapshot for the job and we can resume the job if the
> job is killed unexpectedly. The state is another thing to memorize the
> intermediate result of calculation. I don't think the checkpoint can
> replace state.
>
> 大森林 <ap...@foxmail.com> 于2020年10月7日周三 下午12:26写道：
>
>> Could you tell me:
>>
>> why we need keyed state and operator state when we already have
>> checkpoint?
>>
>> when a running jar crash,we can resume from the checkpoint
>> automatically/manually.
>> So why did we still need keyed state and operator state.
>>
>> Thanks
>>
>
--
Arvid Heise | Senior Java Developer
<https://www.ververica.com/>
Follow us @VervericaData
--
Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference
Stream Processing | Event Driven | Real Time
--
Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
(Toni) Cheng
回复： why we need keyed state and operate state when we already have checkpoint?
Posted by 大森林 <ap...@foxmail.com>.
when the job is killed,state is also misssing.
so why we need keyed state?Is keyed state useful when we try to resuming the killed job?
------------------ 原始邮件 ------------------
发件人: "Shengkai Fang" <fskmine@gmail.com>;
发送时间: 2020年10月7日(星期三) 中午12:43
收件人: "大森林"<appleyuchi@foxmail.com>;
抄送: "user"<user@flink.apache.org>;
主题: Re: why we need keyed state and operate state when we already have checkpoint?
The checkpoint is a snapshot for the job and we can resume the job if the job is killed unexpectedly. The state is another thing to memorize the intermediate result of calculation. I don't think the checkpoint can replace state.
大森林 <appleyuchi@foxmail.com> 于2020年10月7日周三 下午12:26写道：
Could you tell me:
why we need keyed state and operator state when we already have checkpoint?
when a running jar crash,we can resume from the checkpoint automatically/manually.
So why did we still need keyed state and operator state.
Thanks
Re: why we need keyed state and operate state when we already have checkpoint?
Posted by Shengkai Fang <fs...@gmail.com>.
The checkpoint is a snapshot for the job and we can resume the job if the
job is killed unexpectedly. The state is another thing to memorize the
intermediate result of calculation. I don't think the checkpoint can
replace state.
大森林 <ap...@foxmail.com> 于2020年10月7日周三 下午12:26写道：
> Could you tell me:
>
> why we need keyed state and operator state when we already have checkpoint?
>
> when a running jar crash,we can resume from the checkpoint
> automatically/manually.
> So why did we still need keyed state and operator state.
>
> Thanks
>