You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Yassine MARZOUGUI <y....@mindlytix.com> on 2017/02/01 13:03:19 UTC
Externalized Checkpoints vs Periodic Checkpoints
Hi all,
Could someone clarify the difference between externalized checkpoints[1]
and regular periodic checkpoints[2]?
Moreover, I have a question regarding the retention of checkpoints: For
regular checkpoints, does the last checkpoint discard the previous ones? If
yes, is that the case too for the externalized checkpoints? Thank you.
Best,
Yassine
[1] :
https://ci.apache.org/projects/flink/flink-docs-release-1.2/setup/checkpoints.html
[2] :
https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/stream/checkpointing.html
Re: Externalized Checkpoints vs Periodic Checkpoints
Posted by Yassine MARZOUGUI <y....@mindlytix.com>.
Thank you Till for the clarification, that was helpful.
Best,
Yassine
2017-02-02 15:31 GMT+01:00 Till Rohrmann <tr...@apache.org>:
> Hi Yassine,
>
> a periodic checkpoint is checkpoint which will be triggered periodically
> by Flink. The checkpoint itself can have multiple properties and one of
> them is whether the checkpoint is externalized or not.
>
> An externalized checkpoint is a checkpoint for which Flink writes the meta
> information into a target directory. In contrast to that, for a
> non-externalized checkpoint Flink will store the checkpoint meta
> information only in memory. The former has the advantage that you don't
> lose the checkpoints if you shutdown your cluster. They behave similar to
> savepoints and in fact savepoints are externalized checkpoints with some
> more properties.
>
> At the moment, Flink's checkpoint coordinator only retains the last
> successfully completed checkpoint. This means that whenever a new
> checkpoint completes then the last completed checkpoint will be discarded.
> This also applies to externalized checkpoints.
>
> Cheers,
> Till
>
> On Wed, Feb 1, 2017 at 2:03 PM, Yassine MARZOUGUI <
> y.marzougui@mindlytix.com> wrote:
>
>> Hi all,
>>
>> Could someone clarify the difference between externalized checkpoints[1]
>> and regular periodic checkpoints[2]?
>> Moreover, I have a question regarding the retention of checkpoints: For
>> regular checkpoints, does the last checkpoint discard the previous ones? If
>> yes, is that the case too for the externalized checkpoints? Thank you.
>>
>> Best,
>> Yassine
>>
>> [1] : https://ci.apache.org/projects/flink/flink-docs-release-1.2/
>> setup/checkpoints.html
>> [2] : https://ci.apache.org/projects/flink/flink-docs-release-1.
>> 2/dev/stream/checkpointing.html
>>
>
>
Re: Externalized Checkpoints vs Periodic Checkpoints
Posted by Till Rohrmann <tr...@apache.org>.
Hi Yassine,
a periodic checkpoint is checkpoint which will be triggered periodically by
Flink. The checkpoint itself can have multiple properties and one of them
is whether the checkpoint is externalized or not.
An externalized checkpoint is a checkpoint for which Flink writes the meta
information into a target directory. In contrast to that, for a
non-externalized checkpoint Flink will store the checkpoint meta
information only in memory. The former has the advantage that you don't
lose the checkpoints if you shutdown your cluster. They behave similar to
savepoints and in fact savepoints are externalized checkpoints with some
more properties.
At the moment, Flink's checkpoint coordinator only retains the last
successfully completed checkpoint. This means that whenever a new
checkpoint completes then the last completed checkpoint will be discarded.
This also applies to externalized checkpoints.
Cheers,
Till
On Wed, Feb 1, 2017 at 2:03 PM, Yassine MARZOUGUI <y.marzougui@mindlytix.com
> wrote:
> Hi all,
>
> Could someone clarify the difference between externalized checkpoints[1]
> and regular periodic checkpoints[2]?
> Moreover, I have a question regarding the retention of checkpoints: For
> regular checkpoints, does the last checkpoint discard the previous ones? If
> yes, is that the case too for the externalized checkpoints? Thank you.
>
> Best,
> Yassine
>
> [1] : https://ci.apache.org/projects/flink/flink-docs-
> release-1.2/setup/checkpoints.html
> [2] : https://ci.apache.org/projects/flink/flink-docs-
> release-1.2/dev/stream/checkpointing.html
>