You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Stephan Ewen <se...@apache.org> on 2021/08/24 09:56:10 UTC

[ANNOUNCE] Dropping "CheckpointConfig.setPreferCheckpointForRecovery()"

Hi Flink Community!

A quick heads-up: We suggest removing the setting
"CheckpointConfig.setPreferCheckpointForRecovery()" [1].

The setting has been deprecated since Flink 1.12 and is strongly
discouraged, because it can lead to data loss or data duplication in
different scenarios.
Please see also https://issues.apache.org/jira/browse/FLINK-20427 for
background.

Are there any concerns about deprecating this issue? Is anyone relying on
this setting right now?

For a long-term solution to ensuring that there is no slow recovery from
savepoints: Some committers (me included) are working on a proposal to
support more efficient savepoints and to ensure that intermediate
savepoints don't interfere with side effects. Then we can always exclude
them from recovery without risk of data loss or duplication.

Best,
Stephan


[1]
https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/CheckpointConfig.java#L493

Re: [ANNOUNCE] Dropping "CheckpointConfig.setPreferCheckpointForRecovery()"

Posted by Gyula Fóra <gy...@gmail.com>.
Hi Stephan,

I do not know if anyone is still relying on this but I think it makes sense
to drop this feature. So +1 from me.

I think it served a valid purpose originally but if we have a good
improvement in the pipeline using the savepoints directly that will
solve the problem properly.
I would consider this feature a workaround at best anyways.

Regards,
Gyula

On Tue, Aug 24, 2021 at 11:56 AM Stephan Ewen <se...@apache.org> wrote:

> Hi Flink Community!
>
> A quick heads-up: We suggest removing the setting
> "CheckpointConfig.setPreferCheckpointForRecovery()" [1].
>
> The setting has been deprecated since Flink 1.12 and is strongly
> discouraged, because it can lead to data loss or data duplication in
> different scenarios.
> Please see also https://issues.apache.org/jira/browse/FLINK-20427 for
> background.
>
> Are there any concerns about deprecating this issue? Is anyone relying on
> this setting right now?
>
> For a long-term solution to ensuring that there is no slow recovery from
> savepoints: Some committers (me included) are working on a proposal to
> support more efficient savepoints and to ensure that intermediate
> savepoints don't interfere with side effects. Then we can always exclude
> them from recovery without risk of data loss or duplication.
>
> Best,
> Stephan
>
>
> [1]
> https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/CheckpointConfig.java#L493
>

Re: [ANNOUNCE] Dropping "CheckpointConfig.setPreferCheckpointForRecovery()"

Posted by Gyula Fóra <gy...@gmail.com>.
Hi Stephan,

I do not know if anyone is still relying on this but I think it makes sense
to drop this feature. So +1 from me.

I think it served a valid purpose originally but if we have a good
improvement in the pipeline using the savepoints directly that will
solve the problem properly.
I would consider this feature a workaround at best anyways.

Regards,
Gyula

On Tue, Aug 24, 2021 at 11:56 AM Stephan Ewen <se...@apache.org> wrote:

> Hi Flink Community!
>
> A quick heads-up: We suggest removing the setting
> "CheckpointConfig.setPreferCheckpointForRecovery()" [1].
>
> The setting has been deprecated since Flink 1.12 and is strongly
> discouraged, because it can lead to data loss or data duplication in
> different scenarios.
> Please see also https://issues.apache.org/jira/browse/FLINK-20427 for
> background.
>
> Are there any concerns about deprecating this issue? Is anyone relying on
> this setting right now?
>
> For a long-term solution to ensuring that there is no slow recovery from
> savepoints: Some committers (me included) are working on a proposal to
> support more efficient savepoints and to ensure that intermediate
> savepoints don't interfere with side effects. Then we can always exclude
> them from recovery without risk of data loss or duplication.
>
> Best,
> Stephan
>
>
> [1]
> https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/CheckpointConfig.java#L493
>