You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Yun Tang <my...@live.com> on 2018/08/27 08:52:13 UTC
Why documentation always say checkpoint does not support Flink
specific features like rescaling.
Hi, all
From the latest 1.6 (or all the previous verions) documentation https://ci.apache.org/projects/flink/flink-docs-release-1.6/ops/state/checkpoints.html#difference-to-savepoints, it said "Checkpoints have a few differences from savepoints, they do not support Flink specific features like rescaling."
However, if you just take a look at RocksDB-StateBackend's code, you could see clearly that RocksDBStateBackend support rescaling from previous checkpoint. Could anyone give an explanation?
Best
Yun
Re: Why documentation always say checkpoint does not support Flink
specific features like rescaling.
Posted by Andrey Zagrebin <an...@data-artisans.com>.
As I understand, this is more about the documented semantics of checkpoint feature that Flink community agreed to guarantee for users so far in the last releases. Although technically the rescaling is currently possible for full and incremental checkpoints, it can change in future if checkpoints would need breaking optimisations or on contrary it might be fixed in the docs as supported from some release and on. In general, checkpoint API is currently designed for failure recovery of running job, not as savepoints for manual restart including rescaling.
> On 27 Aug 2018, at 11:23, Chesnay Schepler <ch...@apache.org> wrote:
>
> Well that's a single StateBackend. In order to say that "Checkpoints support rescaling" this must apply to all StateBackends, which AFAIK is not the case.
>
> On 27.08.2018 10:52, Yun Tang wrote:
>> Hi, all
>>
>> >From the latest 1.6 (or all the previous verions) documentation https://ci.apache.org/projects/flink/flink-docs-release-1.6/ops/state/checkpoints.html#difference-to-savepoints, it said "Checkpoints have a few differences from savepoints, they do not support Flink specific features like rescaling."
>>
>> However, if you just take a look at RocksDB-StateBackend's code, you could see clearly that RocksDBStateBackend support rescaling from previous checkpoint. Could anyone give an explanation?
>>
>> Best
>> Yun
>>
>
Re: Why documentation always say checkpoint does not support Flink
specific features like rescaling.
Posted by Chesnay Schepler <ch...@apache.org>.
Well that's a single StateBackend. In order to say that "Checkpoints
support rescaling" this must apply to all StateBackends, which AFAIK is
not the case.
On 27.08.2018 10:52, Yun Tang wrote:
> Hi, all
>
> >From the latest 1.6 (or all the previous verions) documentation https://ci.apache.org/projects/flink/flink-docs-release-1.6/ops/state/checkpoints.html#difference-to-savepoints, it said "Checkpoints have a few differences from savepoints, they do not support Flink specific features like rescaling."
>
> However, if you just take a look at RocksDB-StateBackend's code, you could see clearly that RocksDBStateBackend support rescaling from previous checkpoint. Could anyone give an explanation?
>
> Best
> Yun
>