You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by "Bajaj, Abhinav" <ab...@here.com> on 2020/11/17 00:58:23 UTC
Flink 1.10 -> Savepoints referencing to checkpoints or not
Hi,
I am trying to understand the Flink 1.10 savepoints related documentation<https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/state/savepoints.html#triggering-savepoints> that mentions -
“When triggering a savepoint, a new savepoint directory is created where the data as well as the meta data will be stored.”
However, I came across this issue - https://issues.apache.org/jira/browse/FLINK-5763 which mentions the problem -
“For file system based checkpoints (FsStateBackend, RocksDBStateBackend) this results in the savepoint referencing files from the checkpoint directory (usually different than <target>)”
My understanding of the issue is that savepoints were not storing the data and were pointing to checkpoints for actual state data.
The issue has been fixed and changes made into Flink 1.11.
But, I would like to get the behavior for Flink 1.10 confirmed.
Does savepoint in Flink 1.10 store both the data & metadata? OR it has references to checkpoints & need access to checkpoints directory to restore state?
Thanks much,
Abhinav Bajaj
Re: Flink 1.10 -> Savepoints referencing to checkpoints or not
Posted by Congxian Qiu <qc...@gmail.com>.
Hi Bajaj
Savepoint does contain the metadata and data in Flink 1.10, it does
not need to reference any checkpoint data.
Best,
Congxian
Bajaj, Abhinav <ab...@here.com> 于2020年11月17日周二 上午8:58写道：
> Hi,
>
>
>
> I am trying to understand the Flink 1.10 savepoints related documentation
> <https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/state/savepoints.html#triggering-savepoints>
> that mentions -
>
> *“**When triggering a savepoint, a new savepoint directory is created
> where the data as well as the meta data will be stored.**”*
>
>
>
> However, I came across this issue -
> https://issues.apache.org/jira/browse/FLINK-5763 which mentions the
> problem -
>
> *“**For file system based checkpoints (FsStateBackend,
> RocksDBStateBackend) this results in the savepoint referencing files from
> the checkpoint directory (usually different than <target>)**”*
>
>
>
> My understanding of the issue is that savepoints were not storing the data
> and were pointing to checkpoints for actual state data.
>
> The issue has been fixed and changes made into Flink 1.11.
>
>
>
> But, I would like to get the behavior for Flink 1.10 confirmed.
>
>
>
> Does savepoint in Flink 1.10 store both the data & metadata? OR it has
> references to checkpoints & need access to checkpoints directory to restore
> state?
>
>
>
> Thanks much,
>
> Abhinav Bajaj
>
Re: Flink 1.10 -> Savepoints referencing to checkpoints or not
Posted by "Tzu-Li (Gordon) Tai" <tz...@apache.org>.
Hi,
Both the data and metadata is being stored in the savepoint directory, since
Flink 1.3.
The metadata in the savepoint directory does not reference and checkpoint
data files.
In 1.11, what was changed was that the savepoint metadata uses relative
paths to point to the data files in the savepoint directory, instead of
absolute paths.
This would allow you to treat the savepoint directories as self-contained
and free to relocate with filesystem operations outside of Flink.
Hope this clarifies things for you!
Cheers,
Gordon
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/