how to match external checkpoints with jobs during recovery

[xiatao123 <ta...@udacity.com>]

The external checkpoints are in the format of
checkpoint_metadata-0057xxxxxxxx
which I have no idea which job this checkpoint metadata belongs to if I have
multiple jobs running at the same time.

If a job failed unexpected, I need to know which checkpoints belongs to the
failed job. Is there API or someway to show the checkpoints folder for a
certain job?

Thanks for the help
Tao



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: how to match external checkpoints with jobs during recovery

[Aljoscha Krettek <al...@apache.org>]

Hi,

I'm afraid it's currently not possible to distinguish between externalised checkpoints when running multiple jobs on one JobManager because the externalised checkpoints of all jobs would be written to the same directory.

In Flink 1.5 (which is not yet released, but the code for this is in master already) we changed how this part works and the externalised checkpoints data is now written to the checkpointing directory that you configure for each job, making it possible to distinguish externalised checkpoints. There is no more global directory for externalised checkpoints where the data for all jobs is written to.

Best,
Aljoscha

> On 5. Feb 2018, at 20:27, xiatao123 <ta...@udacity.com> wrote:
> 
> The external checkpoints are in the format of
> checkpoint_metadata-0057xxxxxxxx
> which I have no idea which job this checkpoint metadata belongs to if I have
> multiple jobs running at the same time.
> 
> If a job failed unexpected, I need to know which checkpoints belongs to the
> failed job. Is there API or someway to show the checkpoints folder for a
> certain job?
> 
> Thanks for the help
> Tao
> 
> 
> 
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/