You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Dawid Wysakowicz (Jira)" <ji...@apache.org> on 2022/02/15 09:30:00 UTC

[jira] [Updated] (FLINK-26147) Deleting incremental savepoints directory in CLAIM mode

     [ https://issues.apache.org/jira/browse/FLINK-26147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dawid Wysakowicz updated FLINK-26147:
-------------------------------------
    Description: 
Native format supports incremental RocksDB savepoints. For those savepoints Flink puts all SST files inside the savepoints directory. This means such savepoints are self-contained and relocatable. However, when restored in CLAIM mode, subsequent checkpoints might reuse some SST files, which in turn might block deleting the savepoints directory at the time the savepoint is subsumed. Later on Flink will delete the reused shared SST files, but it won’t retry deleting the savepoints directory. Therefore, it is possible Flink leaves an empty savepoints directory if it was restored in CLAIM mode.

We should find a way to clean up that left over directory.

> Deleting incremental savepoints directory in CLAIM mode
> -------------------------------------------------------
>
>                 Key: FLINK-26147
>                 URL: https://issues.apache.org/jira/browse/FLINK-26147
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Runtime / Checkpointing
>    Affects Versions: 1.15.0
>            Reporter: Piotr Nowojski
>            Priority: Major
>
> Native format supports incremental RocksDB savepoints. For those savepoints Flink puts all SST files inside the savepoints directory. This means such savepoints are self-contained and relocatable. However, when restored in CLAIM mode, subsequent checkpoints might reuse some SST files, which in turn might block deleting the savepoints directory at the time the savepoint is subsumed. Later on Flink will delete the reused shared SST files, but it won’t retry deleting the savepoints directory. Therefore, it is possible Flink leaves an empty savepoints directory if it was restored in CLAIM mode.
> We should find a way to clean up that left over directory.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)