You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Liu (Jira)" <ji...@apache.org> on 2020/12/24 12:21:00 UTC
[jira] [Updated] (FLINK-20762) Remove unused checkpointStorage path
after job finished
[ https://issues.apache.org/jira/browse/FLINK-20762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liu updated FLINK-20762:
------------------------
Description:
Current checkpoint structure is as following:
{code:java}
/user-defined-checkpoint-dir
|
+ --shared/
+ --taskowned/
+ --chk-00001/
+ --chk-00002/...
{code}
After cancelling job, the checkpointStorage will not be removed. For some cases, the dictionary can not be removed. For example, external checkpoint is retained or taskowned is used.
If all the sub-dictionaries are empty, maybe the checkpointStorage can be removed. Otherwise, there will be lots of unused dictionaries with time going on. I wonder whether this is a problem. If so, I would like to fix it. Thank you.
was:
Current checkpoint structure is as following:
{code:java}
/user-defined-checkpoint-dir
|
+ --shared/
+ --taskowned/
+ --chk-00001/
+ --chk-00002/...
{code}
After cancelling job, the checkpointStorage will not be removed. For some cases, the dictionary can not removed. For example, external checkpoint is retain or taskowned is used.
If all the sub-dictionaries are empty, maybe the checkpointStorage can be removed. Otherwise, there will be lots of unused dictionaries with time going on. I wonder whether this is a problem. If so, I would like to fix it. Thank you.
> Remove unused checkpointStorage path after job finished
> -------------------------------------------------------
>
> Key: FLINK-20762
> URL: https://issues.apache.org/jira/browse/FLINK-20762
> Project: Flink
> Issue Type: Improvement
> Reporter: Liu
> Priority: Minor
>
> Current checkpoint structure is as following:
> {code:java}
> /user-defined-checkpoint-dir
> |
> + --shared/
> + --taskowned/
> + --chk-00001/
> + --chk-00002/...
> {code}
> After cancelling job, the checkpointStorage will not be removed. For some cases, the dictionary can not be removed. For example, external checkpoint is retained or taskowned is used.
> If all the sub-dictionaries are empty, maybe the checkpointStorage can be removed. Otherwise, there will be lots of unused dictionaries with time going on. I wonder whether this is a problem. If so, I would like to fix it. Thank you.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)