[jira] [Updated] (FLINK-5086) Clean dead snapshot files produced by the tasks failing to acknowledge checkpoints

["Flink Jira Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-5086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Flink Jira Bot updated FLINK-5086:
----------------------------------
      Labels: auto-deprioritized-major auto-deprioritized-minor  (was: auto-deprioritized-major stale-minor)
    Priority: Not a Priority  (was: Minor)

This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion.


> Clean dead snapshot files produced by the tasks failing to acknowledge checkpoints
> ----------------------------------------------------------------------------------
>
>                 Key: FLINK-5086
>                 URL: https://issues.apache.org/jira/browse/FLINK-5086
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / Checkpointing
>            Reporter: Xiaogang Shi
>            Priority: Not a Priority
>              Labels: auto-deprioritized-major, auto-deprioritized-minor
>
> A task may fail when performing checkpoints. In that case, the task may have already copied some data to external storage. But since the task fails to send the state handler to {{CheckpointCoordinator}}, the copied data will not be deleted by {{CheckpointCoordinator}}. 
> I think we must find a method to clean such dead snapshot data to avoid unlimited usage of external storage. 
> One possible method is to clean these dead files when the task recovers. When a task recovers, {{CheckpointCoordinator}} will tell the task all the retained checkpoints. The task then can scan the external storage to delete all the  snapshots not in these retained checkpoints.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)