You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "apus66 (Jira)" <ji...@apache.org> on 2019/09/18 02:05:00 UTC

[jira] [Issue Comment Deleted] (FLINK-13856) Reduce the delete file api when the checkpoint is completed

     [ https://issues.apache.org/jira/browse/FLINK-13856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

apus66 updated FLINK-13856:
---------------------------
    Comment: was deleted

(was: Hi [~StephanEwen] Can you help me review the code and assign this task to me?  We have already verified in the production environment, the effect is very obvious！

The general idea is the following:
* In full checkpoint, to delete the snapshot(chk-) folder directly
* In incremental checkpoint, find IncrementalRemoteKeyedStateHandle and delete the shared state first, and then dropping the exclusive location as a whole.


[reduce delete api | [https://github.com/apache/flink/pull/9602]])

> Reduce the delete file api when the checkpoint is completed
> -----------------------------------------------------------
>
>                 Key: FLINK-13856
>                 URL: https://issues.apache.org/jira/browse/FLINK-13856
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Checkpointing, Runtime / State Backends
>    Affects Versions: 1.8.1, 1.9.0
>            Reporter: andrew.D.lin
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: after.png, before.png, f6cc56b7-2c74-4f4b-bb6a-476d28a22096.png
>
>   Original Estimate: 48h
>          Time Spent: 10m
>  Remaining Estimate: 47h 50m
>
> When the new checkpoint is completed, an old checkpoint will be deleted by calling CompletedCheckpoint.discardOnSubsume().
> When deleting old checkpoints, follow these steps:
> 1, drop the metadata
> 2, discard private state objects
> 3, discard location as a whole
> In some cases, is it possible to delete the checkpoint folder recursively by one call?
> As far as I know the full amount of checkpoint, it should be possible to delete the folder directly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)