[jira] [Resolved] (HDFS-11366) Clean up old .ckpt files after saveNamespace

["Xiao Chen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-11366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xiao Chen resolved HDFS-11366.
------------------------------
    Resolution: Duplicate

Looks like we already have HDFS-3716 in place to take care of this problem. Sorry didn't find that earlier.

It's more aggressive than proposed here, but since the purge only happens after a successful checkpoint, the risk is low.

> Clean up old .ckpt files after saveNamespace
> --------------------------------------------
>
>                 Key: HDFS-11366
>                 URL: https://issues.apache.org/jira/browse/HDFS-11366
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs, namenode
>    Affects Versions: 2.6.0
>            Reporter: Xiao Chen
>            Assignee: Xiao Chen
>
> Checkpoints are done in the NN by writing to {{fsimage.ckpt_TXID}} files, and rename to {{fsimage_TXID}} files upon success.
> If a checkpoint fails half way, the fsimage.ckpt_ file will be left on disk. There is no logic to clean it up at all.
> After talking with [~atm], I understand the historical reason for not immediately cleaning up those files, since they maybe useful for disaster recovery.
> But feels like cleaning those ckpt files after a successful checkpoint, with a larger TXID threshold is also safe to do.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org