You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "He Xiaoqiao (JIRA)" <ji...@apache.org> on 2019/06/03 14:12:00 UTC
[jira] [Updated] (HDFS-14513) FSImage which is saving should be
clean while NameNode shutdown
[ https://issues.apache.org/jira/browse/HDFS-14513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
He Xiaoqiao updated HDFS-14513:
-------------------------------
Attachment: HDFS-14513.004.patch
> FSImage which is saving should be clean while NameNode shutdown
> ---------------------------------------------------------------
>
> Key: HDFS-14513
> URL: https://issues.apache.org/jira/browse/HDFS-14513
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: He Xiaoqiao
> Assignee: He Xiaoqiao
> Priority: Major
> Attachments: HDFS-14513.001.patch, HDFS-14513.002.patch, HDFS-14513.003.patch, HDFS-14513.004.patch
>
>
> Checkpointer/FSImageSaver is regular tasks and dump NameNode meta to disk, at most per hour by default. If it receive some command (e.g. transition to active in HA mode) it will cancel checkpoint and delete tmp files using {{FSImage#deleteCancelledCheckpoint}}. However if NameNode shutdown when checkpoint, the tmp files will not be cleaned anymore.
> Consider there are 500m inodes+blocks, it could cost 5~10min to finish once checkpoint, if we shutdown NameNode during checkpointing, fsimage checkpoint file will never be cleaned, after long time, there could be many useless checkpoint files. So I propose that we should add hook to clean that when shutdown.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org