[jira] [Created] (HDFS-17218) NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers

["Haiyang Hu (Jira)" <ji...@apache.org>]

Haiyang Hu created HDFS-17218:
---------------------------------

             Summary: NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers
                 Key: HDFS-17218
                 URL: https://issues.apache.org/jira/browse/HDFS-17218
             Project: Hadoop HDFS
          Issue Type: Improvement
            Reporter: Haiyang Hu
            Assignee: Haiyang Hu


Currently found that DN will lose all pending DNA_INVALIDATE blocks if it restarts.

*Root case:*
Current DN enables asynchronously deletion, it have many pending deletion blocks in memory.
when DN restarts, these cached blocks may be lost. it causes some blocks in the excess map in the namenode to be leaked and this will result in many blocks having more replicas then expected.

*Currently considering solutions:*
NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers,
this approach will ensure that when processing the DN's full block report, the 'processExtraRedundancy' can be performed according to the actual of the blocks.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org