[jira] [Created] (HDFS-14576) Avoid block report retry and slow down namenode startup

["He Xiaoqiao (JIRA)" <ji...@apache.org>]

He Xiaoqiao created HDFS-14576:
----------------------------------

             Summary: Avoid block report retry and slow down namenode startup
                 Key: HDFS-14576
                 URL: https://issues.apache.org/jira/browse/HDFS-14576
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: namenode
            Reporter: He Xiaoqiao
            Assignee: He Xiaoqiao


During namenode startup, the load will be very high since it has to process every datanodes blockreport one by one. If there are hundreds datanodes block reports pending process, the issue will be more serious even #processFirstBlockReport is processed a lot more efficiently than ordinary block reports. Then some of datanode will retry blockreport and lengthens restart times. I think we should filter the block report request (via datanode blockreport retries) which has be processed and return directly then shorten down restart time. I want to state this proposal may be obvious only for large cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org