You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-dev@hadoop.apache.org by "xuzq (JIRA)" <ji...@apache.org> on 2019/08/13 08:39:00 UTC

[jira] [Created] (HDFS-14728) RBF：GetDatanodeReport causes a large GC pressure on the NameNodes

xuzq created HDFS-14728:
---------------------------

             Summary: RBF：GetDatanodeReport causes a large GC pressure on the NameNodes
                 Key: HDFS-14728
                 URL: https://issues.apache.org/jira/browse/HDFS-14728
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: rbf
            Reporter: xuzq


When a cluster contains millions of DNs, *GetDatanodeReport* is pretty expensive, and it will cause a large GC pressure on NameNode.
When multiple NSs share the millions DNs by federation and the router listens to the NSs, the problem will be more serious.
All the NSs will be GC at the same time.

RBF should cache the datanode report informations and have an option to disable the cache.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org