[jira] [Commented] (HDFS-14728) RBF：GetDatanodeReport causes a large GC pressure on the NameNodes

["Chen Zhang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-14728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906049#comment-16906049 ] 

Chen Zhang commented on HDFS-14728:
-----------------------------------

NamenodeBeanMetrics has implemented a dnCache to cache DataNode report from NameNode, I think we can reuse it for GetDatanodeReport rpc

> RBF：GetDatanodeReport causes a large GC pressure on the NameNodes
> -----------------------------------------------------------------
>
>                 Key: HDFS-14728
>                 URL: https://issues.apache.org/jira/browse/HDFS-14728
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: rbf
>            Reporter: xuzq
>            Priority: Major
>
> When a cluster contains millions of DNs, *GetDatanodeReport* is pretty expensive, and it will cause a large GC pressure on NameNode.
> When multiple NSs share the millions DNs by federation and the router listens to the NSs, the problem will be more serious.
> All the NSs will be GC at the same time.
> RBF should cache the datanode report informations and have an option to disable the cache.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org