You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-dev@hadoop.apache.org by "Surendra Singh Lilhore (Jira)" <ji...@apache.org> on 2019/12/16 19:16:00 UTC

[jira] [Created] (HDFS-15067) Optimize heartbeat for large cluster

Surendra Singh Lilhore created HDFS-15067:
---------------------------------------------

             Summary: Optimize heartbeat for large cluster
                 Key: HDFS-15067
                 URL: https://issues.apache.org/jira/browse/HDFS-15067
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: datanode
    Affects Versions: 3.1.1
            Reporter: Surendra Singh Lilhore
            Assignee: Surendra Singh Lilhore


In a large cluster Namenode spend some time in processing heartbeats. For example, in 10K node cluster namenode process 10K RPC's for heartbeat in each 3sec. This will impact the client response time. This heart beat can be optimized. DN can start skipping one heart beat if no work(Write/replication/Delete) is allocated from long time. DN can start sending heart beat in 6 sec. Once the DN stating getting work from NN , it can start sending heart beat normally.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org