You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Jim Brennan (Jira)" <ji...@apache.org> on 2020/10/27 21:26:00 UTC
[jira] [Created] (YARN-10475) Scale RM-NM heartbeat interval based
on node utilization
Jim Brennan created YARN-10475:
----------------------------------
Summary: Scale RM-NM heartbeat interval based on node utilization
Key: YARN-10475
URL: https://issues.apache.org/jira/browse/YARN-10475
Project: Hadoop YARN
Issue Type: Improvement
Components: yarn
Affects Versions: 2.10.1, 3.4.1
Reporter: Jim Brennan
Assignee: Jim Brennan
Add the ability to scale the RM-NM heartbeat interval based on node cpu utilization compared to overall cluster cpu utilization. If a node is over-utilized compared to the rest of the cluster, it's heartbeat interval slows down. If it is under-utilized compared to the rest of the cluster, it's heartbeat interval speeds up.
This is a feature we have been running with internally in production for several years. It was developed by [~nroberts], based on the observation that larger faster nodes on our cluster were under-utilized compared to smaller slower nodes.
This feature is dependent on [YARN-10450], which added cluster-wide utilization metrics.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org