You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Tao Yang (JIRA)" <ji...@apache.org> on 2018/08/21 10:03:00 UTC
[jira] [Created] (YARN-8692) Support node utilization metrics for
SLS
Tao Yang created YARN-8692:
------------------------------
Summary: Support node utilization metrics for SLS
Key: YARN-8692
URL: https://issues.apache.org/jira/browse/YARN-8692
Project: Hadoop YARN
Issue Type: Improvement
Components: scheduler-load-simulator
Affects Versions: 3.2.0
Reporter: Tao Yang
Assignee: Tao Yang
Attachments: image-2018-08-21-17-50-04-011.png
The distribution of node utilization is an important healthy factor for the YARN cluster, related metrics in SLS can be used to evaluate the scheduling effects and optimize related configurations.
To implement this improvement, we need to do things as below:
(1) Add input configurations (contain avg and stddev for cpu/memory utilization ratio) and generate utilization samples for tasks, not include AM container cause I think it's negligible. (2) Simulate containers and node utilization within node status.
(3) calculate and generate the distribution metrics and use standard deviation metric (stddev for short) to evaluate the effects(smaller is better).
(4) show these metrics on SLS simulator page like this:
!image-2018-08-21-17-50-04-011.png!
For Node memory/CPU utilization distribution graphs, Y-axis is nodes number, and P0 represents 0%~9% utilization ratio(containers-utilization / node-total-resource), P1 represents 10%~19% utilization ratio, P2 represents 20%~29% utilization ratio, ..., at last P9 represents 90%~100% utilization ratio.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org