You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Weiwei Yang (JIRA)" <ji...@apache.org> on 2018/08/23 04:24:00 UTC
[jira] [Commented] (YARN-8692) Support node utilization metrics for
SLS
[ https://issues.apache.org/jira/browse/YARN-8692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16589680#comment-16589680 ]
Weiwei Yang commented on YARN-8692:
-----------------------------------
+1 for the idea, it will be very helpful for testing load distribution. I am curious how node memory/cpu is calculated here? Is it based on the allocated memory/cpu?
> Support node utilization metrics for SLS
> ----------------------------------------
>
> Key: YARN-8692
> URL: https://issues.apache.org/jira/browse/YARN-8692
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: scheduler-load-simulator
> Affects Versions: 3.2.0
> Reporter: Tao Yang
> Assignee: Tao Yang
> Priority: Major
> Attachments: image-2018-08-21-18-04-22-749.png
>
>
> The distribution of node utilization is an important healthy factor for the YARN cluster, related metrics in SLS can be used to evaluate the scheduling effects and optimize related configurations.
> To implement this improvement, we need to do things as below:
> (1) Add input configurations (contain avg and stddev for cpu/memory utilization ratio) and generate utilization samples for tasks, not include AM container cause I think it's negligible.
> (2) Simulate containers and node utilization within node status.
> (3) calculate and generate the distribution metrics and use standard deviation metric (stddev for short) to evaluate the effects(smaller is better).
> (4) show these metrics on SLS simulator page like this:
> !image-2018-08-21-18-04-22-749.png!
> For Node memory/CPU utilization distribution graphs, Y-axis is nodes number, and P0 represents 0%~9% utilization ratio(containers-utilization / node-total-resource), P1 represents 10%~19% utilization ratio, P2 represents 20%~29% utilization ratio, ..., at last P9 represents 90%~100% utilization ratio.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org