You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "zhuqi (Jira)" <ji...@apache.org> on 2020/12/03 10:18:00 UTC
[jira] [Comment Edited] (YARN-10514) Introduce a dominant resource
based schedule policy to increase the resource utilization, avoid heavy
cluster resource fragments.
[ https://issues.apache.org/jira/browse/YARN-10514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17243071#comment-17243071 ]
zhuqi edited comment on YARN-10514 at 12/3/20, 10:17 AM:
---------------------------------------------------------
[~leftnoteasy] [~tangzhankun] [~prabhujoseph] [~sunil.govind@gmail.com] [~jiwq]
If you any advice about this proposal.
I submit a draft patch to support async multi node scheduling mode in CS, but i think in heartbeat schedule mode in CS/FS, we also should to handle resource resource fragments, to help get better resource utilization.
was (Author: zhuqi):
[~leftnoteasy] [~tangzhankun] [~prabhujoseph] [~sunil.govind@gmail.com] [~jiwq]
If you any advice about this proposal.
I submit a draft patch to support async multi node scheduling mode, but i think in heartbeat schedule mode we also should to handle resource resource fragments, to help get better resource utilization.
> Introduce a dominant resource based schedule policy to increase the resource utilization, avoid heavy cluster resource fragments.
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-10514
> URL: https://issues.apache.org/jira/browse/YARN-10514
> Project: Hadoop YARN
> Issue Type: Improvement
> Affects Versions: 3.3.0, 3.4.0
> Reporter: zhuqi
> Assignee: zhuqi
> Priority: Major
> Attachments: YARN-10514.001.patch
>
>
> When we schedule in multi node lookup policy for async scheduling, or just use heartbeat update based scheduling, we both meet scheduling fragments. When cpu-intensive jobs or gpu-intensive or memory-intensive etc, the cluster will meet heavy waste of resources, so this issue will help to move scheduler support dominant resource based schedule, to help our cluster get better resource utilization.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org