[jira] [Commented] (YARN-4981) limit the max containers to assign to one application per heartbeart

["Karthik Kambatla (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15252787#comment-15252787 ] 

Karthik Kambatla commented on YARN-4981:
----------------------------------------

Does limiting to one application per heartbeat solve this problem? What happens if the scheduler assigns multiple CPU-intensive tasks (from different applications) on the same node? If the job is CPU-intensive, may be it should ask for more vcores so the CPU is not oversubscribed? 

Also, [~ywskycn] and I were looking into improving this in the past. We were considering determining maxAssign dynamically based on the delta between the node's allocation and the overall average allocation per node in the cluster. What do you think of that? 

> limit the max containers to assign to one application per heartbeart
> --------------------------------------------------------------------
>
>                 Key: YARN-4981
>                 URL: https://issues.apache.org/jira/browse/YARN-4981
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: fairscheduler
>    Affects Versions: 2.5.0, 2.6.4
>            Reporter: ChenFolin
>         Attachments: YARN-4981.patch
>
>
> When use assignMultiple，and such as set maxAssign=10:
> if a job is high CPU util or high CPU load, it may lead to  a part of nodes   very busy. and the job may cost a long time.
> so , i want to limit the max containers to assign to one application per heartbeart, it may help to nodes use more uniform.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)