[jira] [Commented] (YARN-3676) Disregard 'assignMultiple' directive while scheduling apps with NODE_LOCAL resource requests

["Anubhav Dhoot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14731244#comment-14731244 ] 

Anubhav Dhoot commented on YARN-3676:
-------------------------------------

Thanks [~asuresh] for working on this. I see the patch continues assigning on the node if you have *any* app which has a specific request on that node. But the scheduling attempt (via queueMgr.getRootQueue().assignContainer(node)) does not restrict which apps will get allocation on that node. So one could end up assigning the next container on the node for an app which may not have a specific request for that node. 
I see two choices. 
a) Smaller change - We should allow subsequent assignments only for the node local only apps? And you already have that list in the map. That can end up prioritizing the application's node local request over other applications.
b) BIgger change - Once we have picked the app based on priority, we allow it to assign multiple containers if there are multiple node local requests for that node.
 

> Disregard 'assignMultiple' directive while scheduling apps with NODE_LOCAL resource requests
> --------------------------------------------------------------------------------------------
>
>                 Key: YARN-3676
>                 URL: https://issues.apache.org/jira/browse/YARN-3676
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>         Attachments: YARN-3676.1.patch, YARN-3676.2.patch, YARN-3676.3.patch, YARN-3676.4.patch, YARN-3676.5.patch
>
>
> AssignMultiple is generally set to false to prevent overloading a Node (for eg, new NMs that have just joined)
> A possible scheduling optimization would be to disregard this directive for apps whose allowed locality is NODE_LOCAL



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)