[jira] [Commented] (YARN-392) Make it possible to schedule to specific nodes without dropping locality

["Sandy Ryza (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619136#comment-13619136 ] 

Sandy Ryza commented on YARN-392:
---------------------------------

It seems like we have two slightly different proposals, both of which add a boolean flag to ResourceRequests.  In one, suggested by me earlier on this JIRA and by Bikas in his previous comment, if I want a container only at node1, I submit my request for node1 with the flag turned on.  For this proposal, I think the way that makes most sense would be to require a node-level request with the flag turned on to be accompanied by rack-level and *-level requests with the flag turned on.  An advantage of this approach is that it feels a little more intuitive.  A disadvantage is that it requires modifying the scheduler data structures to separately account for node-specific requests.

In another, suggested by Arun in YARN-398 and implemented by me in the March 28 patch, if I want a container only at node1, I set the flag on the rack that node1 is on.  An advantage of this approach is that it allows blacklisting, i.e. saying I'm ok with a container anywhere but node2.  A disadvantage is that it does not allow some requests at the same priority to be node-specific and others not.

I'm not convinced yet on which approach to take.  Am I representing all the options?  Is there a usecase for blacklisting?  Is there a use case for having some requests at a priority be node-specific and others not?
                
> Make it possible to schedule to specific nodes without dropping locality
> ------------------------------------------------------------------------
>
>                 Key: YARN-392
>                 URL: https://issues.apache.org/jira/browse/YARN-392
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Sandy Ryza
>         Attachments: YARN-392-1.patch, YARN-392.patch
>
>
> Currently its not possible to specify scheduling requests for specific nodes and nowhere else. The RM automatically relaxes locality to rack and * and assigns non-specified machines to the app.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira