[jira] [Commented] (YARN-4888) Changes in RM AppSchedulingInfo for identifying resource-requests explicitly

["Arun Suresh (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376380#comment-15376380 ] 

Arun Suresh commented on YARN-4888:
-----------------------------------

Thanks for the patch [~subru]

Couple of initial comments:
# In {{AppSchedulingInfo::checkForDeactivation()}}, given that you are introducing an inner for loop, the deactivate = false should be set only if ALL requests in the {{mappedRequest.values()}} has {{request.getNumContainers() > 0}} right ?
# In {{AppSchedulingInfo::allocateNodeLocal()}}, That {{rackLocalRequest}} and {{offRackRequest}} selected from the {{resoureRequestMap}} should correspond to the same allocationRequestId of the {{nodeLocalRequest}}. Only when you have a single allocationRequestId, can you guarantee that {{.firstEntry().getValue()}} will correspond to the same nodeLocalRequest in both cases.

With regard to the getMergeResource() method. I have a feeling we should not be merging all requests of a {{Priority}}. Consider this alternate approach, which I feel might allow us to more easily verify if we are breaking any invariants in the Scheduler:
My intuition is based on the fact that If we agree that, in the absence of an *allocateRequestId*, we can simulate the same functionality by using a unique Priority to tie all requests we need to be of the same allocateRequestId....  Then, why not replace the 'Priority' in the {{AppSchedulingInfo::resourceRequestMap}} with a new type (called *SchedulerPriority*) which is essentially a composite of *Priority + allocateRequestId*. If not allocateRequestId is provided, it will default to 0. If the *SchedulerPriority* is a subclass of *Priority*, then we wont even need to change any of the APIs.

Thoughts ?
I can help provide a quick prototype patch to verify if this works..

> Changes in RM AppSchedulingInfo for identifying resource-requests explicitly
> ----------------------------------------------------------------------------
>
>                 Key: YARN-4888
>                 URL: https://issues.apache.org/jira/browse/YARN-4888
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Subru Krishnan
>            Assignee: Subru Krishnan
>         Attachments: YARN-4888-v0.patch
>
>
> YARN-4879 puts forward the notion of identifying allocate requests explicitly. This JIRA is to track the changes in RM app scheduling data structures to accomplish it. Please refer to the design doc in the parent JIRA for details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org