[jira] [Commented] (YARN-4879) Proposal for a simple (delta) allocate protocol

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15239057#comment-15239057 ] 

Steve Loughran commented on YARN-4879:
--------------------------------------

I like the idea of having requestIDs on requests; helps us map from request to response. As you say, today things are an ugly hack with priority.

I would also like to see if the allocated containers could support a role ID field too...nothing much, but enough that on an AM restart their role can be determined. That one, I'd keep separate from the request ID; they serve slightly different purposes. (I could have 5 requests outstanding for containers of role 4; I'd want to track those requests)

> Proposal for a simple (delta) allocate protocol
> -----------------------------------------------
>
>                 Key: YARN-4879
>                 URL: https://issues.apache.org/jira/browse/YARN-4879
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: applications, resourcemanager
>            Reporter: Subru Krishnan
>            Assignee: Subru Krishnan
>         Attachments: SimpleAllocateProtocolProposal-v1.pdf
>
>
> For legacy reasons, the current allocate protocol expects expanded requests which represent the cumulative request for any change in resource constraints. This is not only very difficult to comprehend but makes it impossible for the scheduler to associate container allocations to the original requests. This problem is amplified by the fact that the expansion is managed by the AMRMClient which makes it cumbersome for non-Java clients as they all have to replicate the non-trivial logic. In this JIRA, we are proposing a delta allocate protocol where the AM will need to only specify changes in resource constraints.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)