[jira] [Comment Edited] (YARN-5959) RM changes to support change of container ExecutionType

["Arun Suresh (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-5959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717931#comment-15717931 ] 

Arun Suresh edited comment on YARN-5959 at 12/3/16 11:21 AM:
-------------------------------------------------------------

This can actually be accomplished in a scheduler agnostic manner. 

Promotion can be implemented as follows:
# Create a dummy {{ResourceRequest}} with the same resource, priority etc. of the existing opportunistic container
# Set the required resource name (target location) to the node on which in the original container was allocated.
# Set *relaxLocality* to *false*.
# Set the *allocationRequestId* of the request to be negative, so it will be considered before any other {{ResourceRequest}} with the same priority.
# Once the scheduler allocates an {{RMContainer}} against the dummy request, swap the containerIds with the existing container and release this new container. This will ensure that the resource accounting is correct. The allocation path ensures that available resources in the queue, application and node resources are correctly decremented and the release ensures that only the opportunistic resource (essentially 0 resource container from the perspective of the scheduler) are reclaimed.

Demotion will happen in exactly the reverse:
A dummy OPPORTUNISTIC container is created to swap with the existing GUARANTEED container on the same node. The containerIds are then swapped and the dummy container is release ensuring guaranteed resources are reclaimed.


was (Author: asuresh):
This can actually be accomplished in a scheduler agnostic manner. 

Promotion can be implemented as follows:
# Create a dummy {{ResourceRequest}} with the same resource, priority etc. of the existing opportunistic container
# Set the required resource name (target location) to the node on which in the original container was allocated.
# Set *relaxLocality* to *false*.
# Set the *allocationRequestId* of the request to be negative, so it will be considered before any other {{ResourceRequest}} with the same priority.
# Once the scheduler allocates an {{RMContainer}} against the dummy request, swap the containerIds with the existing container and release this new container. This will ensure that the resource accounting is correct. The allocation path ensures that available resources in the queue, application and node resources are correctly decremented and the release ensures that only the opportunistic resource (essentially 0 resource container from the perspective of the scheduler) are reclaimed.

Demotion will happen in exactly the reverse:
A dummy OPPORTUNISTIC container is created to swap with the existing GUARANTEED container on the same node. The containerIds are then swapped the dummy container is release ensuring guaranteed resources are reclaimed.

> RM changes to support change of container ExecutionType
> -------------------------------------------------------
>
>                 Key: YARN-5959
>                 URL: https://issues.apache.org/jira/browse/YARN-5959
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>         Attachments: YARN-5959.wip.patch
>
>
> RM side changes to allow an AM to ask for change of ExecutionType.
> Currently, there are two cases:
> # *Promotion* : OPPORTUNISTIC to GUARANTEED.
> # *Demotion* : GUARANTEED to OPPORTUNISTIC.
> This is similar in YARN-1197 which allows for change in Container resources. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org