[jira] [Commented] (YARN-1197) Support increasing resources of an allocated container

["Wangda Tan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13768180#comment-13768180 ] 

Wangda Tan commented on YARN-1197:
----------------------------------

Here is the tentative plan:
To make it simple, we can start with only support increase container resource when container state is "ACQUIRED", because when container is running, we need find a way to inform NM to update the quato of a running process, which will affect lots of parts.
# Add a field in ResourceRequest, like set/getExistingContainerId() to indicate the resource will be added to existing container
# When the ask-list submitted to YarnScheduler, it will be normalized:
** When existingContainerId set
*** Check if container with "existingContainerId" in state ACQUIRED, abondon this request if container not exist OR state not ACQUIRED.
*** Check if capability of this container + asked capability > scheduler.maximum-allocation-mb, if yes, normalize ask capability 
*** Other fields will be ignored except "capability", and the "priority/host" will use the existing container's priority/host
*** Update resource requests in AppSchedulingInfo
# Add "increase existing container resource" support in LeafQueue of capa/fifo scheduler, and FSLeafQueue in fair scheduler. We can treat "increase existing container resource request" as same as other resource request except,
** Only "node-local" available
** Don't create new container
# After we successfully allocated resource to an existing container, we need refresh its container token, send back to AM. AM can use the new container to launch processes via NM.

Any thoughts? I wish I could help to move this ahead if we can reach an agreement on a rough idea. :)
                
> Support increasing resources of an allocated container
> ------------------------------------------------------
>
>                 Key: YARN-1197
>                 URL: https://issues.apache.org/jira/browse/YARN-1197
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: api, nodemanager, resourcemanager
>    Affects Versions: 2.1.0-beta
>            Reporter: Wangda Tan
>
> Currently, YARN cannot support merge several containers in one node to a big container, which can make us incrementally ask resources, merge them to a bigger one, and launch our processes. The user scenario is described in the comments.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira