[jira] [Comment Edited] (YARN-4138) Roll back container resource allocation after resource increase token expires

["Jian He (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15057106#comment-15057106 ] 

Jian He edited comment on YARN-4138 at 12/15/15 1:06 AM:
---------------------------------------------------------

{code}
    SchedContainerChangeRequest decreaseRequest =
        new SchedContainerChangeRequest(
            schedulerNode, rmContainer,
                rmContainer.getLastConfirmedResource());
    decreaseContainer(decreaseRequest,
        getCurrentAttemptForContainer(containerId));
{code}
- this scenario may cause resource accounting wrong,  correct me if I'm wrong:
1) AM asks increase 2G -> 8G
2) AM does not increase the container, and asks decrease to 1G
3) LastConfirmedResource becomes 1G
4) In the meantime, containerIncreaseExpiration logic is triggered and rollbackContainerResource is invoked. In this case the resource Delta becomes positive even in decrease case, but some code is assuming the decrease to be negative, which may cause resource accounting wrong ?
{code}
    // Delta capacity is negative when it's a decrease request
    Resource absDelta = Resources.negate(decreaseRequest.getDeltaCapacity());
{code}
-  I have a question about the API semantics for the above mentioned scenario. According to the AMRMClient#requestContainerResourceChange API, the previous pending resource-change-request should be cancelled.  Essentially, the semantics is a setter API.  In that sense, the previous 8G should be cancelled. With this approach, both resource-change-requests are cancelled. That is,10 min later after the expiration is triggered, user will suddenly see its container decreased to 2 GB. will this confuse the user ?
- revert format only changes in RMContainerChangeResourceEvent




was (Author: jianhe):
{code}
    SchedContainerChangeRequest decreaseRequest =
        new SchedContainerChangeRequest(
            schedulerNode, rmContainer,
                rmContainer.getLastConfirmedResource());
    decreaseContainer(decreaseRequest,
        getCurrentAttemptForContainer(containerId));
{code}
- this scenario may cause resource accounting wrong,  correct me if I'm wrong:
1) AM asks increase 2G -> 8G
2) AM does not increase the container, and asks decrease to 1G
3) LastConfirmedResource becomes 1G
4) In the meantime, containerIncreaseExpiration logic is triggered and rollbackContainerResource is invoked. In this case the resource Delta becomes positive even in decrease case, but some code is assuming the decrease to be negative, which may cause resource accounting wrong ?
{code}
    // Delta capacity is negative when it's a decrease request
    Resource absDelta = Resources.negate(decreaseRequest.getDeltaCapacity());
{code}
-  I have a question about the API semantics for the above mentioned scenario. According to the AMRMClient#requestContainerResourceChange API, the previous pending resource-change-request should be cancelled.  Essentially, the semantics is a setter API.  In that sense, the previous 8G should be cancelled. With this approach, both resource-change-requests are cancelled. That is,10 min later after the expiration is triggered, user will suddenly see its container decreased to 2 GB. will this confuse the user ?
- revert  RMContainerChangeResourceEvent



> Roll back container resource allocation after resource increase token expires
> -----------------------------------------------------------------------------
>
>                 Key: YARN-4138
>                 URL: https://issues.apache.org/jira/browse/YARN-4138
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api, nodemanager, resourcemanager
>            Reporter: MENG DING
>            Assignee: MENG DING
>         Attachments: YARN-4138-YARN-1197.1.patch, YARN-4138-YARN-1197.2.patch
>
>
> In YARN-1651, after container resource increase token expires, the running container is killed.
> This ticket will change the behavior such that when a container resource increase token expires, the resource allocation of the container will be reverted back to the value before the increase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)