[jira] [Commented] (YARN-3305) AM-Used Resource for leafqueue is wrongly populated if AM ResourceRequest is less than minimumAllocation

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-3305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15046033#comment-15046033 ] 

ASF GitHub Bot commented on YARN-3305:
--------------------------------------

Github user smarella commented on the pull request:

    https://github.com/apache/hadoop/commit/968425e9f7b850ff9c2ab8ca37a64c3fdbe77dbf#commitcomment-14835835
  
    In hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java:
    In hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java on line 397:
    YARN-3305 seem to break the Myriad builds (see below). As indicated in YARN-3305, https://issues.apache.org/jira/browse/YARN-3996 is trying to fix the problem. YARN-3996 is currently work-in-progress, but we need to back port some of it immediately to unblock Myriad. Once YARN-3996 is fully fixed, it needs to be fully back ported. I've requested Sarjeet to raise two bugs for this (one to provide an immediate fix and second to back port YARN-3996 once it's resolved).
    
    Currently, the latest Myriad builds are effected by this change. The problem is that, the AM container resources are normalized during app submission with ```scheduler.getminimumResourceCapability()```, which comes from yarn.scheduler.minimum-allocation-{mb,vcores,disks}. These values are set to "0" by Myriad. Hence, the AM container allocated will be of zero size and is killed by NM as soon as it is launched.
    
    The reason why Myriad needs yarn.scheduler.minimum-allocation-{mb,vcores,disks} to be 0 is for discussed in https://issues.apache.org/jira/browse/MYRIAD-139.


> AM-Used Resource for leafqueue is wrongly populated if AM ResourceRequest is less than minimumAllocation
> --------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-3305
>                 URL: https://issues.apache.org/jira/browse/YARN-3305
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 2.6.0
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>             Fix For: 2.8.0
>
>         Attachments: 0001-YARN-3305.patch, 0001-YARN-3305.patch, 0002-YARN-3305.patch, 0003-YARN-3305.patch
>
>
> For given any ResourceRequest, {{CS#allocate}} normalizes request to minimumAllocation if requested memory is less than minimumAllocation.
> But AM-used resource is updated with actual ResourceRequest made by user. This results in AM container allocation more than Max ApplicationMaster Resource.
> This is because AM-Used is updated with actual ResourceRequest made by user while activating the applications. But during allocation of container, ResourceRequest is normalized to minimumAllocation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)