[jira] [Updated] (YARN-8720) CapacityScheduler does not enforce yarn.scheduler.capacity..maximum-allocation-mb/vcores when configured

["Tarun Parimi (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tarun Parimi updated YARN-8720:
-------------------------------
    Attachment: YARN-8720.001.patch

> CapacityScheduler does not enforce yarn.scheduler.capacity.<queue-path>.maximum-allocation-mb/vcores when configured
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-8720
>                 URL: https://issues.apache.org/jira/browse/YARN-8720
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler, capacityscheduler, resourcemanager
>    Affects Versions: 2.7.0
>            Reporter: Tarun Parimi
>            Assignee: Tarun Parimi
>            Priority: Major
>         Attachments: YARN-8720.001.patch
>
>
> The value of yarn.scheduler.capacity.<queue-path>.maximum-allocation-mb/vcores is not strictly enforced when applications request containers. An InvalidResourceRequestException is thrown only when the ResourceRequest is greater than the global value of yarn.scheduler.maximum-allocation-mb/vcores . So for an example configuration such as below,
>  
> {code:java}
> yarn.scheduler.maximum-allocation-mb=4096
> yarn.scheduler.capacity.root.test.maximum-allocation-mb=2048
> {code}
>  
> The below DSShell command runs successfully and asks an AM container of size 4096MB which is greater than max 2048MB configured in test queue.
> {code:java}
> yarn jar $YARN_HOME/hadoop-yarn-applications-distributedshell.jar -num_containers 1 -jar $YARN_HOME/hadoop-yarn-applications-distributedshell.jar -shell_command "sleep 60" -container_memory=4096 -master_memory=4096 -queue=test{code}
> Instead it should not launch the application and fail with InvalidResourceRequestException . The child container however will be requested with size 2048MB as DSShell AppMaster does the below check before ResourceRequest ask with RM.
> {code:java}
> // A resource ask cannot exceed the max.
> if (containerMemory > maxMem) {
>  LOG.info("Container memory specified above max threshold of cluster."
>  + " Using max value." + ", specified=" + containerMemory + ", max="
>  + maxMem);
>  containerMemory = maxMem;
> }{code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org