[jira] [Commented] (YARN-10935) AM Total Queue Limit goes below per-user AM Limit if parent is full.

["Eric Payne (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-10935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17411398#comment-17411398 ] 

Eric Payne commented on YARN-10935:
-----------------------------------

For example, In the following screenshot, the advertising queue is a child of root and a parent of 3 sub-queues. One of the sub-queues has consumed all of the advertising parent queue's resources. The second sub-queue has submitted two apps. One of them is schedulable and one is non-schedulable. The second app is non-schedulable because starting the app would put the queue above the queue's AM limit:

 !Screen Shot 2021-09-07 at 12.49.52 PM.png! 

See that the second app can't start because of the following:

 !Screen Shot 2021-09-07 at 12.55.37 PM.png! 
Note that, in this example, the max queue AM limit should never go below 2GB memory and 16 vCores.


> AM Total Queue Limit goes below per-user AM Limit if parent is full.
> --------------------------------------------------------------------
>
>                 Key: YARN-10935
>                 URL: https://issues.apache.org/jira/browse/YARN-10935
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacity scheduler, capacityscheduler
>            Reporter: Eric Payne
>            Assignee: Eric Payne
>            Priority: Major
>         Attachments: Screen Shot 2021-09-07 at 12.49.52 PM.png, Screen Shot 2021-09-07 at 12.55.37 PM.png
>
>
> This happens when DRF is enabled and all of one resource is consumed but the second resources still has plenty available.
> This is reproduceable by setting up a parent queue where the capacity and max capacity are the same, with 2 or more sub-queues whose max capacity is 100%.
> In one of the sub-queues, start a long-running app that consumes all resources in the parent queue's hieararchy. This app will consume all of the memory but not vary many vcores (for example)
> In a second queue, submit an app. The *{{Max Application Master Resources Per User}}* limit is much more than the *{{Max Application Master Resources}}* limit.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org