You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Szilard Nemeth (Jira)" <ji...@apache.org> on 2022/02/16 13:30:00 UTC
[jira] [Updated] (YARN-10396) Max applications calculation per queue disregards queue level settings in absolute mode
[ https://issues.apache.org/jira/browse/YARN-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Szilard Nemeth updated YARN-10396:
----------------------------------
Description:
Looking at the following code in {{org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.java#L1126}}
{code:java}
int maxApplications = (int) (conf.getMaximumSystemApplications()
* childQueue.getQueueCapacities().getAbsoluteCapacity(label));
leafQueue.setMaxApplications(maxApplications);{code}
In Absolute Resources mode, setting the number of maximum applications on queue level gets overridden with the system level setting scaled down to the available resources. This means that the only way to set the maximum number of applications is to change the queue's resource pool. This line should consider the queue's {{yarn.scheduler.capacity.\{queuepath}.maximum-applications }}setting.
was:
Looking at the following code in {{org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.java#L1126}}
{code:java}
int maxApplications = (int) (conf.getMaximumSystemApplications()
* childQueue.getQueueCapacities().getAbsoluteCapacity(label));
leafQueue.setMaxApplications(maxApplications);{code}
In Absolute Resources mode setting the number of maximum applications on queue level gets overridden with the system level setting scaled down to the available resources. This means that the only way to set the maximum number of applications is to change the queue's resource pool. This line should consider the queue's {{yarn.scheduler.capacity.\{queuepath}.maximum-applications }}setting.
> Max applications calculation per queue disregards queue level settings in absolute mode
> ---------------------------------------------------------------------------------------
>
> Key: YARN-10396
> URL: https://issues.apache.org/jira/browse/YARN-10396
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
> Reporter: Benjamin Teke
> Assignee: Benjamin Teke
> Priority: Major
> Fix For: 3.2.2, 3.4.0, 3.3.1
>
> Attachments: YARN-10396.001.patch, YARN-10396.002.patch, YARN-10396.003.patch, YARN-10396.branch-3.1.001.patch, YARN-10396.branch-3.2.001.patch
>
>
> Looking at the following code in {{org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.java#L1126}}
> {code:java}
> int maxApplications = (int) (conf.getMaximumSystemApplications()
> * childQueue.getQueueCapacities().getAbsoluteCapacity(label));
> leafQueue.setMaxApplications(maxApplications);{code}
> In Absolute Resources mode, setting the number of maximum applications on queue level gets overridden with the system level setting scaled down to the available resources. This means that the only way to set the maximum number of applications is to change the queue's resource pool. This line should consider the queue's {{yarn.scheduler.capacity.\{queuepath}.maximum-applications }}setting.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org