You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "ANANDA G B (JIRA)" <ji...@apache.org> on 2019/08/09 00:31:00 UTC
[jira] [Issue Comment Deleted] (YARN-9681) AM resource limit is
incorrect for queue
[ https://issues.apache.org/jira/browse/YARN-9681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ANANDA G B updated YARN-9681:
-----------------------------
Comment: was deleted
(was: The problem we have seen is: assignContainers() invokes LeafQueue.updateCurrentResourceLimits method, where:
*Line1:* this.*{color:#59afe1}cachedResourceLimitsForHeadroom{color}* =
new ResourceLimits(currentResourceLimits.getLimit());
*Line2:* Resource queueMaxResource = getEffectiveMaxCapacityDown(
RMNodeLabelsManager.{color:#59afe1}*NO_LABEL*{color}, minimumAllocation);
*Line3:* this.{color:#59afe1}*cachedResourceLimitsForHeadroom*{color}
.setLimit(Resources._min_(resourceCalculator, clusterResource,
queueMaxResource, currentResourceLimits.getLimit()));
*In Line1:* Setting the *{color:#59afe1}cachedResourceLimitsForHeadroom{color}* {color:#333333}with {color}currentResourceLimits
Which is pool1's queue1 resource limit.
*In Line2:* Setting the queueMaxResource
Which is DEFAULT_PARTITION's queue1 resource limit.
*In Line3:* Setting *{color:#59afe1}cachedResourceLimitsForHeadroom{color}* {color:#333333}= Minimum of queueMaxResource and currentResourceLimits{color}
{color:#333333}Which is pool1's queue1 resource limit. (Look at the attached images for partitions and queues information){color}
{color:#333333}So finally, *{color:#59afe1}cachedResourceLimitsForHeadroom{color}* is setted with the resource limits of pool1's queue1. Then, this *{color:#59afe1}cachedResourceLimitsForHeadroom{color}* value is used to calculate MAX AM RESOURCE of both the partitions (DEFAULT_PARTITION's and pool1's). So MAX AM resource is calculated wrongly for DEFAULT_PARTITION's. {color}
{color:#333333}So, solution is *{color:#59afe1}cachedResourceLimitsForHeadroom{color}* must be maintained for each partitions. So it can be a map where key is partition and value is the resource limit.{color}
)
> AM resource limit is incorrect for queue
> ----------------------------------------
>
> Key: YARN-9681
> URL: https://issues.apache.org/jira/browse/YARN-9681
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
> Affects Versions: 3.1.1, 3.1.2
> Reporter: ANANDA G B
> Assignee: ANANDA G B
> Priority: Major
> Labels: patch
> Attachments: After running job on queue1.png, Before running job on queue1.png, YARN-9681.0001.patch, YARN-9681.0002.patch, YARN-9681.0003.patch, YARN-9681.0004.patch
>
>
> After running the job on Queue1 of Partition1, then Queue1 of DEFAULT_PARTITION's 'Max Application Master Resources' is calculated wrongly. Please find the attachement.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org