You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "zhuqi (Jira)" <ji...@apache.org> on 2021/01/11 14:11:00 UTC
[jira] [Comment Edited] (YARN-10506) Update queue creation logic to
use weight mode and allow the flexible static/dynamic creation
[ https://issues.apache.org/jira/browse/YARN-10506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262660#comment-17262660 ]
zhuqi edited comment on YARN-10506 at 1/11/21, 2:10 PM:
--------------------------------------------------------
[~wangda] [~gandras]
Add an updated YARN-10506-007-10504-010.patch for fix all test in TestCapacitySchedulerNewQueueAutoCreation.
The logic is :
In ParentQueue's addDynamicChildQueue
{code:java}
// New method to add child queue
public CSQueue addDynamicChildQueue(String childQueuePath, boolean isLeaf)
throws SchedulerDynamicEditException {
writeLock.lock();
try {
// Changed here
boolean weightsAreUsed = false;
try {
if (isLeaf && childQueues.isEmpty()) {
weightsAreUsed = true;
} else {
weightsAreUsed = getCapacityConfigurationTypeForQueues(childQueues)
== QueueCapacityType.WEIGHT;
}
} catch (IOException e) {
LOG.warn("Caught Exception during auto queue creation", e);
}
...
}{code}
Reason:
When add a dynamic leaf queue, but we still not update the weight it, when the childQueues is empty, the original getCapacityConfigurationTypeForQueues will return percentage, we should handle this case to weightsAreUsed = true. When the childQueues is not empty, we just can use getCapacityConfigurationTypeForQueues for check.
Any other test fail, may be related queueplacement.
Thanks.
was (Author: zhuqi):
[~wangda] [~gandras]
Add an updated patch for fix all test in TestCapacitySchedulerNewQueueAutoCreation.
In ParentQueue's addDynamicChildQueue
{code:java}
// New method to add child queue
public CSQueue addDynamicChildQueue(String childQueuePath, boolean isLeaf)
throws SchedulerDynamicEditException {
writeLock.lock();
try {
// Changed here
boolean weightsAreUsed = false;
try {
if (isLeaf && childQueues.isEmpty()) {
weightsAreUsed = true;
} else {
weightsAreUsed = getCapacityConfigurationTypeForQueues(childQueues)
== QueueCapacityType.WEIGHT;
}
} catch (IOException e) {
LOG.warn("Caught Exception during auto queue creation", e);
}
...
}{code}
Reason:
When add a dynamic leaf queue, but we still not update the weight it, also the childQueues is empty, when original getCapacityConfigurationTypeForQueues will return percentage, we should handle this case to weightsAreUsed = true, avoid the error.
Any other test fail, may be related queueplacement.
Thanks.
> Update queue creation logic to use weight mode and allow the flexible static/dynamic creation
> ---------------------------------------------------------------------------------------------
>
> Key: YARN-10506
> URL: https://issues.apache.org/jira/browse/YARN-10506
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Benjamin Teke
> Assignee: Andras Gyori
> Priority: Major
> Attachments: YARN-10506-006-10504-010.patch, YARN-10506-007-10504-010.patch, YARN-10506.001.patch, YARN-10506.002.patch, YARN-10506.003.patch, YARN-10506.004.patch, YARN-10506.005.patch, YARN-10506.006-combined.patch, YARN-10506.006.patch
>
>
> The queue creation logic should be updated to use weight mode and support the flexible creation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org