You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Vinod Kumar Vavilapalli (Jira)" <ji...@apache.org> on 2019/11/13 09:27:00 UTC
[jira] [Commented] (YARN-9925) CapacitySchedulerQueueManager allows
unsupported Queue hierarchy
[ https://issues.apache.org/jira/browse/YARN-9925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16973167#comment-16973167 ]
Vinod Kumar Vavilapalli commented on YARN-9925:
-----------------------------------------------
[~prabhujoseph], can you add a negative unit-test to make sure this remains disallowed?
[~manirajv06@gmail.com] / [~tarunparimi], any more objections to this?
> CapacitySchedulerQueueManager allows unsupported Queue hierarchy
> ----------------------------------------------------------------
>
> Key: YARN-9925
> URL: https://issues.apache.org/jira/browse/YARN-9925
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler
> Affects Versions: 3.3.0
> Reporter: Prabhu Joseph
> Assignee: Prabhu Joseph
> Priority: Major
> Attachments: YARN-9925-001.patch, YARN-9925-002.patch, YARN-9925-003.patch
>
>
> CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When creating a queue with same name as an existing parent queue name - it has to fail with below.
> {code:java}
> Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after refresh, which is not allowed.Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after refresh, which is not allowed. at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) ... 70 more
> {code}
> In Some cases, the error is not thrown while creating the queue but thrown at submission of job "Failed to submit application_1571677375269_0002 to YARN : Application application_1571677375269_0002 submitted by user : systest to non-leaf queue : B"
> Below scenarios are allowed but it should not
> {code:java}
> It allows root.A.A1.B when root.B.B1 already exists.
>
> 1. Add root.A
> 2. Add root.A.A1
> 3. Add root.B
> 4. Add root.B.B1
> 5. Allows Add of root.A.A1.B
> It allows two root queues:
>
> 1. Add root.A
> 2. Add root.B
> 3. Add root.A.A1
> 4. Allows Add of root.A.A1.root
>
> {code}
> Below scenario is handled properly:
> {code:java}
> It does not allow root.B.A when root.A.A1 already exists.
>
> 1. Add root.A
> 2. Add root.B
> 3. Add root.A.A1
> 4. Does not Allow Add of root.B.A
> {code}
> This error handling has to be consistent in all scenarios.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org