[jira] [Comment Edited] (YARN-10922) Investigation: Verify if legacy AQC works as documented

["Tamas Domok (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-10922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17468669#comment-17468669 ] 

Tamas Domok edited comment on YARN-10922 at 1/5/22, 8:50 AM:
-------------------------------------------------------------

A pull request is on the way with a unit test for the QueueACL behaviour.

 

I was wrong on both assumption on the first comment.

 - Absolute resource can be configured for dynamically created queues with Legacy AQC, see the new test case in (YARN-11033). It is possible to configure the parent with Absolute resource and the child with Percentage mode using the leaf-queue-template, that leads to this bug (YARN-11010). 

 - QueueACL are partially supported, they can be configured for dynamically created queues with Legacy AQC leaf-queue-template. If a dynamically created queue exists already, then the ACLs configured with the leaf-queue-template applies for that queue. However at queue creation those ACLs are not in effect, that's why I call it partial support. Also the scheduler response doesn't show the ACLs in auto created queue's configuration, which is misleading.

Absolute Resource:

  Question#1: Should I report a bug for a missing config validation for the Absolute+Percentage mix?

QueueACL:

  Question#2-a: Should the documentation be updated with the QueueACLs current behaviour? IMHO it's hard to explain and I don't think this behaviour was intended.

  Question#2-b: Should I report a bug to eliminate this partial support for QueueACL? The new Flexible Auto Queue Creation doesn't support it either, that works as it is documented.

  Question#3b: Should I report bugs to improve the QueueACL support on the Legacy AQC? One for the queue creation and one for the scheduler response? IMHO if this is a Legacy feature then it would make sense to introduce this feature in the Flexible AQC instead.


was (Author: tdomok):
A pull request is on the way with doc update and a unit test for the QueueACL behaviour.

 

I was wrong on both assumption on the first comment.

 - Absolute resource can be configured for dynamically created queues with Legacy AQC, see the new test case in (YARN-11033 isAbsoluteResource is not correct for dynamically created queues). It is possible to configure the parent with Absolute resource and the child with Percentage mode using the leaf-queue-template, that leads to this bug (YARN-11010).

 - QueueACL can be configured for dynamically created queues with Legacy AQC leaf-queue-template. If a dynamically created queue exists already, then the ACLs configured with the leaf-queue-template applies for that queue. However at queue creation those ACLs are not in effect, and the scheduler response doesn't show the ACLs in the dynamic queue.

 

Probably I should open 2 more issue:

 1. ACLs are not correctly filled for the dynamically created queue.

 2. It shouldn't be possible to configure Absolute parent with Percentage child using the leaf-queue-template.

> Investigation: Verify if legacy AQC works as documented
> -------------------------------------------------------
>
>                 Key: YARN-10922
>                 URL: https://issues.apache.org/jira/browse/YARN-10922
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Szilard Nemeth
>            Assignee: Tamas Domok
>            Priority: Minor
>              Labels: pull-request-available
>         Attachments: capacity-scheduler.xml
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Quoting from the Capacity Scheduler documentation: https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html
> Section: "Dynamic Auto-Creation and Management of Leaf Queues"
> The task is to verify if legacy AQC works like this: 
> {quote}
> The parent queue which has been enabled for auto leaf queue creation, supports the configuration of template parameters for automatic configuration of the auto-created leaf queues. The auto-created queues support all of the leaf queue configuration parameters except for Queue ACL, Absolute Resource configurations. Queue ACLs are currently inherited from the parent queue i.e they are not configurable on the leaf queue template
> {quote}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org