[jira] [Created] (YARN-11069) Dynamic Queue ACL handling in Legacy and Flexible Auto Created Queues

["Tamas Domok (Jira)" <ji...@apache.org>]

Tamas Domok created YARN-11069:
----------------------------------

             Summary: Dynamic Queue ACL handling in Legacy and Flexible Auto Created Queues
                 Key: YARN-11069
                 URL: https://issues.apache.org/jira/browse/YARN-11069
             Project: Hadoop YARN
          Issue Type: New Feature
          Components: capacity scheduler, yarn
            Reporter: Tamas Domok
            Assignee: Tamas Domok


During the investigation YARN-10922 I found multiple issues with dynamic ACL handling, no wonder it was documented as unsupported feature. But in some cases it did work and some users utilised that behaviour.

There is a severe bug when using Flexible AQC and dynamic parent queues, *anybody* can submit an app regardless of what ACLs are set (when the dynamic parent queue is not exists yet) - see: YARN-11066.

After this dynamic ACL feature YARN-11060 should be fixed as well.

Examples scenarios:

Legacy AQC:
{code}
yarn.scheduler.capacity.root.managed.auto-create-child-queue.enabled: true

yarn.scheduler.capacity.root.managed.acl_submit_applications: nobody
yarn.scheduler.capacity.root.managed.acl_administer_queue: nobody

yarn.scheduler.capacity.root.managed.acl_submit_applications: user1
yarn.scheduler.capacity.root.managed.acl_administer_queue: admin1

yarn.scheduler.capacity.root.managed.leaf-queue-template.acl_submit_applications: user2
yarn.scheduler.capacity.root.managed.leaf-queue-template.acl_administer_queue: admin2
{code}

user2 can submit an application to root.managed.user2 even if the queue doesn't exist yet due to template

Permissions:
 - root: {nobody, nobody}
 - root.managed: {user1, admin1}
 - root.managed.auto: {user2, admin2}

Flexible AQC:
{code}
yarn.scheduler.capacity.root.auto-queue-creation-v2.enabled: true

yarn.scheduler.capacity.root.acl_submit_applications: user
yarn.scheduler.capacity.root.acl_administer_queue: admin

yarn.scheduler.capacity.root.auto-queue-creation-v2.parent-template.acl_submit_applications: parentUser1
yarn.scheduler.capacity.root.auto-queue-creation-v2.parent-template.acl_administer_queue: parentAdmin1

yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_submit_applications: user1
yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_administer_queue: admin1
{code}

user1 can submit an application to root.autoParent.user1 even if the queue doesn't exist yet due to template

Permissions:
 - root: {user, admin}
 - root.autoParent: {parentUser1, parentAdmin1}
 - root.autoParent.autoLeaf: {user1, admin1}

Flexible AQC with overwritten leaf:
{code}
yarn.scheduler.capacity.root.auto-queue-creation-v2.enabled: true

yarn.scheduler.capacity.root.acl_submit_applications: user
yarn.scheduler.capacity.root.acl_administer_queue: admin

yarn.scheduler.capacity.root.auto-queue-creation-v2.template.acl_submit_applications: commonUser1
yarn.scheduler.capacity.root.auto-queue-creation-v2.template.acl_administer_queue: commonUser2

yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_submit_applications: user1
yarn.scheduler.capacity.root.*.auto-queue-creation-v2.leaf-template.acl_administer_queue: admin1
{code}

user1 can submit an application to root.autoParent.user1 even if the queue doesn't exist yet due to template

Permissions:
 - root: {user, admin}
 - root.autoParent: {commonUser1, commonUser2}
 - root.managed.autoLeaf: {user1, admin1}

NOTE: the *.leaf-template* and *.parent-template* overwrites the *.template* variant.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org