[jira] [Commented] (YARN-5295) YARN queue-mappings to check Queue is present before submitting job

["Sunil G (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15350703#comment-15350703 ] 

Sunil G commented on YARN-5295:
-------------------------------

Hi [~Prabhu Joseph]
Thanks for raising this issue. Couple of doubts here,

If we have {{u:%user:%user}} and {{g:edw:platform}},  and assume *test* is not present in cluster. But *test* user is still a part of *edw* group. So we have to submit to *platform* queue.
If *test* queue was present, this app would have got submitted to *test* itself. Am I correct?

Being said this, we already do a set of checks in CS (I am mentioning 2.7.2 and trunk)
In trunk, we have {{PlacementManager}} which will do the check before the app itself is submitted to scheduler. Ref YARN-3635. Also if we use ACLs, we will have that check also done in {{RMAppManager#submitApplication}} itself.
In 2.7.2,  this check is done in CS itself. So application was added to scheduler and then we will get an APP_REJECTED event.

So I think we do check for queue now, but we are not checking whether its a leaf queue. And ACLS are already handled in trunk. Could you pls help to correct if i understood the problem differently.

> YARN queue-mappings to check Queue is present before submitting job
> -------------------------------------------------------------------
>
>                 Key: YARN-5295
>                 URL: https://issues.apache.org/jira/browse/YARN-5295
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler
>    Affects Versions: 2.7.2
>            Reporter: Prabhu Joseph
>
> In yarn Queue-Mappings, Yarn should check if the queue is present before submitting the job. If not present it should go to next mapping available.
> For example if we have
> yarn.scheduler.capacity.queue-mappings=u:%user:%user,g:edw:platform
> and I submit job with user "test" and if there is no "test" queue then it should check the second mapping (g:edw:platform) in the list and if test is part of edw group it should submit job in platform queue.
> Below Sanity checks has to be done for the mapped queue in the list and if it fails then the the next queue mapping has to be chosen, when there is no queue mapping passing the sanity check, only then the application has to be Rejected.
> 1. is queue present
> 2. is queue not a leaf queue
> 3. is user either have ACL Submit_Applications or Administer_Queue of the queue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org