You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Prabhu Joseph <pr...@gmail.com> on 2016/06/30 17:50:57 UTC
Queue ACLs overridden by yarn.admin.acl
Hi All,
On Hadoop-2.7.1, Yarn CapaictyScheduler, a x user can kill the job
submitted by yarn user even though the x user does not have administer acl
on the queue. The queue -showacls does not show ADMINISTER_QUEUE on that
queue for x user but since yarn.admin.acl is *, it allows x to kill the
job. If we set yarn.admin.acl as yarn, then it works fine but which won't
allow all users to view all jobs in RM UI for secure cluster. So, how to
restrict some x user from killing other user job with yarn.admin.acl as *.
yarn.admin.acl *
yarn.acl.enable true
yarn.scheduler.capacity.root.test.acl_administer_queue=yarn,
yarn.scheduler.capacity.root.test.acl_submit_applications=*
yarn.scheduler.capacity.root.acl_administer_queue=yarn,
yarn.scheduler.capacity.root.acl_submit_applications=*
[x@spark3 root]$ hadoop queue -showacls
Queue acls for user : x
Queue Operations
=====================
root SUBMIT_APPLICATIONS
129671_test1 SUBMIT_APPLICATIONS
default SUBMIT_APPLICATIONS
Thanks,
Prabhu Joseph
Re: Queue ACLs overridden by yarn.admin.acl
Posted by Sunil Govind <su...@gmail.com>.
HI Prabhu,
Could you please check hadoop.http.filter.initializers configuration in
core-site.xml also. This may help to use web ui in secure cluster but not
CLI.
Thanks
- Sunil
On Fri, Jul 1, 2016 at 3:36 PM Prabhu Joseph <pr...@gmail.com>
wrote:
> Thanks Sunil. Yes, it would be nice to have acl_view_applications on queue
> similar to acl_administer_queue. A user can kill other user's job with
> acl_administer_queue. Similarly, a user can be made to view other user's
> job with acl_view_applications.
>
> On Fri, Jul 1, 2016 at 1:40 PM, Sunil Govind <su...@gmail.com>
> wrote:
>
>> Hi Prabhu
>>
>> If "yarn.admin.acl" is configured with "yarn", then "yarn" user can be
>> used to administer apps in cluster generally. This is handled as per admin
>> ACLs. Similarly CS also have Queue ACLs, and these ACLs can be set per
>> queue level.
>>
>> So as per you example (with some minor changes)
>> yarn.admin.acl yarn
>> yarn.acl.enable true
>>
>> yarn.scheduler.capacity.root.test.acl_administer_queue=x
>> yarn.scheduler.capacity.root.test.acl_submit_applications=*
>> yarn.scheduler.capacity.root.acl_administer_queue=yarn,
>> yarn.scheduler.capacity.root.acl_submit_applications=*
>>
>> Users "yarn" and "x" can have administer access in queue "test". Other
>> users now cannot access apps submitted from "yarn" and "x" user.
>>
>> It seems like, you are looking for a read-only user who can view all apps
>> in a queue/queues from UI or from cli. As I see it, we do not have such an
>> option yet, and seems like you are looking for "acl_view_applications"
>> mode. This comes with a complexity as we need clear separation in client
>> apis (read-only vs read-write) for VIEW ACLs. Thoughts?
>>
>>
>> Thanks
>> Sunil
>>
>> On Thu, Jun 30, 2016 at 11:21 PM Prabhu Joseph <
>> prabhujose.gates@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> On Hadoop-2.7.1, Yarn CapaictyScheduler, a x user can kill the job
>>> submitted by yarn user even though the x user does not have administer
>>> acl
>>> on the queue. The queue -showacls does not show ADMINISTER_QUEUE on that
>>> queue for x user but since yarn.admin.acl is *, it allows x to kill the
>>> job. If we set yarn.admin.acl as yarn, then it works fine but which won't
>>> allow all users to view all jobs in RM UI for secure cluster. So, how to
>>> restrict some x user from killing other user job with yarn.admin.acl as
>>> *.
>>>
>>> yarn.admin.acl *
>>> yarn.acl.enable true
>>>
>>> yarn.scheduler.capacity.root.test.acl_administer_queue=yarn,
>>> yarn.scheduler.capacity.root.test.acl_submit_applications=*
>>> yarn.scheduler.capacity.root.acl_administer_queue=yarn,
>>> yarn.scheduler.capacity.root.acl_submit_applications=*
>>>
>>>
>>> [x@spark3 root]$ hadoop queue -showacls
>>> Queue acls for user : x
>>>
>>> Queue Operations
>>> =====================
>>> root SUBMIT_APPLICATIONS
>>> 129671_test1 SUBMIT_APPLICATIONS
>>> default SUBMIT_APPLICATIONS
>>>
>>> Thanks,
>>> Prabhu Joseph
>>>
>>
>
Re: Queue ACLs overridden by yarn.admin.acl
Posted by Prabhu Joseph <pr...@gmail.com>.
Thanks Sunil. Yes, it would be nice to have acl_view_applications on queue
similar to acl_administer_queue. A user can kill other user's job with
acl_administer_queue. Similarly, a user can be made to view other user's
job with acl_view_applications.
On Fri, Jul 1, 2016 at 1:40 PM, Sunil Govind <su...@gmail.com> wrote:
> Hi Prabhu
>
> If "yarn.admin.acl" is configured with "yarn", then "yarn" user can be
> used to administer apps in cluster generally. This is handled as per admin
> ACLs. Similarly CS also have Queue ACLs, and these ACLs can be set per
> queue level.
>
> So as per you example (with some minor changes)
> yarn.admin.acl yarn
> yarn.acl.enable true
>
> yarn.scheduler.capacity.root.test.acl_administer_queue=x
> yarn.scheduler.capacity.root.test.acl_submit_applications=*
> yarn.scheduler.capacity.root.acl_administer_queue=yarn,
> yarn.scheduler.capacity.root.acl_submit_applications=*
>
> Users "yarn" and "x" can have administer access in queue "test". Other
> users now cannot access apps submitted from "yarn" and "x" user.
>
> It seems like, you are looking for a read-only user who can view all apps
> in a queue/queues from UI or from cli. As I see it, we do not have such an
> option yet, and seems like you are looking for "acl_view_applications"
> mode. This comes with a complexity as we need clear separation in client
> apis (read-only vs read-write) for VIEW ACLs. Thoughts?
>
>
> Thanks
> Sunil
>
> On Thu, Jun 30, 2016 at 11:21 PM Prabhu Joseph <pr...@gmail.com>
> wrote:
>
>> Hi All,
>>
>> On Hadoop-2.7.1, Yarn CapaictyScheduler, a x user can kill the job
>> submitted by yarn user even though the x user does not have administer acl
>> on the queue. The queue -showacls does not show ADMINISTER_QUEUE on that
>> queue for x user but since yarn.admin.acl is *, it allows x to kill the
>> job. If we set yarn.admin.acl as yarn, then it works fine but which won't
>> allow all users to view all jobs in RM UI for secure cluster. So, how to
>> restrict some x user from killing other user job with yarn.admin.acl as *.
>>
>> yarn.admin.acl *
>> yarn.acl.enable true
>>
>> yarn.scheduler.capacity.root.test.acl_administer_queue=yarn,
>> yarn.scheduler.capacity.root.test.acl_submit_applications=*
>> yarn.scheduler.capacity.root.acl_administer_queue=yarn,
>> yarn.scheduler.capacity.root.acl_submit_applications=*
>>
>>
>> [x@spark3 root]$ hadoop queue -showacls
>> Queue acls for user : x
>>
>> Queue Operations
>> =====================
>> root SUBMIT_APPLICATIONS
>> 129671_test1 SUBMIT_APPLICATIONS
>> default SUBMIT_APPLICATIONS
>>
>> Thanks,
>> Prabhu Joseph
>>
>
Re: Queue ACLs overridden by yarn.admin.acl
Posted by Sunil Govind <su...@gmail.com>.
Hi Prabhu
If "yarn.admin.acl" is configured with "yarn", then "yarn" user can be
used to administer apps in cluster generally. This is handled as per admin
ACLs. Similarly CS also have Queue ACLs, and these ACLs can be set per
queue level.
So as per you example (with some minor changes)
yarn.admin.acl yarn
yarn.acl.enable true
yarn.scheduler.capacity.root.test.acl_administer_queue=x
yarn.scheduler.capacity.root.test.acl_submit_applications=*
yarn.scheduler.capacity.root.acl_administer_queue=yarn,
yarn.scheduler.capacity.root.acl_submit_applications=*
Users "yarn" and "x" can have administer access in queue "test". Other
users now cannot access apps submitted from "yarn" and "x" user.
It seems like, you are looking for a read-only user who can view all apps
in a queue/queues from UI or from cli. As I see it, we do not have such an
option yet, and seems like you are looking for "acl_view_applications"
mode. This comes with a complexity as we need clear separation in client
apis (read-only vs read-write) for VIEW ACLs. Thoughts?
Thanks
Sunil
On Thu, Jun 30, 2016 at 11:21 PM Prabhu Joseph <pr...@gmail.com>
wrote:
> Hi All,
>
> On Hadoop-2.7.1, Yarn CapaictyScheduler, a x user can kill the job
> submitted by yarn user even though the x user does not have administer acl
> on the queue. The queue -showacls does not show ADMINISTER_QUEUE on that
> queue for x user but since yarn.admin.acl is *, it allows x to kill the
> job. If we set yarn.admin.acl as yarn, then it works fine but which won't
> allow all users to view all jobs in RM UI for secure cluster. So, how to
> restrict some x user from killing other user job with yarn.admin.acl as *.
>
> yarn.admin.acl *
> yarn.acl.enable true
>
> yarn.scheduler.capacity.root.test.acl_administer_queue=yarn,
> yarn.scheduler.capacity.root.test.acl_submit_applications=*
> yarn.scheduler.capacity.root.acl_administer_queue=yarn,
> yarn.scheduler.capacity.root.acl_submit_applications=*
>
>
> [x@spark3 root]$ hadoop queue -showacls
> Queue acls for user : x
>
> Queue Operations
> =====================
> root SUBMIT_APPLICATIONS
> 129671_test1 SUBMIT_APPLICATIONS
> default SUBMIT_APPLICATIONS
>
> Thanks,
> Prabhu Joseph
>