You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ranger.apache.org by Arvind S <ar...@gmail.com> on 2015/08/03 10:56:09 UTC
Re: issue with Group permissions in Ranger
looking into this further i got to this article ..
http://hortonworks.com/blog/hadoop-groupmapping-ldap-integration/
which mentions that groups need to be managed at LDAP/AD side .. for
assertion .. is this still a valid scenario with ranger in play?
Does this mean groups are to come exclusively from LDAP/AD for both the
service users (hdfs,hive ..etc) and user defined groups?
or is there a mechanism to fall back to linux level groups if they are not
in LDAP/AD ?
*Cheers !!*
Arvind
On Fri, Jul 31, 2015 at 1:32 PM, Loïc Chanel <lo...@telecomnancy.net>
wrote:
> Hi,
>
> I experienced that issue too. Most of the time, this problem is related to
> the identity assertion of the user on the NameNode. Actually, Ranger plugin
> for HDFS is deployed on the NameNode, and therefore the user you try to
> define policies for must be able to be fully recognized by HDFS on this
> machine.
> To be sure that its groups are recognized by HDFS, I highly recommend you
> try to make a hdfs groups on the NameNode and see if the groups your are
> trying to make policies with are recognized by Hadoop.
>
> Hope this helps,
> Regards,
>
>
> Loïc
>
> Loïc CHANEL
> Engineering student at TELECOM Nancy
> Trainee at Worldline - Villeurbanne
>
> 2015-07-31 9:43 GMT+02:00 Bradman, Dale <da...@capgemini.com>:
>
>> I too have experienced this issue with Ranger 0.4. Assigning policies to
>> groups does not work!! Instead you have to assign policies to each
>> individual user. Is there a fix for this?
>>
>> Thanks
>>
>>
>> -------- Original message --------
>> From: Arvind S <ar...@gmail.com>
>> Date: 31/07/2015 07:45 (GMT+00:00)
>> To: user@ranger.incubator.apache.org
>> Subject: issue with Group permissions in Ranger
>>
>> hi .
>>> I have configured Ranger (4.0) on my 4 node node HDP 2.2.6 cluster.
>>> User sync and validation is through windows AD (2008).
>>>
>>> ----WHAT WORKS-----
>>> Designated users and corresponding groups are successfully sync'd into
>>> ranger admin. I am able to login with my AD id/pass into ranger as user.. i
>>> also see appropriate groups associated to each user.
>>>
>>> ---- ISSUES ------
>>> When i assign HDFS policy to groups the same is not effective and gives
>>> me access denied on the resource. But if same policy is assigned to the
>>> users directly then i am able to access the resource. Does any one have
>>> hints to help on this?
>>>
>>> in addition to this .. while AD imported users are marked as "external"
>>> ..the groups are getting marked as "internal" . ...
>>>
>>> Ranger admin/ portal access logs are not helping much ..
>>>
>>> *Thanks in Advance !!*
>>> Arvind
>>>
>>
>>
>> ------------------------------
>>
>> Capgemini is a trading name used by the Capgemini Group of companies
>> which includes Capgemini UK plc, a company registered in England and Wales
>> (number 943935) whose registered office is at No. 1, Forge End, Woking,
>> Surrey, GU21 6DB.
>> This message contains information that may be privileged or confidential
>> and is the property of the Capgemini Group. It is intended only for the
>> person to whom it is addressed. If you are not the intended recipient, you
>> are not authorized to read, print, retain, copy, disseminate, distribute,
>> or use this message or any part thereof. If you receive this message in
>> error, please notify the sender immediately and delete all copies of this
>> message.
>>
>
>
Re: issue with Group permissions in Ranger
Posted by Loïc Chanel <lo...@telecomnancy.net>.
I am not very familiar with the precise configuration of SSSD and NSSwitch,
but I can try to learn more and make some documentation, as this is how my
company synchronizes users and their groups from the central LDAP for each
machine.
Still, I am not sure this will be quite useful as Hadoop provides its own
way to synchronize user groups directly from LDAP with Hadoop groupmapping.
Anyway, tell me if this would help, and where to put the documentation.
Regards,
Loïc
Loïc CHANEL
Engineering student at TELECOM Nancy
Trainee at Worldline - Villeurbanne
2015-08-03 19:13 GMT+02:00 Don Bosco Durai <bo...@apache.org>:
> Loïc is correct on both counts.
>
> Ranger UserSync is just to import the users/groups into Ranger so that it
> is easy to create the policy. You still have to do the second step for
> Hadoop group level security to work.
>
> For Hadoop components to use the AD/LDAP group mapping, we need to map it
> in core-site.xml (or sync via SSSD/Centrify).
>
> @Arvind, regarding the blog, I think some of the links are broken.
> Probably, we should update it.
>
> @Aneela, I know you recently did the group mapping using the same blog.
> Any feedback you can share with us?
>
> @Loïc, can we write a small write up on in the Ranger Wiki for using SSSD?
> I think this is a good way for users to use LDAP/AD within Hadoop.
>
> Thanks
>
> Bosco
>
>
> From: Loïc Chanel <lo...@telecomnancy.net>
> Reply-To: "user@ranger.incubator.apache.org" <
> user@ranger.incubator.apache.org>
> Date: Monday, August 3, 2015 at 2:29 AM
> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
> Subject: Re: issue with Group permissions in Ranger
>
> As far as I can tell, there are two distinct mechanisms :
> - Ranger UserSync, which pulls users from a LDAP in order to create and
> manage more easily security policies for Hadoop
> - Hadoop groupmapping, which pulls users from a LDAP in order to assert
> group identity when using Hadoop, which will be used to enforce group
> policies.
> As my users and their group are synchronized from a LDAP via SSSD, I never
> tested Hadoop groupmapping, but I can tell you that external Unix users are
> found by Ranger usersync, as they belong to the Unix platform it is
> deployed on.
>
> Hope this helps,
> Regards,
>
>
> Loïc
>
> Loïc CHANEL
> Engineering student at TELECOM Nancy
> Trainee at Worldline - Villeurbanne
>
> 2015-08-03 10:56 GMT+02:00 Arvind S <ar...@gmail.com>:
>
>> looking into this further i got to this article ..
>> http://hortonworks.com/blog/hadoop-groupmapping-ldap-integration/
>> which mentions that groups need to be managed at LDAP/AD side .. for
>> assertion .. is this still a valid scenario with ranger in play?
>>
>> Does this mean groups are to come exclusively from LDAP/AD for both the
>> service users (hdfs,hive ..etc) and user defined groups?
>> or is there a mechanism to fall back to linux level groups if they are
>> not in LDAP/AD ?
>>
>> *Cheers !!*
>> Arvind
>>
>> On Fri, Jul 31, 2015 at 1:32 PM, Loïc Chanel <
>> loic.chanel@telecomnancy.net> wrote:
>>
>>> Hi,
>>>
>>> I experienced that issue too. Most of the time, this problem is related
>>> to the identity assertion of the user on the NameNode. Actually, Ranger
>>> plugin for HDFS is deployed on the NameNode, and therefore the user you try
>>> to define policies for must be able to be fully recognized by HDFS on this
>>> machine.
>>> To be sure that its groups are recognized by HDFS, I highly recommend
>>> you try to make a hdfs groups on the NameNode and see if the groups your
>>> are trying to make policies with are recognized by Hadoop.
>>>
>>> Hope this helps,
>>> Regards,
>>>
>>>
>>> Loïc
>>>
>>> Loïc CHANEL
>>> Engineering student at TELECOM Nancy
>>> Trainee at Worldline - Villeurbanne
>>>
>>> 2015-07-31 9:43 GMT+02:00 Bradman, Dale <da...@capgemini.com>:
>>>
>>>> I too have experienced this issue with Ranger 0.4. Assigning policies
>>>> to groups does not work!! Instead you have to assign policies to each
>>>> individual user. Is there a fix for this?
>>>>
>>>> Thanks
>>>>
>>>>
>>>> -------- Original message --------
>>>> From: Arvind S <ar...@gmail.com>
>>>> Date: 31/07/2015 07:45 (GMT+00:00)
>>>> To: user@ranger.incubator.apache.org
>>>> Subject: issue with Group permissions in Ranger
>>>>
>>>> hi .
>>>>> I have configured Ranger (4.0) on my 4 node node HDP 2.2.6 cluster.
>>>>> User sync and validation is through windows AD (2008).
>>>>>
>>>>> ----WHAT WORKS-----
>>>>> Designated users and corresponding groups are successfully sync'd into
>>>>> ranger admin. I am able to login with my AD id/pass into ranger as user.. i
>>>>> also see appropriate groups associated to each user.
>>>>>
>>>>> ---- ISSUES ------
>>>>> When i assign HDFS policy to groups the same is not effective and
>>>>> gives me access denied on the resource. But if same policy is assigned to
>>>>> the users directly then i am able to access the resource. Does any one have
>>>>> hints to help on this?
>>>>>
>>>>> in addition to this .. while AD imported users are marked as
>>>>> "external" ..the groups are getting marked as "internal" . ...
>>>>>
>>>>> Ranger admin/ portal access logs are not helping much ..
>>>>>
>>>>> *Thanks in Advance !!*
>>>>> Arvind
>>>>>
>>>>
>>>>
>>>> ------------------------------
>>>>
>>>> Capgemini is a trading name used by the Capgemini Group of companies
>>>> which includes Capgemini UK plc, a company registered in England and Wales
>>>> (number 943935) whose registered office is at No. 1, Forge End, Woking,
>>>> Surrey, GU21 6DB.
>>>> This message contains information that may be privileged or
>>>> confidential and is the property of the Capgemini Group. It is intended
>>>> only for the person to whom it is addressed. If you are not the intended
>>>> recipient, you are not authorized to read, print, retain, copy,
>>>> disseminate, distribute, or use this message or any part thereof. If you
>>>> receive this message in error, please notify the sender immediately and
>>>> delete all copies of this message.
>>>>
>>>
>>>
>>
>
Re: issue with Group permissions in Ranger
Posted by Don Bosco Durai <bo...@apache.org>.
Loïc is correct on both counts.
Ranger UserSync is just to import the users/groups into Ranger so that it is
easy to create the policy. You still have to do the second step for Hadoop
group level security to work.
For Hadoop components to use the AD/LDAP group mapping, we need to map it in
core-site.xml (or sync via SSSD/Centrify).
@Arvind, regarding the blog, I think some of the links are broken. Probably,
we should update it.
@Aneela, I know you recently did the group mapping using the same blog. Any
feedback you can share with us?
@Loïc, can we write a small write up on in the Ranger Wiki for using SSSD? I
think this is a good way for users to use LDAP/AD within Hadoop.
Thanks
Bosco
From: Loïc Chanel <lo...@telecomnancy.net>
Reply-To: "user@ranger.incubator.apache.org"
<us...@ranger.incubator.apache.org>
Date: Monday, August 3, 2015 at 2:29 AM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: issue with Group permissions in Ranger
> As far as I can tell, there are two distinct mechanisms :
> - Ranger UserSync, which pulls users from a LDAP in order to create and manage
> more easily security policies for Hadoop
> - Hadoop groupmapping, which pulls users from a LDAP in order to assert group
> identity when using Hadoop, which will be used to enforce group policies.
> As my users and their group are synchronized from a LDAP via SSSD, I never
> tested Hadoop groupmapping, but I can tell you that external Unix users are
> found by Ranger usersync, as they belong to the Unix platform it is deployed
> on.
>
> Hope this helps,
> Regards,
>
>
> Loïc
>
> Loïc CHANEL
> Engineering student at TELECOM Nancy
> Trainee at Worldline - Villeurbanne
>
> 2015-08-03 10:56 GMT+02:00 Arvind S <ar...@gmail.com>:
>> looking into this further i got to this article ..
>> http://hortonworks.com/blog/hadoop-groupmapping-ldap-integration/
>> which mentions that groups need to be managed at LDAP/AD side .. for
>> assertion .. is this still a valid scenario with ranger in play?
>>
>> Does this mean groups are to come exclusively from LDAP/AD for both the
>> service users (hdfs,hive ..etc) and user defined groups?
>> or is there a mechanism to fall back to linux level groups if they are not in
>> LDAP/AD ?
>>
>> Cheers !!
>> Arvind
>>
>> On Fri, Jul 31, 2015 at 1:32 PM, Loïc Chanel <lo...@telecomnancy.net>
>> wrote:
>>> Hi,
>>>
>>> I experienced that issue too. Most of the time, this problem is related to
>>> the identity assertion of the user on the NameNode. Actually, Ranger plugin
>>> for HDFS is deployed on the NameNode, and therefore the user you try to
>>> define policies for must be able to be fully recognized by HDFS on this
>>> machine.
>>> To be sure that its groups are recognized by HDFS, I highly recommend you
>>> try to make a hdfs groups on the NameNode and see if the groups your are
>>> trying to make policies with are recognized by Hadoop.
>>>
>>> Hope this helps,
>>> Regards,
>>>
>>>
>>> Loïc
>>>
>>> Loïc CHANEL
>>> Engineering student at TELECOM Nancy
>>> Trainee at Worldline - Villeurbanne
>>>
>>> 2015-07-31 9:43 GMT+02:00 Bradman, Dale <da...@capgemini.com>:
>>>> I too have experienced this issue with Ranger 0.4. Assigning policies to
>>>> groups does not work!! Instead you have to assign policies to each
>>>> individual user. Is there a fix for this?
>>>>
>>>> Thanks
>>>>
>>>>
>>>> -------- Original message --------
>>>> From: Arvind S <ar...@gmail.com>
>>>> Date: 31/07/2015 07:45 (GMT+00:00)
>>>> To: user@ranger.incubator.apache.org
>>>> Subject: issue with Group permissions in Ranger
>>>>
>>>>> hi .
>>>>> I have configured Ranger (4.0) on my 4 node node HDP 2.2.6 cluster.
>>>>> User sync and validation is through windows AD (2008).
>>>>>
>>>>> ----WHAT WORKS-----
>>>>> Designated users and corresponding groups are successfully sync'd into
>>>>> ranger admin. I am able to login with my AD id/pass into ranger as user..
>>>>> i also see appropriate groups associated to each user.
>>>>>
>>>>> ---- ISSUES ------
>>>>> When i assign HDFS policy to groups the same is not effective and gives me
>>>>> access denied on the resource. But if same policy is assigned to the users
>>>>> directly then i am able to access the resource. Does any one have hints to
>>>>> help on this?
>>>>>
>>>>> in addition to this .. while AD imported users are marked as "external"
>>>>> ..the groups are getting marked as "internal" . ...
>>>>>
>>>>> Ranger admin/ portal access logs are not helping much ..
>>>>>
>>>>> Thanks in Advance !!
>>>>> Arvind
>>>>
>>>>
>>>>
>>>>
>>>> Capgemini is a trading name used by the Capgemini Group of companies which
>>>> includes Capgemini UK plc, a company registered in England and Wales
>>>> (number 943935) whose registered office is at No. 1, Forge End, Woking,
>>>> Surrey, GU21 6DB.
>>>> This message contains information that may be privileged or confidential
>>>> and is the property of the Capgemini Group. It is intended only for the
>>>> person to whom it is addressed. If you are not the intended recipient, you
>>>> are not authorized to read, print, retain, copy, disseminate, distribute,
>>>> or use this message or any part thereof. If you receive this message in
>>>> error, please notify the sender immediately and delete all copies of this
>>>> message.
>>>
>>
>
Re: issue with Group permissions in Ranger
Posted by Loïc Chanel <lo...@telecomnancy.net>.
As far as I can tell, there are two distinct mechanisms :
- Ranger UserSync, which pulls users from a LDAP in order to create and
manage more easily security policies for Hadoop
- Hadoop groupmapping, which pulls users from a LDAP in order to assert
group identity when using Hadoop, which will be used to enforce group
policies.
As my users and their group are synchronized from a LDAP via SSSD, I never
tested Hadoop groupmapping, but I can tell you that external Unix users are
found by Ranger usersync, as they belong to the Unix platform it is
deployed on.
Hope this helps,
Regards,
Loïc
Loïc CHANEL
Engineering student at TELECOM Nancy
Trainee at Worldline - Villeurbanne
2015-08-03 10:56 GMT+02:00 Arvind S <ar...@gmail.com>:
> looking into this further i got to this article ..
> http://hortonworks.com/blog/hadoop-groupmapping-ldap-integration/
> which mentions that groups need to be managed at LDAP/AD side .. for
> assertion .. is this still a valid scenario with ranger in play?
>
> Does this mean groups are to come exclusively from LDAP/AD for both the
> service users (hdfs,hive ..etc) and user defined groups?
> or is there a mechanism to fall back to linux level groups if they are not
> in LDAP/AD ?
>
> *Cheers !!*
> Arvind
>
> On Fri, Jul 31, 2015 at 1:32 PM, Loïc Chanel <loic.chanel@telecomnancy.net
> > wrote:
>
>> Hi,
>>
>> I experienced that issue too. Most of the time, this problem is related
>> to the identity assertion of the user on the NameNode. Actually, Ranger
>> plugin for HDFS is deployed on the NameNode, and therefore the user you try
>> to define policies for must be able to be fully recognized by HDFS on this
>> machine.
>> To be sure that its groups are recognized by HDFS, I highly recommend you
>> try to make a hdfs groups on the NameNode and see if the groups your are
>> trying to make policies with are recognized by Hadoop.
>>
>> Hope this helps,
>> Regards,
>>
>>
>> Loïc
>>
>> Loïc CHANEL
>> Engineering student at TELECOM Nancy
>> Trainee at Worldline - Villeurbanne
>>
>> 2015-07-31 9:43 GMT+02:00 Bradman, Dale <da...@capgemini.com>:
>>
>>> I too have experienced this issue with Ranger 0.4. Assigning policies to
>>> groups does not work!! Instead you have to assign policies to each
>>> individual user. Is there a fix for this?
>>>
>>> Thanks
>>>
>>>
>>> -------- Original message --------
>>> From: Arvind S <ar...@gmail.com>
>>> Date: 31/07/2015 07:45 (GMT+00:00)
>>> To: user@ranger.incubator.apache.org
>>> Subject: issue with Group permissions in Ranger
>>>
>>> hi .
>>>> I have configured Ranger (4.0) on my 4 node node HDP 2.2.6 cluster.
>>>> User sync and validation is through windows AD (2008).
>>>>
>>>> ----WHAT WORKS-----
>>>> Designated users and corresponding groups are successfully sync'd into
>>>> ranger admin. I am able to login with my AD id/pass into ranger as user.. i
>>>> also see appropriate groups associated to each user.
>>>>
>>>> ---- ISSUES ------
>>>> When i assign HDFS policy to groups the same is not effective and gives
>>>> me access denied on the resource. But if same policy is assigned to the
>>>> users directly then i am able to access the resource. Does any one have
>>>> hints to help on this?
>>>>
>>>> in addition to this .. while AD imported users are marked as "external"
>>>> ..the groups are getting marked as "internal" . ...
>>>>
>>>> Ranger admin/ portal access logs are not helping much ..
>>>>
>>>> *Thanks in Advance !!*
>>>> Arvind
>>>>
>>>
>>>
>>> ------------------------------
>>>
>>> Capgemini is a trading name used by the Capgemini Group of companies
>>> which includes Capgemini UK plc, a company registered in England and Wales
>>> (number 943935) whose registered office is at No. 1, Forge End, Woking,
>>> Surrey, GU21 6DB.
>>> This message contains information that may be privileged or confidential
>>> and is the property of the Capgemini Group. It is intended only for the
>>> person to whom it is addressed. If you are not the intended recipient, you
>>> are not authorized to read, print, retain, copy, disseminate, distribute,
>>> or use this message or any part thereof. If you receive this message in
>>> error, please notify the sender immediately and delete all copies of this
>>> message.
>>>
>>
>>
>