You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Koert Kuipers <ko...@tresata.com> on 2012/10/08 23:00:46 UTC
Secure hadoop and group permission on HDFS
With secure hadoop the user name is authenticated by the kerberos server.
But what about the groups that the user is a member of? Are these simple
the groups that the user is a member of on the namenode machine?
Is it viable to manage access to files on HDFS using groups on a secure
hadoop cluster?
RE: Secure hadoop and group permission on HDFS
Posted by "Zheng, Kai" <ka...@intel.com>.
Thanks Natty and I understand.
Also replied in common-dev@hadoop.apache.org<ma...@hadoop.apache.org> mailing list for further discussion.
From: Jonathan Natkins [mailto:natty@cloudera.com]
Sent: Friday, October 19, 2012 8:58 AM
To: user@hadoop.apache.org
Subject: Re: Secure hadoop and group permission on HDFS
Hi Kai,
1. To the best of my knowledge, you can only use one group mapping service at a time. In order to do what you're suggesting, you'd have to write a customized group mapping service.
2. Currently multiple ADs are not supported, but it's certainly an improvement that could be made.
3. The LdapGroupsMapping already supports OpenLDAP. It's pretty heavily configurable for the purpose of supporting multiple types of LDAP implementations. The defaults just happen to be geared towards Active Directory.
Thanks,
Natty
On Mon, Oct 15, 2012 at 8:07 PM, Zheng, Kai <ka...@intel.com>> wrote:
Hi Koert & Harsh,
Regarding LdapGroupsMapping, I have questions:
1. Is it possible to use ShellBasedUnixGroupsMapping for Hadoop service principals/users, and LdapGroupsMapping for end user accounts?
In our environment, normal end users (along with their groups info) for Hadoop cluster are from AD, and for them we should use the ldap mapping;
but for hdfs/mapred service principals, the default shell based one is enough, and we don't want to create the user/group entries in AD just for that.
2. Can we support multiple ADs? Hadoop users might come from more than ONE AD in big org.
3. Is there any technical issue not to support LDAPs like OpenLDAP? In my understanding, one possible difficulity might be that it's not easy to extract common
group lookup mechanism with common filters/configurations both for AD and OpenLDAP like, right?
I'm wondering if these are just limits for current implementation, and if so if we need to improve that. Might the community has already been going for that?
Thanks
Kai
From: Ivan Frain [mailto:ivan.frain@gmail.com<ma...@gmail.com>]
Sent: Tuesday, October 09, 2012 2:46 PM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Secure hadoop and group permission on HDFS
Hi Koert,
Another option is to use the LdapGroupsMapping which picks up the group membership from a LDAP directory.
You can find more details on the JIRA issue: https://issues.apache.org/jira/browse/HADOOP-8121
Up to now, it is available for ActiveDirectory and released in hadoop-2.0.0-alpha and next releases.
You can easily apply the patch on a 0.23.1, I already did that and it works well.
OpenLdap with POSIX groups is not yet supported by this patch, it was tailored for ActiveDirectory.
BR,
Ivan
2012/10/9 Harsh J <ha...@cloudera.com>>
Koert,
If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
class (via hadoop.security.group.mapping), then yes the NameNode's
view of the local unix groups (and the primary group) of the user is
the final say on what groups the user belongs to. This can be relied
on - but note that HDFS uses BSD style semantics when it comes to
groups and when creating directories/files, the parent directory
groups are inherited automatically unless altered after creation.
On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com>> wrote:
> With secure hadoop the user name is authenticated by the kerberos server.
> But what about the groups that the user is a member of? Are these simple the
> groups that the user is a member of on the namenode machine?
> Is it viable to manage access to files on HDFS using groups on a secure
> hadoop cluster?
>
--
Harsh J
--
Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07<tel:%2B33%20%280%296%2052%2052%2047%2007>
RE: Secure hadoop and group permission on HDFS
Posted by "Zheng, Kai" <ka...@intel.com>.
Thanks Natty and I understand.
Also replied in common-dev@hadoop.apache.org<ma...@hadoop.apache.org> mailing list for further discussion.
From: Jonathan Natkins [mailto:natty@cloudera.com]
Sent: Friday, October 19, 2012 8:58 AM
To: user@hadoop.apache.org
Subject: Re: Secure hadoop and group permission on HDFS
Hi Kai,
1. To the best of my knowledge, you can only use one group mapping service at a time. In order to do what you're suggesting, you'd have to write a customized group mapping service.
2. Currently multiple ADs are not supported, but it's certainly an improvement that could be made.
3. The LdapGroupsMapping already supports OpenLDAP. It's pretty heavily configurable for the purpose of supporting multiple types of LDAP implementations. The defaults just happen to be geared towards Active Directory.
Thanks,
Natty
On Mon, Oct 15, 2012 at 8:07 PM, Zheng, Kai <ka...@intel.com>> wrote:
Hi Koert & Harsh,
Regarding LdapGroupsMapping, I have questions:
1. Is it possible to use ShellBasedUnixGroupsMapping for Hadoop service principals/users, and LdapGroupsMapping for end user accounts?
In our environment, normal end users (along with their groups info) for Hadoop cluster are from AD, and for them we should use the ldap mapping;
but for hdfs/mapred service principals, the default shell based one is enough, and we don't want to create the user/group entries in AD just for that.
2. Can we support multiple ADs? Hadoop users might come from more than ONE AD in big org.
3. Is there any technical issue not to support LDAPs like OpenLDAP? In my understanding, one possible difficulity might be that it's not easy to extract common
group lookup mechanism with common filters/configurations both for AD and OpenLDAP like, right?
I'm wondering if these are just limits for current implementation, and if so if we need to improve that. Might the community has already been going for that?
Thanks
Kai
From: Ivan Frain [mailto:ivan.frain@gmail.com<ma...@gmail.com>]
Sent: Tuesday, October 09, 2012 2:46 PM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Secure hadoop and group permission on HDFS
Hi Koert,
Another option is to use the LdapGroupsMapping which picks up the group membership from a LDAP directory.
You can find more details on the JIRA issue: https://issues.apache.org/jira/browse/HADOOP-8121
Up to now, it is available for ActiveDirectory and released in hadoop-2.0.0-alpha and next releases.
You can easily apply the patch on a 0.23.1, I already did that and it works well.
OpenLdap with POSIX groups is not yet supported by this patch, it was tailored for ActiveDirectory.
BR,
Ivan
2012/10/9 Harsh J <ha...@cloudera.com>>
Koert,
If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
class (via hadoop.security.group.mapping), then yes the NameNode's
view of the local unix groups (and the primary group) of the user is
the final say on what groups the user belongs to. This can be relied
on - but note that HDFS uses BSD style semantics when it comes to
groups and when creating directories/files, the parent directory
groups are inherited automatically unless altered after creation.
On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com>> wrote:
> With secure hadoop the user name is authenticated by the kerberos server.
> But what about the groups that the user is a member of? Are these simple the
> groups that the user is a member of on the namenode machine?
> Is it viable to manage access to files on HDFS using groups on a secure
> hadoop cluster?
>
--
Harsh J
--
Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07<tel:%2B33%20%280%296%2052%2052%2047%2007>
RE: Secure hadoop and group permission on HDFS
Posted by "Zheng, Kai" <ka...@intel.com>.
Thanks Natty and I understand.
Also replied in common-dev@hadoop.apache.org<ma...@hadoop.apache.org> mailing list for further discussion.
From: Jonathan Natkins [mailto:natty@cloudera.com]
Sent: Friday, October 19, 2012 8:58 AM
To: user@hadoop.apache.org
Subject: Re: Secure hadoop and group permission on HDFS
Hi Kai,
1. To the best of my knowledge, you can only use one group mapping service at a time. In order to do what you're suggesting, you'd have to write a customized group mapping service.
2. Currently multiple ADs are not supported, but it's certainly an improvement that could be made.
3. The LdapGroupsMapping already supports OpenLDAP. It's pretty heavily configurable for the purpose of supporting multiple types of LDAP implementations. The defaults just happen to be geared towards Active Directory.
Thanks,
Natty
On Mon, Oct 15, 2012 at 8:07 PM, Zheng, Kai <ka...@intel.com>> wrote:
Hi Koert & Harsh,
Regarding LdapGroupsMapping, I have questions:
1. Is it possible to use ShellBasedUnixGroupsMapping for Hadoop service principals/users, and LdapGroupsMapping for end user accounts?
In our environment, normal end users (along with their groups info) for Hadoop cluster are from AD, and for them we should use the ldap mapping;
but for hdfs/mapred service principals, the default shell based one is enough, and we don't want to create the user/group entries in AD just for that.
2. Can we support multiple ADs? Hadoop users might come from more than ONE AD in big org.
3. Is there any technical issue not to support LDAPs like OpenLDAP? In my understanding, one possible difficulity might be that it's not easy to extract common
group lookup mechanism with common filters/configurations both for AD and OpenLDAP like, right?
I'm wondering if these are just limits for current implementation, and if so if we need to improve that. Might the community has already been going for that?
Thanks
Kai
From: Ivan Frain [mailto:ivan.frain@gmail.com<ma...@gmail.com>]
Sent: Tuesday, October 09, 2012 2:46 PM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Secure hadoop and group permission on HDFS
Hi Koert,
Another option is to use the LdapGroupsMapping which picks up the group membership from a LDAP directory.
You can find more details on the JIRA issue: https://issues.apache.org/jira/browse/HADOOP-8121
Up to now, it is available for ActiveDirectory and released in hadoop-2.0.0-alpha and next releases.
You can easily apply the patch on a 0.23.1, I already did that and it works well.
OpenLdap with POSIX groups is not yet supported by this patch, it was tailored for ActiveDirectory.
BR,
Ivan
2012/10/9 Harsh J <ha...@cloudera.com>>
Koert,
If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
class (via hadoop.security.group.mapping), then yes the NameNode's
view of the local unix groups (and the primary group) of the user is
the final say on what groups the user belongs to. This can be relied
on - but note that HDFS uses BSD style semantics when it comes to
groups and when creating directories/files, the parent directory
groups are inherited automatically unless altered after creation.
On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com>> wrote:
> With secure hadoop the user name is authenticated by the kerberos server.
> But what about the groups that the user is a member of? Are these simple the
> groups that the user is a member of on the namenode machine?
> Is it viable to manage access to files on HDFS using groups on a secure
> hadoop cluster?
>
--
Harsh J
--
Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07<tel:%2B33%20%280%296%2052%2052%2047%2007>
RE: Secure hadoop and group permission on HDFS
Posted by "Zheng, Kai" <ka...@intel.com>.
Thanks Natty and I understand.
Also replied in common-dev@hadoop.apache.org<ma...@hadoop.apache.org> mailing list for further discussion.
From: Jonathan Natkins [mailto:natty@cloudera.com]
Sent: Friday, October 19, 2012 8:58 AM
To: user@hadoop.apache.org
Subject: Re: Secure hadoop and group permission on HDFS
Hi Kai,
1. To the best of my knowledge, you can only use one group mapping service at a time. In order to do what you're suggesting, you'd have to write a customized group mapping service.
2. Currently multiple ADs are not supported, but it's certainly an improvement that could be made.
3. The LdapGroupsMapping already supports OpenLDAP. It's pretty heavily configurable for the purpose of supporting multiple types of LDAP implementations. The defaults just happen to be geared towards Active Directory.
Thanks,
Natty
On Mon, Oct 15, 2012 at 8:07 PM, Zheng, Kai <ka...@intel.com>> wrote:
Hi Koert & Harsh,
Regarding LdapGroupsMapping, I have questions:
1. Is it possible to use ShellBasedUnixGroupsMapping for Hadoop service principals/users, and LdapGroupsMapping for end user accounts?
In our environment, normal end users (along with their groups info) for Hadoop cluster are from AD, and for them we should use the ldap mapping;
but for hdfs/mapred service principals, the default shell based one is enough, and we don't want to create the user/group entries in AD just for that.
2. Can we support multiple ADs? Hadoop users might come from more than ONE AD in big org.
3. Is there any technical issue not to support LDAPs like OpenLDAP? In my understanding, one possible difficulity might be that it's not easy to extract common
group lookup mechanism with common filters/configurations both for AD and OpenLDAP like, right?
I'm wondering if these are just limits for current implementation, and if so if we need to improve that. Might the community has already been going for that?
Thanks
Kai
From: Ivan Frain [mailto:ivan.frain@gmail.com<ma...@gmail.com>]
Sent: Tuesday, October 09, 2012 2:46 PM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: Secure hadoop and group permission on HDFS
Hi Koert,
Another option is to use the LdapGroupsMapping which picks up the group membership from a LDAP directory.
You can find more details on the JIRA issue: https://issues.apache.org/jira/browse/HADOOP-8121
Up to now, it is available for ActiveDirectory and released in hadoop-2.0.0-alpha and next releases.
You can easily apply the patch on a 0.23.1, I already did that and it works well.
OpenLdap with POSIX groups is not yet supported by this patch, it was tailored for ActiveDirectory.
BR,
Ivan
2012/10/9 Harsh J <ha...@cloudera.com>>
Koert,
If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
class (via hadoop.security.group.mapping), then yes the NameNode's
view of the local unix groups (and the primary group) of the user is
the final say on what groups the user belongs to. This can be relied
on - but note that HDFS uses BSD style semantics when it comes to
groups and when creating directories/files, the parent directory
groups are inherited automatically unless altered after creation.
On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com>> wrote:
> With secure hadoop the user name is authenticated by the kerberos server.
> But what about the groups that the user is a member of? Are these simple the
> groups that the user is a member of on the namenode machine?
> Is it viable to manage access to files on HDFS using groups on a secure
> hadoop cluster?
>
--
Harsh J
--
Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07<tel:%2B33%20%280%296%2052%2052%2047%2007>
Re: Secure hadoop and group permission on HDFS
Posted by Jonathan Natkins <na...@cloudera.com>.
Hi Kai,
1. To the best of my knowledge, you can only use one group mapping service
at a time. In order to do what you're suggesting, you'd have to write a
customized group mapping service.
2. Currently multiple ADs are not supported, but it's certainly an
improvement that could be made.
3. The LdapGroupsMapping already supports OpenLDAP. It's pretty heavily
configurable for the purpose of supporting multiple types of LDAP
implementations. The defaults just happen to be geared towards Active
Directory.
Thanks,
Natty
On Mon, Oct 15, 2012 at 8:07 PM, Zheng, Kai <ka...@intel.com> wrote:
> Hi Koert & Harsh,****
>
> ** **
>
> Regarding LdapGroupsMapping, I have questions:****
>
> **1. **Is it possible to use ShellBasedUnixGroupsMapping for Hadoop
> service principals/users, and LdapGroupsMapping for end user accounts?****
>
> In our environment, normal end users (along with their groups info) for
> Hadoop cluster are from AD, and for them we should use the ldap mapping;**
> **
>
> but for hdfs/mapred service principals, the default shell based one is
> enough, and we don’t want to create the user/group entries in AD just for
> that.****
>
> **2. **Can we support multiple ADs? Hadoop users might come from
> more than ONE AD in big org.****
>
> **3. **Is there any technical issue not to support LDAPs like
> OpenLDAP? In my understanding, one possible difficulity might be that it’s
> not easy to extract common****
>
> group lookup mechanism with common filters/configurations both for AD and
> OpenLDAP like, right?****
>
> ** **
>
> I’m wondering if these are just limits for current implementation, and if
> so if we need to improve that. Might the community has already been going
> for that?****
>
> ** **
>
> Thanks****
>
> Kai****
>
> ** **
>
> *From:* Ivan Frain [mailto:ivan.frain@gmail.com]
> *Sent:* Tuesday, October 09, 2012 2:46 PM
> *To:* user@hadoop.apache.org
> *Subject:* Re: Secure hadoop and group permission on HDFS****
>
> ** **
>
> Hi Koert,****
>
> ** **
>
> Another option is to use the LdapGroupsMapping which picks up the group
> membership from a LDAP directory.****
>
> You can find more details on the JIRA issue:
> https://issues.apache.org/jira/browse/HADOOP-8121****
>
> Up to now, it is available for ActiveDirectory and released in
> hadoop-2.0.0-alpha and next releases.****
>
> You can easily apply the patch on a 0.23.1, I already did that and it
> works well.****
>
> ** **
>
> OpenLdap with POSIX groups is not yet supported by this patch, it was
> tailored for ActiveDirectory.****
>
> ** **
>
> ** **
>
> BR,****
>
> Ivan****
>
> ** **
>
> 2012/10/9 Harsh J <ha...@cloudera.com>****
>
> Koert,
>
> If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> class (via hadoop.security.group.mapping), then yes the NameNode's
> view of the local unix groups (and the primary group) of the user is
> the final say on what groups the user belongs to. This can be relied
> on - but note that HDFS uses BSD style semantics when it comes to
> groups and when creating directories/files, the parent directory
> groups are inherited automatically unless altered after creation.****
>
>
> On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com> wrote:
> > With secure hadoop the user name is authenticated by the kerberos server.
> > But what about the groups that the user is a member of? Are these simple
> the
> > groups that the user is a member of on the namenode machine?
> > Is it viable to manage access to files on HDFS using groups on a secure
> > hadoop cluster?
> >
>
>
> ****
>
> --
> Harsh J****
>
>
>
> ****
>
> ** **
>
> --
> Ivan Frain
> 11, route de Grenade****
>
> 31530 Saint-Paul-sur-Save****
>
> mobile: +33 (0)6 52 52 47 07****
>
> ** **
>
> ** **
>
Re: Secure hadoop and group permission on HDFS
Posted by Jonathan Natkins <na...@cloudera.com>.
Hi Kai,
1. To the best of my knowledge, you can only use one group mapping service
at a time. In order to do what you're suggesting, you'd have to write a
customized group mapping service.
2. Currently multiple ADs are not supported, but it's certainly an
improvement that could be made.
3. The LdapGroupsMapping already supports OpenLDAP. It's pretty heavily
configurable for the purpose of supporting multiple types of LDAP
implementations. The defaults just happen to be geared towards Active
Directory.
Thanks,
Natty
On Mon, Oct 15, 2012 at 8:07 PM, Zheng, Kai <ka...@intel.com> wrote:
> Hi Koert & Harsh,****
>
> ** **
>
> Regarding LdapGroupsMapping, I have questions:****
>
> **1. **Is it possible to use ShellBasedUnixGroupsMapping for Hadoop
> service principals/users, and LdapGroupsMapping for end user accounts?****
>
> In our environment, normal end users (along with their groups info) for
> Hadoop cluster are from AD, and for them we should use the ldap mapping;**
> **
>
> but for hdfs/mapred service principals, the default shell based one is
> enough, and we don’t want to create the user/group entries in AD just for
> that.****
>
> **2. **Can we support multiple ADs? Hadoop users might come from
> more than ONE AD in big org.****
>
> **3. **Is there any technical issue not to support LDAPs like
> OpenLDAP? In my understanding, one possible difficulity might be that it’s
> not easy to extract common****
>
> group lookup mechanism with common filters/configurations both for AD and
> OpenLDAP like, right?****
>
> ** **
>
> I’m wondering if these are just limits for current implementation, and if
> so if we need to improve that. Might the community has already been going
> for that?****
>
> ** **
>
> Thanks****
>
> Kai****
>
> ** **
>
> *From:* Ivan Frain [mailto:ivan.frain@gmail.com]
> *Sent:* Tuesday, October 09, 2012 2:46 PM
> *To:* user@hadoop.apache.org
> *Subject:* Re: Secure hadoop and group permission on HDFS****
>
> ** **
>
> Hi Koert,****
>
> ** **
>
> Another option is to use the LdapGroupsMapping which picks up the group
> membership from a LDAP directory.****
>
> You can find more details on the JIRA issue:
> https://issues.apache.org/jira/browse/HADOOP-8121****
>
> Up to now, it is available for ActiveDirectory and released in
> hadoop-2.0.0-alpha and next releases.****
>
> You can easily apply the patch on a 0.23.1, I already did that and it
> works well.****
>
> ** **
>
> OpenLdap with POSIX groups is not yet supported by this patch, it was
> tailored for ActiveDirectory.****
>
> ** **
>
> ** **
>
> BR,****
>
> Ivan****
>
> ** **
>
> 2012/10/9 Harsh J <ha...@cloudera.com>****
>
> Koert,
>
> If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> class (via hadoop.security.group.mapping), then yes the NameNode's
> view of the local unix groups (and the primary group) of the user is
> the final say on what groups the user belongs to. This can be relied
> on - but note that HDFS uses BSD style semantics when it comes to
> groups and when creating directories/files, the parent directory
> groups are inherited automatically unless altered after creation.****
>
>
> On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com> wrote:
> > With secure hadoop the user name is authenticated by the kerberos server.
> > But what about the groups that the user is a member of? Are these simple
> the
> > groups that the user is a member of on the namenode machine?
> > Is it viable to manage access to files on HDFS using groups on a secure
> > hadoop cluster?
> >
>
>
> ****
>
> --
> Harsh J****
>
>
>
> ****
>
> ** **
>
> --
> Ivan Frain
> 11, route de Grenade****
>
> 31530 Saint-Paul-sur-Save****
>
> mobile: +33 (0)6 52 52 47 07****
>
> ** **
>
> ** **
>
Re: Secure hadoop and group permission on HDFS
Posted by Jonathan Natkins <na...@cloudera.com>.
Hi Kai,
1. To the best of my knowledge, you can only use one group mapping service
at a time. In order to do what you're suggesting, you'd have to write a
customized group mapping service.
2. Currently multiple ADs are not supported, but it's certainly an
improvement that could be made.
3. The LdapGroupsMapping already supports OpenLDAP. It's pretty heavily
configurable for the purpose of supporting multiple types of LDAP
implementations. The defaults just happen to be geared towards Active
Directory.
Thanks,
Natty
On Mon, Oct 15, 2012 at 8:07 PM, Zheng, Kai <ka...@intel.com> wrote:
> Hi Koert & Harsh,****
>
> ** **
>
> Regarding LdapGroupsMapping, I have questions:****
>
> **1. **Is it possible to use ShellBasedUnixGroupsMapping for Hadoop
> service principals/users, and LdapGroupsMapping for end user accounts?****
>
> In our environment, normal end users (along with their groups info) for
> Hadoop cluster are from AD, and for them we should use the ldap mapping;**
> **
>
> but for hdfs/mapred service principals, the default shell based one is
> enough, and we don’t want to create the user/group entries in AD just for
> that.****
>
> **2. **Can we support multiple ADs? Hadoop users might come from
> more than ONE AD in big org.****
>
> **3. **Is there any technical issue not to support LDAPs like
> OpenLDAP? In my understanding, one possible difficulity might be that it’s
> not easy to extract common****
>
> group lookup mechanism with common filters/configurations both for AD and
> OpenLDAP like, right?****
>
> ** **
>
> I’m wondering if these are just limits for current implementation, and if
> so if we need to improve that. Might the community has already been going
> for that?****
>
> ** **
>
> Thanks****
>
> Kai****
>
> ** **
>
> *From:* Ivan Frain [mailto:ivan.frain@gmail.com]
> *Sent:* Tuesday, October 09, 2012 2:46 PM
> *To:* user@hadoop.apache.org
> *Subject:* Re: Secure hadoop and group permission on HDFS****
>
> ** **
>
> Hi Koert,****
>
> ** **
>
> Another option is to use the LdapGroupsMapping which picks up the group
> membership from a LDAP directory.****
>
> You can find more details on the JIRA issue:
> https://issues.apache.org/jira/browse/HADOOP-8121****
>
> Up to now, it is available for ActiveDirectory and released in
> hadoop-2.0.0-alpha and next releases.****
>
> You can easily apply the patch on a 0.23.1, I already did that and it
> works well.****
>
> ** **
>
> OpenLdap with POSIX groups is not yet supported by this patch, it was
> tailored for ActiveDirectory.****
>
> ** **
>
> ** **
>
> BR,****
>
> Ivan****
>
> ** **
>
> 2012/10/9 Harsh J <ha...@cloudera.com>****
>
> Koert,
>
> If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> class (via hadoop.security.group.mapping), then yes the NameNode's
> view of the local unix groups (and the primary group) of the user is
> the final say on what groups the user belongs to. This can be relied
> on - but note that HDFS uses BSD style semantics when it comes to
> groups and when creating directories/files, the parent directory
> groups are inherited automatically unless altered after creation.****
>
>
> On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com> wrote:
> > With secure hadoop the user name is authenticated by the kerberos server.
> > But what about the groups that the user is a member of? Are these simple
> the
> > groups that the user is a member of on the namenode machine?
> > Is it viable to manage access to files on HDFS using groups on a secure
> > hadoop cluster?
> >
>
>
> ****
>
> --
> Harsh J****
>
>
>
> ****
>
> ** **
>
> --
> Ivan Frain
> 11, route de Grenade****
>
> 31530 Saint-Paul-sur-Save****
>
> mobile: +33 (0)6 52 52 47 07****
>
> ** **
>
> ** **
>
Re: Secure hadoop and group permission on HDFS
Posted by Jonathan Natkins <na...@cloudera.com>.
Hi Kai,
1. To the best of my knowledge, you can only use one group mapping service
at a time. In order to do what you're suggesting, you'd have to write a
customized group mapping service.
2. Currently multiple ADs are not supported, but it's certainly an
improvement that could be made.
3. The LdapGroupsMapping already supports OpenLDAP. It's pretty heavily
configurable for the purpose of supporting multiple types of LDAP
implementations. The defaults just happen to be geared towards Active
Directory.
Thanks,
Natty
On Mon, Oct 15, 2012 at 8:07 PM, Zheng, Kai <ka...@intel.com> wrote:
> Hi Koert & Harsh,****
>
> ** **
>
> Regarding LdapGroupsMapping, I have questions:****
>
> **1. **Is it possible to use ShellBasedUnixGroupsMapping for Hadoop
> service principals/users, and LdapGroupsMapping for end user accounts?****
>
> In our environment, normal end users (along with their groups info) for
> Hadoop cluster are from AD, and for them we should use the ldap mapping;**
> **
>
> but for hdfs/mapred service principals, the default shell based one is
> enough, and we don’t want to create the user/group entries in AD just for
> that.****
>
> **2. **Can we support multiple ADs? Hadoop users might come from
> more than ONE AD in big org.****
>
> **3. **Is there any technical issue not to support LDAPs like
> OpenLDAP? In my understanding, one possible difficulity might be that it’s
> not easy to extract common****
>
> group lookup mechanism with common filters/configurations both for AD and
> OpenLDAP like, right?****
>
> ** **
>
> I’m wondering if these are just limits for current implementation, and if
> so if we need to improve that. Might the community has already been going
> for that?****
>
> ** **
>
> Thanks****
>
> Kai****
>
> ** **
>
> *From:* Ivan Frain [mailto:ivan.frain@gmail.com]
> *Sent:* Tuesday, October 09, 2012 2:46 PM
> *To:* user@hadoop.apache.org
> *Subject:* Re: Secure hadoop and group permission on HDFS****
>
> ** **
>
> Hi Koert,****
>
> ** **
>
> Another option is to use the LdapGroupsMapping which picks up the group
> membership from a LDAP directory.****
>
> You can find more details on the JIRA issue:
> https://issues.apache.org/jira/browse/HADOOP-8121****
>
> Up to now, it is available for ActiveDirectory and released in
> hadoop-2.0.0-alpha and next releases.****
>
> You can easily apply the patch on a 0.23.1, I already did that and it
> works well.****
>
> ** **
>
> OpenLdap with POSIX groups is not yet supported by this patch, it was
> tailored for ActiveDirectory.****
>
> ** **
>
> ** **
>
> BR,****
>
> Ivan****
>
> ** **
>
> 2012/10/9 Harsh J <ha...@cloudera.com>****
>
> Koert,
>
> If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> class (via hadoop.security.group.mapping), then yes the NameNode's
> view of the local unix groups (and the primary group) of the user is
> the final say on what groups the user belongs to. This can be relied
> on - but note that HDFS uses BSD style semantics when it comes to
> groups and when creating directories/files, the parent directory
> groups are inherited automatically unless altered after creation.****
>
>
> On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com> wrote:
> > With secure hadoop the user name is authenticated by the kerberos server.
> > But what about the groups that the user is a member of? Are these simple
> the
> > groups that the user is a member of on the namenode machine?
> > Is it viable to manage access to files on HDFS using groups on a secure
> > hadoop cluster?
> >
>
>
> ****
>
> --
> Harsh J****
>
>
>
> ****
>
> ** **
>
> --
> Ivan Frain
> 11, route de Grenade****
>
> 31530 Saint-Paul-sur-Save****
>
> mobile: +33 (0)6 52 52 47 07****
>
> ** **
>
> ** **
>
RE: Secure hadoop and group permission on HDFS
Posted by "Zheng, Kai" <ka...@intel.com>.
Hi Koert & Harsh,
Regarding LdapGroupsMapping, I have questions:
1. Is it possible to use ShellBasedUnixGroupsMapping for Hadoop service principals/users, and LdapGroupsMapping for end user accounts?
In our environment, normal end users (along with their groups info) for Hadoop cluster are from AD, and for them we should use the ldap mapping;
but for hdfs/mapred service principals, the default shell based one is enough, and we don't want to create the user/group entries in AD just for that.
2. Can we support multiple ADs? Hadoop users might come from more than ONE AD in big org.
3. Is there any technical issue not to support LDAPs like OpenLDAP? In my understanding, one possible difficulity might be that it's not easy to extract common
group lookup mechanism with common filters/configurations both for AD and OpenLDAP like, right?
I'm wondering if these are just limits for current implementation, and if so if we need to improve that. Might the community has already been going for that?
Thanks
Kai
From: Ivan Frain [mailto:ivan.frain@gmail.com]
Sent: Tuesday, October 09, 2012 2:46 PM
To: user@hadoop.apache.org
Subject: Re: Secure hadoop and group permission on HDFS
Hi Koert,
Another option is to use the LdapGroupsMapping which picks up the group membership from a LDAP directory.
You can find more details on the JIRA issue: https://issues.apache.org/jira/browse/HADOOP-8121
Up to now, it is available for ActiveDirectory and released in hadoop-2.0.0-alpha and next releases.
You can easily apply the patch on a 0.23.1, I already did that and it works well.
OpenLdap with POSIX groups is not yet supported by this patch, it was tailored for ActiveDirectory.
BR,
Ivan
2012/10/9 Harsh J <ha...@cloudera.com>>
Koert,
If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
class (via hadoop.security.group.mapping), then yes the NameNode's
view of the local unix groups (and the primary group) of the user is
the final say on what groups the user belongs to. This can be relied
on - but note that HDFS uses BSD style semantics when it comes to
groups and when creating directories/files, the parent directory
groups are inherited automatically unless altered after creation.
On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com>> wrote:
> With secure hadoop the user name is authenticated by the kerberos server.
> But what about the groups that the user is a member of? Are these simple the
> groups that the user is a member of on the namenode machine?
> Is it viable to manage access to files on HDFS using groups on a secure
> hadoop cluster?
>
--
Harsh J
--
Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07
RE: Secure hadoop and group permission on HDFS
Posted by "Zheng, Kai" <ka...@intel.com>.
Hi Koert & Harsh,
Regarding LdapGroupsMapping, I have questions:
1. Is it possible to use ShellBasedUnixGroupsMapping for Hadoop service principals/users, and LdapGroupsMapping for end user accounts?
In our environment, normal end users (along with their groups info) for Hadoop cluster are from AD, and for them we should use the ldap mapping;
but for hdfs/mapred service principals, the default shell based one is enough, and we don't want to create the user/group entries in AD just for that.
2. Can we support multiple ADs? Hadoop users might come from more than ONE AD in big org.
3. Is there any technical issue not to support LDAPs like OpenLDAP? In my understanding, one possible difficulity might be that it's not easy to extract common
group lookup mechanism with common filters/configurations both for AD and OpenLDAP like, right?
I'm wondering if these are just limits for current implementation, and if so if we need to improve that. Might the community has already been going for that?
Thanks
Kai
From: Ivan Frain [mailto:ivan.frain@gmail.com]
Sent: Tuesday, October 09, 2012 2:46 PM
To: user@hadoop.apache.org
Subject: Re: Secure hadoop and group permission on HDFS
Hi Koert,
Another option is to use the LdapGroupsMapping which picks up the group membership from a LDAP directory.
You can find more details on the JIRA issue: https://issues.apache.org/jira/browse/HADOOP-8121
Up to now, it is available for ActiveDirectory and released in hadoop-2.0.0-alpha and next releases.
You can easily apply the patch on a 0.23.1, I already did that and it works well.
OpenLdap with POSIX groups is not yet supported by this patch, it was tailored for ActiveDirectory.
BR,
Ivan
2012/10/9 Harsh J <ha...@cloudera.com>>
Koert,
If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
class (via hadoop.security.group.mapping), then yes the NameNode's
view of the local unix groups (and the primary group) of the user is
the final say on what groups the user belongs to. This can be relied
on - but note that HDFS uses BSD style semantics when it comes to
groups and when creating directories/files, the parent directory
groups are inherited automatically unless altered after creation.
On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com>> wrote:
> With secure hadoop the user name is authenticated by the kerberos server.
> But what about the groups that the user is a member of? Are these simple the
> groups that the user is a member of on the namenode machine?
> Is it viable to manage access to files on HDFS using groups on a secure
> hadoop cluster?
>
--
Harsh J
--
Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07
RE: Secure hadoop and group permission on HDFS
Posted by "Zheng, Kai" <ka...@intel.com>.
Hi Koert & Harsh,
Regarding LdapGroupsMapping, I have questions:
1. Is it possible to use ShellBasedUnixGroupsMapping for Hadoop service principals/users, and LdapGroupsMapping for end user accounts?
In our environment, normal end users (along with their groups info) for Hadoop cluster are from AD, and for them we should use the ldap mapping;
but for hdfs/mapred service principals, the default shell based one is enough, and we don't want to create the user/group entries in AD just for that.
2. Can we support multiple ADs? Hadoop users might come from more than ONE AD in big org.
3. Is there any technical issue not to support LDAPs like OpenLDAP? In my understanding, one possible difficulity might be that it's not easy to extract common
group lookup mechanism with common filters/configurations both for AD and OpenLDAP like, right?
I'm wondering if these are just limits for current implementation, and if so if we need to improve that. Might the community has already been going for that?
Thanks
Kai
From: Ivan Frain [mailto:ivan.frain@gmail.com]
Sent: Tuesday, October 09, 2012 2:46 PM
To: user@hadoop.apache.org
Subject: Re: Secure hadoop and group permission on HDFS
Hi Koert,
Another option is to use the LdapGroupsMapping which picks up the group membership from a LDAP directory.
You can find more details on the JIRA issue: https://issues.apache.org/jira/browse/HADOOP-8121
Up to now, it is available for ActiveDirectory and released in hadoop-2.0.0-alpha and next releases.
You can easily apply the patch on a 0.23.1, I already did that and it works well.
OpenLdap with POSIX groups is not yet supported by this patch, it was tailored for ActiveDirectory.
BR,
Ivan
2012/10/9 Harsh J <ha...@cloudera.com>>
Koert,
If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
class (via hadoop.security.group.mapping), then yes the NameNode's
view of the local unix groups (and the primary group) of the user is
the final say on what groups the user belongs to. This can be relied
on - but note that HDFS uses BSD style semantics when it comes to
groups and when creating directories/files, the parent directory
groups are inherited automatically unless altered after creation.
On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com>> wrote:
> With secure hadoop the user name is authenticated by the kerberos server.
> But what about the groups that the user is a member of? Are these simple the
> groups that the user is a member of on the namenode machine?
> Is it viable to manage access to files on HDFS using groups on a secure
> hadoop cluster?
>
--
Harsh J
--
Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07
RE: Secure hadoop and group permission on HDFS
Posted by "Zheng, Kai" <ka...@intel.com>.
Hi Koert & Harsh,
Regarding LdapGroupsMapping, I have questions:
1. Is it possible to use ShellBasedUnixGroupsMapping for Hadoop service principals/users, and LdapGroupsMapping for end user accounts?
In our environment, normal end users (along with their groups info) for Hadoop cluster are from AD, and for them we should use the ldap mapping;
but for hdfs/mapred service principals, the default shell based one is enough, and we don't want to create the user/group entries in AD just for that.
2. Can we support multiple ADs? Hadoop users might come from more than ONE AD in big org.
3. Is there any technical issue not to support LDAPs like OpenLDAP? In my understanding, one possible difficulity might be that it's not easy to extract common
group lookup mechanism with common filters/configurations both for AD and OpenLDAP like, right?
I'm wondering if these are just limits for current implementation, and if so if we need to improve that. Might the community has already been going for that?
Thanks
Kai
From: Ivan Frain [mailto:ivan.frain@gmail.com]
Sent: Tuesday, October 09, 2012 2:46 PM
To: user@hadoop.apache.org
Subject: Re: Secure hadoop and group permission on HDFS
Hi Koert,
Another option is to use the LdapGroupsMapping which picks up the group membership from a LDAP directory.
You can find more details on the JIRA issue: https://issues.apache.org/jira/browse/HADOOP-8121
Up to now, it is available for ActiveDirectory and released in hadoop-2.0.0-alpha and next releases.
You can easily apply the patch on a 0.23.1, I already did that and it works well.
OpenLdap with POSIX groups is not yet supported by this patch, it was tailored for ActiveDirectory.
BR,
Ivan
2012/10/9 Harsh J <ha...@cloudera.com>>
Koert,
If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
class (via hadoop.security.group.mapping), then yes the NameNode's
view of the local unix groups (and the primary group) of the user is
the final say on what groups the user belongs to. This can be relied
on - but note that HDFS uses BSD style semantics when it comes to
groups and when creating directories/files, the parent directory
groups are inherited automatically unless altered after creation.
On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com>> wrote:
> With secure hadoop the user name is authenticated by the kerberos server.
> But what about the groups that the user is a member of? Are these simple the
> groups that the user is a member of on the namenode machine?
> Is it viable to manage access to files on HDFS using groups on a secure
> hadoop cluster?
>
--
Harsh J
--
Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07
Re: Secure hadoop and group permission on HDFS
Posted by Ivan Frain <iv...@gmail.com>.
Hi Koert,
Another option is to use the LdapGroupsMapping which picks up the group
membership from a LDAP directory.
You can find more details on the JIRA issue:
https://issues.apache.org/jira/browse/HADOOP-8121
Up to now, it is available for ActiveDirectory and released in
hadoop-2.0.0-alpha and next releases.
You can easily apply the patch on a 0.23.1, I already did that and it works
well.
OpenLdap with POSIX groups is not yet supported by this patch, it was
tailored for ActiveDirectory.
BR,
Ivan
2012/10/9 Harsh J <ha...@cloudera.com>
> Koert,
>
> If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> class (via hadoop.security.group.mapping), then yes the NameNode's
> view of the local unix groups (and the primary group) of the user is
> the final say on what groups the user belongs to. This can be relied
> on - but note that HDFS uses BSD style semantics when it comes to
> groups and when creating directories/files, the parent directory
> groups are inherited automatically unless altered after creation.
>
> On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com> wrote:
> > With secure hadoop the user name is authenticated by the kerberos server.
> > But what about the groups that the user is a member of? Are these simple
> the
> > groups that the user is a member of on the namenode machine?
> > Is it viable to manage access to files on HDFS using groups on a secure
> > hadoop cluster?
> >
>
>
>
> --
> Harsh J
>
--
Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07
Re: Secure hadoop and group permission on HDFS
Posted by Ivan Frain <iv...@gmail.com>.
Hi Koert,
Another option is to use the LdapGroupsMapping which picks up the group
membership from a LDAP directory.
You can find more details on the JIRA issue:
https://issues.apache.org/jira/browse/HADOOP-8121
Up to now, it is available for ActiveDirectory and released in
hadoop-2.0.0-alpha and next releases.
You can easily apply the patch on a 0.23.1, I already did that and it works
well.
OpenLdap with POSIX groups is not yet supported by this patch, it was
tailored for ActiveDirectory.
BR,
Ivan
2012/10/9 Harsh J <ha...@cloudera.com>
> Koert,
>
> If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> class (via hadoop.security.group.mapping), then yes the NameNode's
> view of the local unix groups (and the primary group) of the user is
> the final say on what groups the user belongs to. This can be relied
> on - but note that HDFS uses BSD style semantics when it comes to
> groups and when creating directories/files, the parent directory
> groups are inherited automatically unless altered after creation.
>
> On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com> wrote:
> > With secure hadoop the user name is authenticated by the kerberos server.
> > But what about the groups that the user is a member of? Are these simple
> the
> > groups that the user is a member of on the namenode machine?
> > Is it viable to manage access to files on HDFS using groups on a secure
> > hadoop cluster?
> >
>
>
>
> --
> Harsh J
>
--
Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07
Re: Secure hadoop and group permission on HDFS
Posted by Ivan Frain <iv...@gmail.com>.
Hi Koert,
Another option is to use the LdapGroupsMapping which picks up the group
membership from a LDAP directory.
You can find more details on the JIRA issue:
https://issues.apache.org/jira/browse/HADOOP-8121
Up to now, it is available for ActiveDirectory and released in
hadoop-2.0.0-alpha and next releases.
You can easily apply the patch on a 0.23.1, I already did that and it works
well.
OpenLdap with POSIX groups is not yet supported by this patch, it was
tailored for ActiveDirectory.
BR,
Ivan
2012/10/9 Harsh J <ha...@cloudera.com>
> Koert,
>
> If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> class (via hadoop.security.group.mapping), then yes the NameNode's
> view of the local unix groups (and the primary group) of the user is
> the final say on what groups the user belongs to. This can be relied
> on - but note that HDFS uses BSD style semantics when it comes to
> groups and when creating directories/files, the parent directory
> groups are inherited automatically unless altered after creation.
>
> On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com> wrote:
> > With secure hadoop the user name is authenticated by the kerberos server.
> > But what about the groups that the user is a member of? Are these simple
> the
> > groups that the user is a member of on the namenode machine?
> > Is it viable to manage access to files on HDFS using groups on a secure
> > hadoop cluster?
> >
>
>
>
> --
> Harsh J
>
--
Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07
Re: Secure hadoop and group permission on HDFS
Posted by Ivan Frain <iv...@gmail.com>.
Hi Koert,
Another option is to use the LdapGroupsMapping which picks up the group
membership from a LDAP directory.
You can find more details on the JIRA issue:
https://issues.apache.org/jira/browse/HADOOP-8121
Up to now, it is available for ActiveDirectory and released in
hadoop-2.0.0-alpha and next releases.
You can easily apply the patch on a 0.23.1, I already did that and it works
well.
OpenLdap with POSIX groups is not yet supported by this patch, it was
tailored for ActiveDirectory.
BR,
Ivan
2012/10/9 Harsh J <ha...@cloudera.com>
> Koert,
>
> If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
> class (via hadoop.security.group.mapping), then yes the NameNode's
> view of the local unix groups (and the primary group) of the user is
> the final say on what groups the user belongs to. This can be relied
> on - but note that HDFS uses BSD style semantics when it comes to
> groups and when creating directories/files, the parent directory
> groups are inherited automatically unless altered after creation.
>
> On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com> wrote:
> > With secure hadoop the user name is authenticated by the kerberos server.
> > But what about the groups that the user is a member of? Are these simple
> the
> > groups that the user is a member of on the namenode machine?
> > Is it viable to manage access to files on HDFS using groups on a secure
> > hadoop cluster?
> >
>
>
>
> --
> Harsh J
>
--
Ivan Frain
11, route de Grenade
31530 Saint-Paul-sur-Save
mobile: +33 (0)6 52 52 47 07
Re: Secure hadoop and group permission on HDFS
Posted by Harsh J <ha...@cloudera.com>.
Koert,
If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
class (via hadoop.security.group.mapping), then yes the NameNode's
view of the local unix groups (and the primary group) of the user is
the final say on what groups the user belongs to. This can be relied
on - but note that HDFS uses BSD style semantics when it comes to
groups and when creating directories/files, the parent directory
groups are inherited automatically unless altered after creation.
On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com> wrote:
> With secure hadoop the user name is authenticated by the kerberos server.
> But what about the groups that the user is a member of? Are these simple the
> groups that the user is a member of on the namenode machine?
> Is it viable to manage access to files on HDFS using groups on a secure
> hadoop cluster?
>
--
Harsh J
Re: Secure hadoop and group permission on HDFS
Posted by Harsh J <ha...@cloudera.com>.
Koert,
If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
class (via hadoop.security.group.mapping), then yes the NameNode's
view of the local unix groups (and the primary group) of the user is
the final say on what groups the user belongs to. This can be relied
on - but note that HDFS uses BSD style semantics when it comes to
groups and when creating directories/files, the parent directory
groups are inherited automatically unless altered after creation.
On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com> wrote:
> With secure hadoop the user name is authenticated by the kerberos server.
> But what about the groups that the user is a member of? Are these simple the
> groups that the user is a member of on the namenode machine?
> Is it viable to manage access to files on HDFS using groups on a secure
> hadoop cluster?
>
--
Harsh J
Re: Secure hadoop and group permission on HDFS
Posted by Harsh J <ha...@cloudera.com>.
Koert,
If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
class (via hadoop.security.group.mapping), then yes the NameNode's
view of the local unix groups (and the primary group) of the user is
the final say on what groups the user belongs to. This can be relied
on - but note that HDFS uses BSD style semantics when it comes to
groups and when creating directories/files, the parent directory
groups are inherited automatically unless altered after creation.
On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com> wrote:
> With secure hadoop the user name is authenticated by the kerberos server.
> But what about the groups that the user is a member of? Are these simple the
> groups that the user is a member of on the namenode machine?
> Is it viable to manage access to files on HDFS using groups on a secure
> hadoop cluster?
>
--
Harsh J
Re: Secure hadoop and group permission on HDFS
Posted by Harsh J <ha...@cloudera.com>.
Koert,
If you use the org.apache.hadoop.security.ShellBasedUnixGroupsMapping
class (via hadoop.security.group.mapping), then yes the NameNode's
view of the local unix groups (and the primary group) of the user is
the final say on what groups the user belongs to. This can be relied
on - but note that HDFS uses BSD style semantics when it comes to
groups and when creating directories/files, the parent directory
groups are inherited automatically unless altered after creation.
On Tue, Oct 9, 2012 at 2:30 AM, Koert Kuipers <ko...@tresata.com> wrote:
> With secure hadoop the user name is authenticated by the kerberos server.
> But what about the groups that the user is a member of? Are these simple the
> groups that the user is a member of on the namenode machine?
> Is it viable to manage access to files on HDFS using groups on a secure
> hadoop cluster?
>
--
Harsh J