You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Shashi Vishwakarma <sh...@gmail.com> on 2016/02/19 08:29:42 UTC
Get member list of group in HDFS using Java/Script
Hi
I need to find how many users belong to particular group in HDFS using
Java/Script.
Let's say I have a directory /user/data in HDFS and it belongs to a group
'abc' .
I need to find how may other users belongs to 'abc' group.
Can anyone suggest something.
Thanks
Re: Get member list of group in HDFS using Java/Script
Posted by Shashi Vishwakarma <sh...@gmail.com>.
Thanks Chris. Information you provided is very much helpful..
On 20 Feb 2016 12:18 am, "Chris Nauroth" <cn...@hortonworks.com> wrote:
> Hello Shashi,
>
> HDFS itself doesn't have any knowledge of membership of users in groups.
> The user and group field of an HDFS file or directory are just arbitrary
> strings. For consulting a user's group memberships, HDFS runs a group
> mapping provider [1]. This is pluggable, but the common case is either
> querying the underlying OS directly or querying an LDAP server.
>
> For a given username, you can check what HDFS thinks its group memberships
> are by running the "hdfs groups" command [2]. For example:
>
> > hdfs groups chris
> chris : staff everyone localaccounts _appserverusr admin _appserveradm
> _lpadmin _appstore _lpoperator _developer com.apple.access_screensharing
> com.apple.access_ssh
>
> However, there is no way to query a list of all users in a group through
> HDFS. HDFS doesn't have a use case for this kind of query, so it doesn't
> implement one. For that, I'd expect you'd need to query directly whatever
> your group membership infrastructure is (i.e. run the "groups" command at
> the OS layer or query an LDAP server).
>
> I hope this helps.
>
> --Chris Nauroth
>
> [1]
> http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#Group_Mapping
> [2]
> http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#groups
>
>
> From: Shashi Vishwakarma <sh...@gmail.com>
> Date: Thursday, February 18, 2016 at 11:29 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: Get member list of group in HDFS using Java/Script
>
> Hi
>
> I need to find how many users belong to particular group in HDFS using
> Java/Script.
>
> Let's say I have a directory /user/data in HDFS and it belongs to a group
> 'abc' .
>
> I need to find how may other users belongs to 'abc' group.
>
> Can anyone suggest something.
>
> Thanks
>
Re: Get member list of group in HDFS using Java/Script
Posted by Shashi Vishwakarma <sh...@gmail.com>.
Thanks Chris. Information you provided is very much helpful..
On 20 Feb 2016 12:18 am, "Chris Nauroth" <cn...@hortonworks.com> wrote:
> Hello Shashi,
>
> HDFS itself doesn't have any knowledge of membership of users in groups.
> The user and group field of an HDFS file or directory are just arbitrary
> strings. For consulting a user's group memberships, HDFS runs a group
> mapping provider [1]. This is pluggable, but the common case is either
> querying the underlying OS directly or querying an LDAP server.
>
> For a given username, you can check what HDFS thinks its group memberships
> are by running the "hdfs groups" command [2]. For example:
>
> > hdfs groups chris
> chris : staff everyone localaccounts _appserverusr admin _appserveradm
> _lpadmin _appstore _lpoperator _developer com.apple.access_screensharing
> com.apple.access_ssh
>
> However, there is no way to query a list of all users in a group through
> HDFS. HDFS doesn't have a use case for this kind of query, so it doesn't
> implement one. For that, I'd expect you'd need to query directly whatever
> your group membership infrastructure is (i.e. run the "groups" command at
> the OS layer or query an LDAP server).
>
> I hope this helps.
>
> --Chris Nauroth
>
> [1]
> http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#Group_Mapping
> [2]
> http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#groups
>
>
> From: Shashi Vishwakarma <sh...@gmail.com>
> Date: Thursday, February 18, 2016 at 11:29 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: Get member list of group in HDFS using Java/Script
>
> Hi
>
> I need to find how many users belong to particular group in HDFS using
> Java/Script.
>
> Let's say I have a directory /user/data in HDFS and it belongs to a group
> 'abc' .
>
> I need to find how may other users belongs to 'abc' group.
>
> Can anyone suggest something.
>
> Thanks
>
Re: Get member list of group in HDFS using Java/Script
Posted by Shashi Vishwakarma <sh...@gmail.com>.
Thanks Chris. Information you provided is very much helpful..
On 20 Feb 2016 12:18 am, "Chris Nauroth" <cn...@hortonworks.com> wrote:
> Hello Shashi,
>
> HDFS itself doesn't have any knowledge of membership of users in groups.
> The user and group field of an HDFS file or directory are just arbitrary
> strings. For consulting a user's group memberships, HDFS runs a group
> mapping provider [1]. This is pluggable, but the common case is either
> querying the underlying OS directly or querying an LDAP server.
>
> For a given username, you can check what HDFS thinks its group memberships
> are by running the "hdfs groups" command [2]. For example:
>
> > hdfs groups chris
> chris : staff everyone localaccounts _appserverusr admin _appserveradm
> _lpadmin _appstore _lpoperator _developer com.apple.access_screensharing
> com.apple.access_ssh
>
> However, there is no way to query a list of all users in a group through
> HDFS. HDFS doesn't have a use case for this kind of query, so it doesn't
> implement one. For that, I'd expect you'd need to query directly whatever
> your group membership infrastructure is (i.e. run the "groups" command at
> the OS layer or query an LDAP server).
>
> I hope this helps.
>
> --Chris Nauroth
>
> [1]
> http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#Group_Mapping
> [2]
> http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#groups
>
>
> From: Shashi Vishwakarma <sh...@gmail.com>
> Date: Thursday, February 18, 2016 at 11:29 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: Get member list of group in HDFS using Java/Script
>
> Hi
>
> I need to find how many users belong to particular group in HDFS using
> Java/Script.
>
> Let's say I have a directory /user/data in HDFS and it belongs to a group
> 'abc' .
>
> I need to find how may other users belongs to 'abc' group.
>
> Can anyone suggest something.
>
> Thanks
>
Re: Get member list of group in HDFS using Java/Script
Posted by Shashi Vishwakarma <sh...@gmail.com>.
Thanks Chris. Information you provided is very much helpful..
On 20 Feb 2016 12:18 am, "Chris Nauroth" <cn...@hortonworks.com> wrote:
> Hello Shashi,
>
> HDFS itself doesn't have any knowledge of membership of users in groups.
> The user and group field of an HDFS file or directory are just arbitrary
> strings. For consulting a user's group memberships, HDFS runs a group
> mapping provider [1]. This is pluggable, but the common case is either
> querying the underlying OS directly or querying an LDAP server.
>
> For a given username, you can check what HDFS thinks its group memberships
> are by running the "hdfs groups" command [2]. For example:
>
> > hdfs groups chris
> chris : staff everyone localaccounts _appserverusr admin _appserveradm
> _lpadmin _appstore _lpoperator _developer com.apple.access_screensharing
> com.apple.access_ssh
>
> However, there is no way to query a list of all users in a group through
> HDFS. HDFS doesn't have a use case for this kind of query, so it doesn't
> implement one. For that, I'd expect you'd need to query directly whatever
> your group membership infrastructure is (i.e. run the "groups" command at
> the OS layer or query an LDAP server).
>
> I hope this helps.
>
> --Chris Nauroth
>
> [1]
> http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#Group_Mapping
> [2]
> http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#groups
>
>
> From: Shashi Vishwakarma <sh...@gmail.com>
> Date: Thursday, February 18, 2016 at 11:29 PM
> To: "user@hadoop.apache.org" <us...@hadoop.apache.org>
> Subject: Get member list of group in HDFS using Java/Script
>
> Hi
>
> I need to find how many users belong to particular group in HDFS using
> Java/Script.
>
> Let's say I have a directory /user/data in HDFS and it belongs to a group
> 'abc' .
>
> I need to find how may other users belongs to 'abc' group.
>
> Can anyone suggest something.
>
> Thanks
>
Re: Get member list of group in HDFS using Java/Script
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Shashi,
HDFS itself doesn't have any knowledge of membership of users in groups. The user and group field of an HDFS file or directory are just arbitrary strings. For consulting a user's group memberships, HDFS runs a group mapping provider [1]. This is pluggable, but the common case is either querying the underlying OS directly or querying an LDAP server.
For a given username, you can check what HDFS thinks its group memberships are by running the "hdfs groups" command [2]. For example:
> hdfs groups chris
chris : staff everyone localaccounts _appserverusr admin _appserveradm _lpadmin _appstore _lpoperator _developer com.apple.access_screensharing com.apple.access_ssh
However, there is no way to query a list of all users in a group through HDFS. HDFS doesn't have a use case for this kind of query, so it doesn't implement one. For that, I'd expect you'd need to query directly whatever your group membership infrastructure is (i.e. run the "groups" command at the OS layer or query an LDAP server).
I hope this helps.
--Chris Nauroth
[1] http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#Group_Mapping
[2] http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#groups
From: Shashi Vishwakarma <sh...@gmail.com>>
Date: Thursday, February 18, 2016 at 11:29 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Get member list of group in HDFS using Java/Script
Hi
I need to find how many users belong to particular group in HDFS using Java/Script.
Let's say I have a directory /user/data in HDFS and it belongs to a group 'abc' .
I need to find how may other users belongs to 'abc' group.
Can anyone suggest something.
Thanks
Re: Get member list of group in HDFS using Java/Script
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Shashi,
HDFS itself doesn't have any knowledge of membership of users in groups. The user and group field of an HDFS file or directory are just arbitrary strings. For consulting a user's group memberships, HDFS runs a group mapping provider [1]. This is pluggable, but the common case is either querying the underlying OS directly or querying an LDAP server.
For a given username, you can check what HDFS thinks its group memberships are by running the "hdfs groups" command [2]. For example:
> hdfs groups chris
chris : staff everyone localaccounts _appserverusr admin _appserveradm _lpadmin _appstore _lpoperator _developer com.apple.access_screensharing com.apple.access_ssh
However, there is no way to query a list of all users in a group through HDFS. HDFS doesn't have a use case for this kind of query, so it doesn't implement one. For that, I'd expect you'd need to query directly whatever your group membership infrastructure is (i.e. run the "groups" command at the OS layer or query an LDAP server).
I hope this helps.
--Chris Nauroth
[1] http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#Group_Mapping
[2] http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#groups
From: Shashi Vishwakarma <sh...@gmail.com>>
Date: Thursday, February 18, 2016 at 11:29 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Get member list of group in HDFS using Java/Script
Hi
I need to find how many users belong to particular group in HDFS using Java/Script.
Let's say I have a directory /user/data in HDFS and it belongs to a group 'abc' .
I need to find how may other users belongs to 'abc' group.
Can anyone suggest something.
Thanks
Re: Get member list of group in HDFS using Java/Script
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Shashi,
HDFS itself doesn't have any knowledge of membership of users in groups. The user and group field of an HDFS file or directory are just arbitrary strings. For consulting a user's group memberships, HDFS runs a group mapping provider [1]. This is pluggable, but the common case is either querying the underlying OS directly or querying an LDAP server.
For a given username, you can check what HDFS thinks its group memberships are by running the "hdfs groups" command [2]. For example:
> hdfs groups chris
chris : staff everyone localaccounts _appserverusr admin _appserveradm _lpadmin _appstore _lpoperator _developer com.apple.access_screensharing com.apple.access_ssh
However, there is no way to query a list of all users in a group through HDFS. HDFS doesn't have a use case for this kind of query, so it doesn't implement one. For that, I'd expect you'd need to query directly whatever your group membership infrastructure is (i.e. run the "groups" command at the OS layer or query an LDAP server).
I hope this helps.
--Chris Nauroth
[1] http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#Group_Mapping
[2] http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#groups
From: Shashi Vishwakarma <sh...@gmail.com>>
Date: Thursday, February 18, 2016 at 11:29 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Get member list of group in HDFS using Java/Script
Hi
I need to find how many users belong to particular group in HDFS using Java/Script.
Let's say I have a directory /user/data in HDFS and it belongs to a group 'abc' .
I need to find how may other users belongs to 'abc' group.
Can anyone suggest something.
Thanks
Re: Get member list of group in HDFS using Java/Script
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Shashi,
HDFS itself doesn't have any knowledge of membership of users in groups. The user and group field of an HDFS file or directory are just arbitrary strings. For consulting a user's group memberships, HDFS runs a group mapping provider [1]. This is pluggable, but the common case is either querying the underlying OS directly or querying an LDAP server.
For a given username, you can check what HDFS thinks its group memberships are by running the "hdfs groups" command [2]. For example:
> hdfs groups chris
chris : staff everyone localaccounts _appserverusr admin _appserveradm _lpadmin _appstore _lpoperator _developer com.apple.access_screensharing com.apple.access_ssh
However, there is no way to query a list of all users in a group through HDFS. HDFS doesn't have a use case for this kind of query, so it doesn't implement one. For that, I'd expect you'd need to query directly whatever your group membership infrastructure is (i.e. run the "groups" command at the OS layer or query an LDAP server).
I hope this helps.
--Chris Nauroth
[1] http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#Group_Mapping
[2] http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html#groups
From: Shashi Vishwakarma <sh...@gmail.com>>
Date: Thursday, February 18, 2016 at 11:29 PM
To: "user@hadoop.apache.org<ma...@hadoop.apache.org>" <us...@hadoop.apache.org>>
Subject: Get member list of group in HDFS using Java/Script
Hi
I need to find how many users belong to particular group in HDFS using Java/Script.
Let's say I have a directory /user/data in HDFS and it belongs to a group 'abc' .
I need to find how may other users belongs to 'abc' group.
Can anyone suggest something.
Thanks