You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Dhirendra Khanka (JIRA)" <ji...@apache.org> on 2018/02/13 12:04:00 UTC
[jira] [Comment Edited] (HADOOP-15213)
JniBasedUnixGroupsNetgroupMapping.java and
ShellBasedUnixGroupsNetgroupMapping.java use netgroup.substring(1)
[ https://issues.apache.org/jira/browse/HADOOP-15213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16362125#comment-16362125 ]
Dhirendra Khanka edited comment on HADOOP-15213 at 2/13/18 12:03 PM:
---------------------------------------------------------------------
{quote}added a comment - 5 days ago
In {{JniBasedUnixGroupsNetgroupMapping}}, netgroup is only used for service ACL. I.e. the groups specified in hadoop-policy.xml or queue config will be preloaded by netgroup membership lookups. For normal user-group mapping, unix group is used. If the box is configured to use netgroup for normal unix group lookups as well, everything will be netgroup-based.
{quote}
Do you mean that by enabling the Service ACL ( hadoop.security.authorization = true ) and setting security.client.protocol.acl = * ( all users/groups)
should trigger the netgroup membership lookup?
My guess is i need to specifically enter a netgroup with @ notation.
was (Author: dhirensk@gmail.com):
{quote}added a comment - 5 days ago
In {{JniBasedUnixGroupsNetgroupMapping}}, netgroup is only used for service ACL. I.e. the groups specified in hadoop-policy.xml or queue config will be preloaded by netgroup membership lookups. For normal user-group mapping, unix group is used. If the box is configured to use netgroup for normal unix group lookups as well, everything will be netgroup-based.
{quote}
Do you mean that by enabling the Service ACL ( hadoop.security.authorization = true ) and setting security.client.protocol.acl = * ( all users/groups)
should trigger the netgroup membership lookup?
> JniBasedUnixGroupsNetgroupMapping.java and ShellBasedUnixGroupsNetgroupMapping.java use netgroup.substring(1)
> --------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-15213
> URL: https://issues.apache.org/jira/browse/HADOOP-15213
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
> Environment: SUSE Linux Enterprise Server 11 (x86_64)
> VERSION = 11
> PATCHLEVEL = 3
> Reporter: Dhirendra Khanka
> Priority: Minor
>
>
> Part of the code below shown from below 2 classes
> org.apache.hadoop.security.JniBasedUnixGroupsNetgroupMapping.java
> {code:java}
> protected synchronized List<String> getUsersForNetgroup(String netgroup) {
> String[] users = null;
> try {
> // JNI code does not expect '@' at the begining of the group name
> users = getUsersForNetgroupJNI(netgroup.substring(1));
> } catch (Exception e) {
> if (LOG.isDebugEnabled()) {
> LOG.debug("Error getting users for netgroup " + netgroup, e);
> } else {
> LOG.info("Error getting users for netgroup " + netgroup +
> ": " + e.getMessage());
> }
> }
> if (users != null && users.length != 0) {
> return Arrays.asList(users);
> }
> return new LinkedList<String>();
> }{code}
> org.apache.hadoop.security.ShellBasedUnixGroupsNetgroupMapping.java
>
> {code:java}
> protected String execShellGetUserForNetgroup(final String netgroup)
> throws IOException {
> String result = "";
> try
> { // shell command does not expect '@' at the begining of the group name
> result = Shell.execCommand( Shell.getUsersForNetgroupCommand(netgroup.substring(1))); }
> catch (ExitCodeException e)
> { // if we didn't get the group - just return empty list; LOG.warn("error getting users for netgroup " + netgroup, e); }
> return result;
> }
> {code}
> The comments from the code above expect the input to contain '@' , however when executing the shell directly the output has the below form which does not contain any ampersand symbol.
> {code:java}
> :~> getent netgroup mynetgroup1
> mynetgroup1 ( , a3xsds, ) ( , beekvkl, ) ( , redcuan, ) ( , uedfmst, ){code}
>
> I have created a test code and removed the substring function and then ran it on the cluster using hadoop jar. The code returned netgroups correctly after the modification. I have limited knowledge on netgroup. The issue was discovered when
> hadoop.security.group.mapping = *org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback* was added to core-site.xml and it failed to apply netgroup access.
>
> Also find below debug comment to see negroup api calls in action
> tdms@casatdhdp01master01:~> hdfs dfs -ls /user/tdms
> 18/02/09 09:47:30 DEBUG util.Shell: setsid exited with exit code 0
> 18/02/09 09:47:30 DEBUG conf.Configuration: parsing URL jar:file:/usr/hdp/2.5.3.0-37/hadoop/hadoop-common-2.7.3.2.5.3.0-37.jar!/core-default.xml
> 18/02/09 09:47:30 DEBUG conf.Configuration: parsing input stream sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@78186a70
> 18/02/09 09:47:30 DEBUG conf.Configuration: parsing URL file:/etc/hadoop/2.5.3.0-37/0/core-site.xml
> 18/02/09 09:47:30 DEBUG conf.Configuration: parsing input stream java.io.BufferedInputStream@15d9bc04
> 18/02/09 09:47:30 DEBUG security.SecurityUtil: Setting hadoop.security.token.service.use_ip to true
> 18/02/09 09:47:30 DEBUG util.KerberosName: Kerberos krb5 configuration not found, setting default realm to empty
> 18/02/09 09:47:30 DEBUG security.Groups: Creating new Groups object
> 18/02/09 09:47:30 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
> 18/02/09 09:47:30 DEBUG util.NativeCodeLoader: Loaded the native-hadoop library
> 18/02/09 09:47:30 DEBUG security.JniBasedUnixGroupsMapping: Using JniBasedUnixGroupsMapping for Group resolution
> 18/02/09 09:47:30 DEBUG security.JniBasedUnixGroupsNetgroupMapping: Using JniBasedUnixGroupsNetgroupMapping for Netgroup resolution
> 18/02/09 09:47:30 DEBUG security.JniBasedUnixGroupsNetgroupMappingWithFallback: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsNetgroupMapping
> 18/02/09 09:47:30 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsNetgroupMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: hadoop login
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: hadoop login commit
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: tdms
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: Using user: "UnixPrincipal: tdms" with name tdms
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: User entry: "tdms"
> 18/02/09 09:47:30 DEBUG security.UserGroupInformation: UGI loginUser:tdms (auth:SIMPLE)
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org