[jira] [Commented] (YARN-4353) Provide short circuit user group mapping for NM/AM

["Wilfred Spiegelenburg (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16475269#comment-16475269 ] 

Wilfred Spiegelenburg commented on YARN-4353:
---------------------------------------------

[~templedf] it has been really quiet on this Jira for a long time. I have recently started to run into a similar issue as described here.
Ii dug a bit deeper and found that the group lookup is used especially in the ACL checks. This is taken from a NM log:
{code}
2018-03-09 19:14:50,881 DEBUG org.apache.hadoop.yarn.webapp.View: Rendering class org.apache.hadoop.yarn.server.nodemanager.webapp.ContainerLogsPage$ContainersLogsBlock @5
2018-03-09 19:14:50,882 DEBUG org.apache.hadoop.yarn.server.security.ApplicationACLsManager: Verifying access-type VIEW_APP for wilfred (auth:SIMPLE) on application application_1520622831944_0001 owned by systest
2018-03-09 19:14:50,888 DEBUG org.mortbay.log: loaded class com.sun.jndi.ldap.LdapCtxFactory from null
...
2018-03-09 19:14:51,163 WARN org.apache.hadoop.security.LdapGroupsMapping: Exception trying to get groups for user wilfred: [LDAP: error code 1 - 000004DC: LdapErr: DSID-0C09075A, comment: In order to perform this operation a successful bind must be completed on the connection., data 0, v1db1^@]
2018-03-09 19:14:51,164 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user wilfred
{code}

The group resolution is triggered when you set an ACL which has groups listed as allowed. The lookup will be triggered if the user that is requesting access is not the application owner, an admin or allowed access as the user.

Using the {{NullGroupMapping}} would break ACLs.

The other proposed solution to pass in the resolved groups to the AM is also not scalable. In the case that there are thousands of users in the LDAP server and hundreds of groups you would add a large overhead to the NM and then to the AM. You would also get into trouble with long running applications. The group data would become stale and thus cause a security issue.
The AM also uses it for the RPC protocol ACLs if you have that configured so again a {{NullGroupMapping}} would break ACLs there too.

I propose to close this as a won't fix. If you want to use the {{LdapGroupsMapping}} you need to set the configuration up in the correct way to use it.
 

> Provide short circuit user group mapping for NM/AM
> --------------------------------------------------
>
>                 Key: YARN-4353
>                 URL: https://issues.apache.org/jira/browse/YARN-4353
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager
>    Affects Versions: 2.7.1
>            Reporter: Daniel Templeton
>            Assignee: Daniel Templeton
>            Priority: Major
>         Attachments: YARN-4353.prelim.patch
>
>
> When the NM launches an AM, the {{ContainerLocalizer}} gets the current user from {{UserGroupInformation}}, which triggers user group mapping, even though the user groups are never accessed.  If secure LDAP is configured for group mapping, then there are some additional complications created by the unnecessary group resolution.  Additionally, it adds unnecessary latency to the container launch time.
> To address the issue, before getting the current user, the {{ContainerLocalizer}} should configure {{UserGroupInformation}} with a null group mapping service that quickly and quietly returns an empty group list for all users.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org