You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by wa...@apache.org on 2016/03/23 22:57:24 UTC
hadoop git commit: HADOOP-12947. Update documentation Hadoop Groups
Mapping to add static group mapping,
negative cache. Contributed by Wei-Chiu Chuang.
Repository: hadoop
Updated Branches:
refs/heads/trunk 738155063 -> 938222b2e
HADOOP-12947. Update documentation Hadoop Groups Mapping to add static group mapping, negative cache. Contributed by Wei-Chiu Chuang.
Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/938222b2
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/938222b2
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/938222b2
Branch: refs/heads/trunk
Commit: 938222b2e20791d6eb410e36f8fec3e1520e1078
Parents: 7381550
Author: Andrew Wang <wa...@apache.org>
Authored: Wed Mar 23 14:57:14 2016 -0700
Committer: Andrew Wang <wa...@apache.org>
Committed: Wed Mar 23 14:57:14 2016 -0700
----------------------------------------------------------------------
.../src/site/markdown/GroupsMapping.md | 26 +++++++++++---------
1 file changed, 15 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hadoop/blob/938222b2/hadoop-common-project/hadoop-common/src/site/markdown/GroupsMapping.md
----------------------------------------------------------------------
diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/GroupsMapping.md b/hadoop-common-project/hadoop-common/src/site/markdown/GroupsMapping.md
index 5a67bd1..a742029 100644
--- a/hadoop-common-project/hadoop-common/src/site/markdown/GroupsMapping.md
+++ b/hadoop-common-project/hadoop-common/src/site/markdown/GroupsMapping.md
@@ -15,14 +15,7 @@
Hadoop Groups Mapping
===================
-* [Hadoop Groups Mapping](#Hadoop_Groups_Mapping)
- * [Overview](#Overview)
- * [LDAP Groups Mapping](#LDAP_Groups_Mapping)
- * [Active Directory](#Active_Directory)
- * [POSIX Groups](#POSIX_Groups)
- * [SSL](#SSL)
- * [Composite Groups Mapping](#Composite_Groups_Mapping)
- * [Multiple group mapping providers configuration sample](#Multiple_group_mapping_providers_configuration_sample)
+<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Overview
--------
@@ -61,6 +54,17 @@ For HDFS, the mapping of users to groups is performed on the NameNode. Thus, the
Note that HDFS stores the user and group of a file or directory as strings; there is no conversion from user and group identity numbers as is conventional in Unix.
+Static Mapping
+--------
+It is possible to statically map users to groups by defining the mapping in `hadoop.user.group.static.mapping.overrides` in the format `user1=group1,group2;user2=;user3=group2`.
+This property overrides any group mapping service provider. If a user's groups are defined in it, the groups are returned without more lookups; otherwise, the service provider defined in `hadoop.security.group.mapping` is used to look up the groups. By default, `dr.who=;` is defined, so the fake user dr.who will not have any groups.
+
+Caching/Negative caching
+--------
+Since the group mapping resolution relies on external mechanisms, the NameNode performance may be impacted. To reduce the impact due to repeated lookups, Hadoop caches the groups returned by the service provider. The cache invalidate is configurable via `hadoop.security.groups.cache.secs`, and the default is 300 seconds.
+
+To avoid spamming NameNode with unknown users, Hadoop employs negative caching so that if the result of the lookup is empty, return an empty group directly instead of performing more group mapping queries,
+The cache invalidation is configurable via `hadoop.security.groups.negative-cache.secs`. The default is 30 seconds, so if group mapping service providers returns no group for a user, no lookup will be performed for the same user within 30 seconds.
LDAP Groups Mapping
--------
@@ -85,9 +89,9 @@ in order to be considered a member.
The default configuration supports LDAP group name resolution with an Active Directory server.
### POSIX Groups ###
-If the LDAP server supports POSIX group semantics, Hadoop can perform LDAP group resolution queries to the server by setting both
-`hadoop.security.group.mapping.ldap.search.filter.user` to `posixAccount` and
-`hadoop.security.group.mapping.ldap.search.filter.group` to `posixGroup`.
+If the LDAP server supports POSIX group semantics (RFC-2307), Hadoop can perform LDAP group resolution queries to the server by setting both
+`hadoop.security.group.mapping.ldap.search.filter.user` to `(&(objectClass=posixAccount)(uid={0}))` and
+`hadoop.security.group.mapping.ldap.search.filter.group` to `(objectClass=posixGroup)`.
### SSL ###
To secure the connection, the implementation supports LDAP over SSL (LDAPS). SSL is enable by setting `hadoop.security.group.mapping.ldap.ssl` to `true`.