You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@knox.apache.org by su...@apache.org on 2015/04/08 00:51:18 UTC
svn commit: r1671969 - in /knox/trunk/books/0.6.0: book_gateway-details.md
config_ldap_authc_cache.md
Author: sumit
Date: Tue Apr 7 22:51:17 2015
New Revision: 1671969
URL: http://svn.apache.org/r1671969
Log:
KNOX-524 docs for ldap caching
Added:
knox/trunk/books/0.6.0/config_ldap_authc_cache.md
- copied, changed from r1671932, knox/trunk/books/0.6.0/config_ldap_group_lookup.md
Modified:
knox/trunk/books/0.6.0/book_gateway-details.md
Modified: knox/trunk/books/0.6.0/book_gateway-details.md
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/book_gateway-details.md?rev=1671969&r1=1671968&r2=1671969&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/book_gateway-details.md (original)
+++ knox/trunk/books/0.6.0/book_gateway-details.md Tue Apr 7 22:51:17 2015
@@ -81,6 +81,7 @@ Their values can also be provided via th
<<config_authn.md>>
<<config_advanced_ldap.md>>
<<config_ldap_group_lookup.md>>
+<<config_ldap_authc_cache.md>>
<<config_id_assertion.md>>
<<config_authz.md>>
<<config_kerberos.md>>
Copied: knox/trunk/books/0.6.0/config_ldap_authc_cache.md (from r1671932, knox/trunk/books/0.6.0/config_ldap_group_lookup.md)
URL: http://svn.apache.org/viewvc/knox/trunk/books/0.6.0/config_ldap_authc_cache.md?p2=knox/trunk/books/0.6.0/config_ldap_authc_cache.md&p1=knox/trunk/books/0.6.0/config_ldap_group_lookup.md&r1=1671932&r2=1671969&rev=1671969&view=diff
==============================================================================
--- knox/trunk/books/0.6.0/config_ldap_group_lookup.md (original)
+++ knox/trunk/books/0.6.0/config_ldap_authc_cache.md Tue Apr 7 22:51:17 2015
@@ -15,214 +15,198 @@
limitations under the License.
--->
-### LDAPGroupLookup ###
+### LDAP Authentication Caching ###
-Knox can be configured to look up LDAP groups that the authenticated user belong to.
-Knox can look up both Static LDAP Groups and Dynamic LDAP Groups.
-The looked up groups are populated as Principal(s) in the Java Subject of authenticated user.
-Therefore service authorization rules can be defined in terms of LDAPGroups looked up from LDAP directory.
-
-To look up LDAPGroups of autheticated user from LDAP, you have to use org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm in Shiro configuration.
-
-Please see below a sample Shiro configuration snippet from a topology file that was tested looking LDAPGroups.
-
- <provider>
- <role>authentication</role>
- <name>ShiroProvider</name>
- <enabled>true</enabled>
- <!--
- session timeout in minutes, this is really idle timeout,
- defaults to 30mins, if the property value is not defined,,
- current client authentication would expire if client idles contiuosly for more than this value
- -->
- <!-- defaults to: 30 minutes
- <param>
- <name>sessionTimeout</name>
- <value>30</value>
- </param>
- -->
-
- <!--
- Use single KnoxLdapRealm to do authentication and ldap group look up
- -->
- <param>
- <name>main.ldapRealm</name>
- <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm</value>
- </param>
- <param>
- <name>main.ldapGroupContextFactory</name>
- <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory</value>
- </param>
- <param>
- <name>main.ldapRealm.contextFactory</name>
- <value>$ldapGroupContextFactory</value>
- </param>
- <!-- defaults to: simple
- <param>
- <name>main.ldapRealm.contextFactory.authenticationMechanism</name>
- <value>simple</value>
- </param>
- -->
- <param>
- <name>main.ldapRealm.contextFactory.url</name>
- <value>ldap://localhost:33389</value>
- </param>
- <param>
- <name>main.ldapRealm.userDnTemplate</name>
- <value>uid={0},ou=people,dc=hadoop,dc=apache,dc=org</value>
- </param>
-
- <param>
- <name>main.ldapRealm.authorizationEnabled</name>
- <!-- defaults to: false -->
- <value>true</value>
- </param>
- <!-- defaults to: simple
- <param>
- <name>main.ldapRealm.contextFactory.systemAuthenticationMechanism</name>
- <value>simple</value>
- </param>
- -->
- <param>
- <name>main.ldapRealm.searchBase</name>
- <value>ou=groups,dc=hadoop,dc=apache,dc=org</value>
- </param>
- <!-- defaults to: groupOfNames
- <param>
- <name>main.ldapRealm.groupObjectClass</name>
- <value>groupOfNames</value>
- </param>
- -->
- <!-- defaults to: member
- <param>
- <name>main.ldapRealm.memberAttribute</name>
- <value>member</value>
- </param>
- -->
- <param>
- <name>main.cacheManager</name>
- <value>org.apache.shiro.cache.MemoryConstrainedCacheManager</value>
- </param>
- <param>
- <name>main.securityManager.cacheManager</name>
- <value>$cacheManager</value>
- </param>
- <param>
- <name>main.ldapRealm.memberAttributeValueTemplate</name>
- <value>uid={0},ou=people,dc=hadoop,dc=apache,dc=org</value>
- </param>
- <!-- the above element is the template for most ldap servers
- for active directory use the following instead and
- remove the above configuration.
- <param>
- <name>main.ldapRealm.memberAttributeValueTemplate</name>
- <value>cn={0},ou=people,dc=hadoop,dc=apache,dc=org</value>
- </param>
- -->
- <param>
- <name>main.ldapRealm.contextFactory.systemUsername</name>
- <value>uid=guest,ou=people,dc=hadoop,dc=apache,dc=org</value>
- </param>
- <param>
- <name>main.ldapRealm.contextFactory.systemPassword</name>
- <value>${ALIAS=ldcSystemPassword}</value>
- </param>
-
- <param>
- <name>urls./**</name>
- <value>authcBasic</value>
- </param>
-
- </provider>
-
-The configuration shown above would look up Static LDAP groups of authenticated user and populate the group principals in the Java Subject corresponding to authenticated user.
-
-If you want to look up Dynamic LDAP Groups instead of Static LDAP Groups, you would have to speciify groupObjectClass and memberAttribute params as shown below:
-
- <param>
- <name>main.ldapRealm.groupObjectClass</name>
- <value>groupOfUrls</value>
- </param>
- <param>
- <name>main.ldapRealm.memberAttribute</name>
- <value>memberUrl</value>
- </param>
-
-### Template topology files and LDIF files to try out LDAP Group Look up ###
+Knox can be configured to cache LDAP authentication information. Knox leverages Shiro's built in
+caching mechanisms and has been tested with Shiro's EhCache cache manager implementation.
-Knox bundles some template topology files and ldif files that you can use to try and test LDAP Group Lookup and associated authorization acls.
-All these template files are located under {GATEWAY_HOME}/templates.
+The following provider snippet demonstrates how to configure turning on the cache using the ShiroProvider. In addition to
+using org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm in the Shiro configuration, and setting up the cache you *must* set
+the flag for enabling caching authentication to true. Please see the property, main.ldapRealm.authenticationCachingEnabled
+below.
+
+
+ <provider>
+ <role>authentication</role>
+ <name>ShiroProvider</name>
+ <enabled>true</enabled>
+ <param>
+ <name>main.ldapRealm</name>
+ <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm</value>
+ </param>
+ <param>
+ <name>main.ldapGroupContextFactory</name>
+ <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory</value>
+ </param>
+ <param>
+ <name>main.ldapRealm.contextFactory</name>
+ <value>$ldapGroupContextFactory</value>
+ </param>
+ <param>
+ <name>main.ldapRealm.contextFactory.url</name>
+ <value>ldap://localhost:33389</value>
+ </param>
+ <param>
+ <name>main.ldapRealm.userDnTemplate</name>
+ <value>uid={0},ou=people,dc=hadoop,dc=apache,dc=org</value>
+ </param>
+ <param>
+ <name>main.ldapRealm.authorizationEnabled</name>
+ <!-- defaults to: false -->
+ <value>true</value>
+ </param>
+ <param>
+ <name>main.ldapRealm.searchBase</name>
+ <value>ou=groups,dc=hadoop,dc=apache,dc=org</value>
+ </param>
+ <param>
+ <name>main.cacheManager</name>
+ <value>org.apache.shiro.cache.ehcache.EhCacheManager</value>
+ </param>
+ <param>
+ <name>main.securityManager.cacheManager</name>
+ <value>$cacheManager</value>
+ </param>
+ <param>
+ <name>main.ldapRealm.authenticationCachingEnabled</name>
+ <value>true</value>
+ </param>
+ <param>
+ <name>main.ldapRealm.memberAttributeValueTemplate</name>
+ <value>uid={0},ou=people,dc=hadoop,dc=apache,dc=org</value>
+ </param>
+ <param>
+ <name>main.ldapRealm.contextFactory.systemUsername</name>
+ <value>uid=guest,ou=people,dc=hadoop,dc=apache,dc=org</value>
+ </param>
+ <param>
+ <name>main.ldapRealm.contextFactory.systemPassword</name>
+ <value>guest-password</value>
+ </param>
+ <param>
+ <name>urls./**</name>
+ <value>authcBasic</value>
+ </param>
+ </provider>
-#### LDAP Static Group Lookup Templates, authentication and group lookup from the same directory ####
+### Trying out caching ###
-topology file: sandbox.knoxrealm1.xml
-ldif file : users.ldapgroups.ldif
+Knox bundles a template topology files that can be used to try out the caching functionality.
+The template file located under {GATEWAY_HOME}/templates is sandbox.knoxrealm.ehcache.xml.
To try this out
cd {GATEWAY_HOME}
-cp templates/sandbox.knoxrealm1.xml deployments/sandbox.xml
-cp templates/users.ldapgroups.ldif conf/users.ldif
-java -jar bin/ldap.jar conf
-java -Dsandbox.ldcSystemPassword=guest-password -jar bin/gateway.jar -persist-master
+cp templates/sandbox.knoxrealm.ehcache.xml deployments/sandbox.xml
+bin/ldap.sh start
+bin/gateway.sh start
-Following call to WebHDFS should report HTTP/1.1 401 Unauthorized
-As guest is not a member of group "analyst", authorization prvoider states user should be member of group "analyst"
+The following call to WebHDFS should report: {"Path":"/user/sam"}
curl -i -v -k -u guest:guest-password -X GET https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY
-Following call to WebHDFS should report: {"Path":"/user/sam"}
-As sam is a member of group "analyst", authorization prvoider states user should be member of group "analyst"
+In order to see the cache working, LDAP can now be shutdown and the user will still authenticate successfully.
-curl -i -v -k -u sam:sam-password -X GET https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY
+bin/ldap.sh stop
-
-#### LDAP Static Group Lookup Templates, authentication and group lookup from different directories ####
-
-topology file: sandbox.knoxrealm2.xml
-ldif file : users.ldapgroups.ldif
-
-To try this out
-
-cd {GATEWAY_HOME}
-cp templates/sandbox.knoxrealm2.xml deployments/sandbox.xml
-cp templates/users.ldapgroups.ldif conf/users.ldif
-java -jar bin/ldap.jar conf
-java -Dsandbox.ldcSystemPassword=guest-password -jar bin/gateway.jar -persist-master
-
-Following call to WebHDFS should report HTTP/1.1 401 Unauthorized
-As guest is not a member of group "analyst", authorization prvoider states user should be member of group "analyst"
+and then the following should still return successfully like it did earlier.
curl -i -v -k -u guest:guest-password -X GET https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY
-Following call to WebHDFS should report: {"Path":"/user/sam"}
-As sam is a member of group "analyst", authorization prvoider states user should be member of group "analyst"
-
-curl -i -v -k -u sam:sam-password -X GET https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY
-#### LDAP Dynamic Group Lookup Templates, authentication and dynamic group lookup from same directory ####
+#### Advanced Caching Config ####
-topology file: sandbox.knoxrealmdg.xml
-ldif file : users.ldapdynamicgroups.ldif
+By default the ehcache support in shiro contains a ehcache.xml in its classpath which is the following
-To try this out
-
-cd {GATEWAY_HOME}
-cp templates/sandbox.knoxrealmdg.xml deployments/sandbox.xml
-cp templates/users.ldapdynamicgroups.ldif conf/users.ldif
-java -jar bin/ldap.jar conf
-java -Dsandbox.ldcSystemPassword=guest-password -jar bin/gateway.jar -persist-master
-
-Please note that user.ldapdynamicgroups.ldif also loads ncessary schema to create dynamic groups in Apache DS.
-
-Following call to WebHDFS should report HTTP/1.1 401 Unauthorized
-As guest is not a member of dynamic group "directors", authorization prvoider states user should be member of group "directors"
-
-curl -i -v -k -u guest:guest-password -X GET https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY
+ <ehcache>
-Following call to WebHDFS should report: {"Path":"/user/bob"}
-As bob is a member of dynamic group "directors", authorization prvoider states user should be member of group "directors"
+ <!-- Sets the path to the directory where cache .data files are created.
-curl -i -v -k -u sam:sam-password -X GET https://localhost:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY
+ If the path is a Java System Property it is replaced by
+ its value in the running VM. The following properties are translated:
+
+ user.home - User's home directory
+ user.dir - User's current working directory
+ java.io.tmpdir - Default temp file path
+ -->
+ <diskStore path="java.io.tmpdir/shiro-ehcache"/>
+
+
+ <!--Default Cache configuration. These will applied to caches programmatically created through
+ the CacheManager.
+
+ The following attributes are required:
+
+ maxElementsInMemory - Sets the maximum number of objects that will be created in memory
+ eternal - Sets whether elements are eternal. If eternal, timeouts are ignored and the
+ element is never expired.
+ overflowToDisk - Sets whether elements can overflow to disk when the in-memory cache
+ has reached the maxInMemory limit.
+
+ The following attributes are optional:
+ timeToIdleSeconds - Sets the time to idle for an element before it expires.
+ i.e. The maximum amount of time between accesses before an element expires
+ Is only used if the element is not eternal.
+ Optional attribute. A value of 0 means that an Element can idle for infinity.
+ The default value is 0.
+ timeToLiveSeconds - Sets the time to live for an element before it expires.
+ i.e. The maximum time between creation time and when an element expires.
+ Is only used if the element is not eternal.
+ Optional attribute. A value of 0 means that and Element can live for infinity.
+ The default value is 0.
+ diskPersistent - Whether the disk store persists between restarts of the Virtual Machine.
+ The default value is false.
+ diskExpiryThreadIntervalSeconds- The number of seconds between runs of the disk expiry thread. The default value
+ is 120 seconds.
+ memoryStoreEvictionPolicy - Policy would be enforced upon reaching the maxElementsInMemory limit. Default
+ policy is Least Recently Used (specified as LRU). Other policies available -
+ First In First Out (specified as FIFO) and Less Frequently Used
+ (specified as LFU)
+ -->
+
+ <defaultCache
+ maxElementsInMemory="10000"
+ eternal="false"
+ timeToIdleSeconds="120"
+ timeToLiveSeconds="120"
+ overflowToDisk="false"
+ diskPersistent="false"
+ diskExpiryThreadIntervalSeconds="120"
+ />
+
+ <!-- We want eternal="true" and no timeToIdle or timeToLive settings because Shiro manages session
+ expirations explicitly. If we set it to false and then set corresponding timeToIdle and timeToLive properties,
+ ehcache would evict sessions without Shiro's knowledge, which would cause many problems
+ (e.g. "My Shiro session timeout is 30 minutes - why isn't a session available after 2 minutes?"
+ Answer - ehcache expired it due to the timeToIdle property set to 120 seconds.)
+
+ diskPersistent=true since we want an enterprise session management feature - ability to use sessions after
+ even after a JVM restart. -->
+ <cache name="shiro-activeSessionCache"
+ maxElementsInMemory="10000"
+ overflowToDisk="true"
+ eternal="true"
+ timeToLiveSeconds="0"
+ timeToIdleSeconds="0"
+ diskPersistent="true"
+ diskExpiryThreadIntervalSeconds="600"/>
+
+ <cache name="org.apache.shiro.realm.text.PropertiesRealm-0-accounts"
+ maxElementsInMemory="1000"
+ eternal="true"
+ overflowToDisk="true"/>
+
+ </ehcache>
+
+A custom configuration file (ehcache.xml) can be used in place of this in order to set specific caching configuration.
+
+In order to set the ehcache.xml file to use for a particular topology, set the following parameter in the configuration
+for the ShiroProvider:
+
+ <param>
+ <name>main.cacheManager.cacheManagerConfigFile</name>
+ <value>classpath:ehcache.xml</value>
+ </param>
+In the above example, place the ehcache.xml file under {GATEWAY_HOME}/conf and restart the gateway server.