You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@knox.apache.org by Darpan Patel <da...@gmail.com> on 2015/12/07 18:59:31 UTC
Need help setting up Knox for A/D integrated Kerberized Cluster
Hi All,
I am stuck on an issue from last two days. I would be really grateful if
someone can help on this.
We have HDP 2.3 implemented over 8 node cluster and the same cluster has
been Kerberized and later on we have integrated it with Active Directory
(Which runs in the same VPN). We also verified that Windows 2012 A/D
integration with Ranger works fine for defining policies and audit log. But
I am stuck at Knox bit. I am trying to replicate the same configuration
properties which I have set for Ranger LDAP-AD Integration.
I am taking reference of the Hortonworks documentation and also Apache Knox
documentation.
The A/D domain name is TEST.COM and all the users are under Users
[image: Inline images 1]
Under the Users we have few users one of the them is knox, darpan,
test,etc.
When we issue following command on the node on which Knox Server is running
(topology name is default)
*curl -iv -k -u knox@test.com:#123Password -X GET
"https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS
<https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS>" OR*
*curl -iv -k -u knox:#123Password -X GET
"https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS
<https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS>"*
Every time I see < HTTP/1.1 401 Unauthorized HTTP/1.1 401 Unauthorized on
the console.
Entries in the *gateway-audit.log *are like this :
gateway-audit.log
==================
15/12/07 17:11:08
||38606993-17e2-4c3e-ad4b-e3faea293aae|audit|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|unavailable|
15/12/07 17:11:09
||38606993-17e2-4c3e-ad4b-e3faea293aae|audit|WEBHDFS||||authentication|
*principal*|*knox@TEST.COM <kn...@TEST.COM>*|failure|*LDAP authentication
failed.*
15/12/07 17:11:09
||38606993-17e2-4c3e-ad4b-e3faea293aae|audit|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|success|Response
status: 401
15/12/07 17:05:28
||5b436e43-b874-40f7-b111-7b262fe5125d|audit|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|unavailable|
15/12/07 17:05:29
||5b436e43-b874-40f7-b111-7b262fe5125d|audit|WEBHDFS||||authentication|
*principal*|knox|failure|*LDAP authentication failed.*
15/12/07 17:05:29
||5b436e43-b874-40f7-b111-7b262fe5125d|audit|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1/?op=LISTSTATUS|success|Response
status: 401
*Gateway.log*
*===========*
2015-12-07 17:05:28,620 INFO hadoop.gateway
(KnoxLdapRealm.java:getUserDn(550)) - Computed userDn:
cn=knox,CN=users,DC=test,DC=com using dnTemplate for principal: knox
Following is the part of our *default.xml *topology:
<gateway>
<provider>
<role>authentication</role>
<name>ShiroProvider</name>
<enabled>true</enabled>
<param>
<name>sessionTimeout</name>
<value>30</value>
</param>
<param>
<name>*main.ldapRealm*</name>
<value>org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm</value>
</param>
<param>
<name>*main.ldapContextFactory*</name>
<value>org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory</value>
</param>
<param>
<name>*main.ldapRealm.userDnTemplate*</name>
<value>cn={0},CN=users,DC=test,DC=com</value>
<!-- also tried following values -->
<value>uid={0},CN=users,DC=test,DC=com</value>
<value>cn={0},DC=test,DC=com</value>
</param>
<param>
<name>*main.ldapRealm.contextFactory.url*</name>
<!-- IP Address of the WINDOSWS 2012 Acive
Directory Server which works for Ranger -->
<value>*ldap://IP_OF_WINDOWS_AD:389*</value>
</param>
<param>
<name>*main.ldapRealm.authorizationEnabled*</name>
<value>true</value>
</param>
<param>
<name>*main.ldapRealm.searchBase*</name>
<value>cn=users,dc=test,dc=com</value>
</param>
<param>
<param>
<name>*main.ldapRealm.memberAttributeValueTemplate*
</name>
<value>cn={0},cn=users,dc=test,dc=com</value>
<!-- also tried uid={0} -->
</param>
<param>
<name>
*main.ldapRealm.contextFactory.authenticationMechanism<*/name>
<value>simple</value>
</param>
<param>
<name>urls./**</name>
<value>authcBasic</value>
</param>
</provider>
<provider>
<role>*identity-assertion*</role>
<name>Default</name>
<enabled>true</enabled>
<param>
<name>*group.principal.mapping*</name>
<value>*=users;hdfs=admin</value>
</param>
</provider>
<provider>
<role>*authorization*</role>
<name>AclsAuthz</name>
<enabled>true</enabled>
</provider>
</gateway>
And following is the console output while trying to access webhdfs using
curl
curl -iv -k -u knox:#123Password -X GET "
https://localhost:8443/gateway/default/webhdfs/v1/?op=LISTSTATUS"
*Console Output:*
----------------
* About to connect() to localhost port 8443 (#0)
* Trying ::1...
* Connected to localhost (::1) port 8443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_DHE_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
* subject:
CN=FQDN_OF_My_gateway_HOST,OU=Test,O=Hadoop,L=Test,ST=Test,C=US
* start date: Nov 27 20:36:22 2015 GMT
* expire date: Nov 26 20:36:22 2016 GMT
* common name: FQDN_OF_My_gateway_HOST
* issuer:
CN=FQDN_OF_My_gateway_HOST,OU=Test,O=Hadoop,L=Test,ST=Test,C=US
* Server auth using Basic with user 'knox'
> GET /gateway/default/webhdfs/v1/?op=LISTSTATUS HTTP/1.1
> Authorization: Basic a25veDojMTIzUGFzc3dvcmQ=
> User-Agent: curl/7.29.0
> Host: localhost:8443
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
HTTP/1.1 401 Unauthorized
* Authentication problem. Ignoring this.
< WWW-Authenticate: BASIC realm="application"
WWW-Authenticate: BASIC realm="application"
< Content-Length: 0
Content-Length: 0
< Server: Jetty(8.1.14.v20131031)
Server: Jetty(8.1.14.v20131031)
Please let me know if any additional information is required.
Thanks,
DP