You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@knox.apache.org by Lian Jiang <ji...@gmail.com> on 2018/06/30 06:22:25 UTC
Knox OS authentication fail due to "password check failed for user"
I am using OS auth for knox and have verified the username and password
work:
sudo pamtester -v knox guest authenticate
pamtester: invoking pam_start(knox, guest, ...)
pamtester: performing operation - authenticate
Password:
pamtester: successfully authenticated
However, my curl command failed:
curl -ik -u guest:"{PASSWORD}"
http://test-namenode.subnet1.hadoop.oraclevcn.com:8443/gateway/ui/webhdfs/v1/user/?op=LISTSTATUS
The error is:
Jun 30 06:16:03 test-namenode unix_chkpwd[37385]: check pass; user unknown
Jun 30 06:16:03 test-namenode unix_chkpwd[37385]: password check failed for
user (guest)
Jun 30 06:16:03 test-namenode java: pam_unix(knox:auth): authentication
failure; logname= uid=2018 euid=2018 tty= ruser= rhost= user=guest
Any idea how I can debug? Appreciate any help.
Re: Knox OS authentication fail due to "password check failed for user"
Posted by Lian Jiang <ji...@gmail.com>.
pamtester works:
[lianjia@prod1-namenode ~]$ sudo pamtester -v login guest authenticate
pamtester: invoking pam_start(login, guest, ...)
pamtester: performing operation - authenticate
Password:
pamtester: successfully authenticated
knoxcli failed:
[lianjia@prod1-namenode knox-server]$ sudo bin/knoxcli.sh user-auth-test
--cluster ui --u guest --p "{PASSWORD}" --d
org.apache.shiro.authc.AuthenticationException: Authentication failed for
token submission [org.apache.shiro.authc.UsernamePasswordToken - guest,
rememberMe=false]. Possible unexpected error? (Typical or expected login
exceptions should extend from AuthenticationException).
/tmp/jna-3506402/jna4211705767471308463.tmp:
/tmp/jna-3506402/jna4211705767471308463.tmp: failed to map segment from
shared object: Operation not permitted
org.apache.shiro.authc.AuthenticationException: Authentication failed for
token submission [org.apache.shiro.authc.UsernamePasswordToken - guest,
rememberMe=false]. Possible unexpected error? (Typical or expected login
exceptions should extend from AuthenticationException).
at
org.apache.shiro.authc.AbstractAuthenticator.authenticate(AbstractAuthenticator.java:214)
at
org.apache.shiro.mgt.AuthenticatingSecurityManager.authenticate(AuthenticatingSecurityManager.java:106)
at
org.apache.shiro.mgt.DefaultSecurityManager.login(DefaultSecurityManager.java:270)
at
org.apache.shiro.subject.support.DelegatingSubject.login(DelegatingSubject.java:256)
at
org.apache.hadoop.gateway.util.KnoxCLI$LDAPCommand.authenticateUser(KnoxCLI.java:1171)
at
org.apache.hadoop.gateway.util.KnoxCLI$LDAPCommand.authenticateUser(KnoxCLI.java:1206)
at
org.apache.hadoop.gateway.util.KnoxCLI$LDAPAuthCommand.execute(KnoxCLI.java:1502)
at org.apache.hadoop.gateway.util.KnoxCLI.run(KnoxCLI.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.gateway.util.KnoxCLI.main(KnoxCLI.java:1777)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.gateway.launcher.Invoker.invokeMainMethod(Invoker.java:70)
at org.apache.hadoop.gateway.launcher.Invoker.invoke(Invoker.java:39)
at org.apache.hadoop.gateway.launcher.Command.run(Command.java:99)
at org.apache.hadoop.gateway.launcher.Launcher.run(Launcher.java:69)
at org.apache.hadoop.gateway.launcher.Launcher.main(Launcher.java:46)
Caused by: java.lang.UnsatisfiedLinkError:
/tmp/jna-3506402/jna4211705767471308463.tmp:
/tmp/jna-3506402/jna4211705767471308463.tmp: failed to map segment from
shared object: Operation not permitted
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
at java.lang.Runtime.load0(Runtime.java:809)
at java.lang.System.load(System.java:1086)
at
com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:761)
at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:736)
at com.sun.jna.Native.<clinit>(Native.java:131)
at com.sun.jna.Pointer.<clinit>(Pointer.java:41)
at com.sun.jna.Structure.<clinit>(Structure.java:1949)
at org.jvnet.libpam.PAM.<init>(PAM.java:73)
at
org.apache.hadoop.gateway.shirorealm.KnoxPamRealm.doGetAuthenticationInfo(KnoxPamRealm.java:135)
at
org.apache.shiro.realm.AuthenticatingRealm.getAuthenticationInfo(AuthenticatingRealm.java:568)
at
org.apache.shiro.authc.pam.ModularRealmAuthenticator.doSingleRealmAuthentication(ModularRealmAuthenticator.java:180)
at
org.apache.shiro.authc.pam.ModularRealmAuthenticator.doAuthenticate(ModularRealmAuthenticator.java:267)
at
org.apache.shiro.authc.AbstractAuthenticator.authenticate(AbstractAuthenticator.java:198)
... 18 more
ERR: Unable to authenticate user: guest
According to https://github.com/traccar/traccar/issues/2709, it is because
/tmp disabled exec permission. The "bad" host is in our production and
/tmp is disabled for exec. Any workaround? Thanks.
On Sun, Jul 1, 2018 at 4:28 PM, Lian Jiang <ji...@gmail.com> wrote:
> Interestingly, on a different host, knox failed to authenticate a user due
> to:
>
> On the "bad" host:
> 2018-07-01 22:59:27,466 DEBUG authc.BasicHttpAuthenticationFilter (
> BasicHttpAuthenticationFilter.java:createToken(308)) - Attempting to
> execute login with headers [Basic Z3Vlc3Q6emhIQSVBQzIzKSg=]
> 2018-07-01 22:51:22,811 WARN authc.AbstractAuthenticator
> (AbstractAuthenticator.java:authenticate(216)) - Authentication failed
> for token submission [org.apache.shiro.authc.UsernamePasswordToken -
> admin, rememberMe=false (10.0.21.117)]. Possible unexpected error?
> (Typical or expected login exceptions should extend from
> AuthenticationException).
> java.lang.NoClassDefFoundError: Could not initialize class
> org.jvnet.libpam.impl.PAMLibrary$pam_conv
> at org.jvnet.libpam.PAM.<init>(PAM.java:73)
> at org.apache.hadoop.gateway.shirorealm.KnoxPamRealm.
> doGetAuthenticationInfo(KnoxPamRealm.java:135)
> at org.apache.shiro.realm.AuthenticatingRealm.getAuthenticationInfo(
> AuthenticatingRealm.java:568)
> at org.apache.shiro.authc.pam.ModularRealmAuthenticator.
> doSingleRealmAuthentication(ModularRealmAuthenticator.java:180)
> at org.apache.shiro.authc.pam.ModularRealmAuthenticator.
> doAuthenticate(ModularRealmAuthenticator.java:267)
> at org.apache.shiro.authc.AbstractAuthenticator.authenticate(
> AbstractAuthenticator.java:198)
> at org.apache.shiro.mgt.AuthenticatingSecurityManager.authenticate(
> AuthenticatingSecurityManager.java:106)
> at org.apache.shiro.mgt.DefaultSecurityManager.login(
> DefaultSecurityManager.java:270)
> at org.apache.shiro.subject.support.DelegatingSubject.
> login(DelegatingSubject.java:256)
> at org.apache.shiro.web.filter.authc.AuthenticatingFilter.
> executeLogin(AuthenticatingFilter.java:53)
> at org.apache.shiro.web.filter.authc.BasicHttpAuthenticationFilter.
> onAccessDenied(BasicHttpAuthenticationFilter.java:190)
> at org.apache.shiro.web.filter.AccessControlFilter.onAccessDenied(
> AccessControlFilter.java:133)
> at org.apache.shiro.web.filter.AccessControlFilter.onPreHandle(
> AccessControlFilter.java:162)
> at org.apache.shiro.web.filter.PathMatchingFilter.
> isFilterChainContinued(PathMatchingFilter.java:203)
> at org.apache.shiro.web.filter.PathMatchingFilter.preHandle(
> PathMatchingFilter.java:178)
> at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(
> AdviceFilter.java:131)
> at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(
> OncePerRequestFilter.java:125)
> at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(
> ProxiedFilterChain.java:66)
> at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(
> AbstractShiroFilter.java:449)
> at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(
> AbstractShiroFilter.java:365)
> at org.apache.shiro.subject.support.SubjectCallable.
> doCall(SubjectCallable.java:90)
> at org.apache.shiro.subject.support.SubjectCallable.call(
> SubjectCallable.java:83)
> at org.apache.shiro.subject.support.DelegatingSubject.
> execute(DelegatingSubject.java:383)
> at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(
> AbstractShiroFilter.java:362)
> at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(
> OncePerRequestFilter.java:125)
> at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(
> GatewayFilter.java:332)
> at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(
> GatewayFilter.java:232)
> at org.apache.hadoop.gateway.filter.ResponseCookieFilter.
> doFilter(ResponseCookieFilter.java:50)
> at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(
> AbstractGatewayFilter.java:61)
> at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(
> GatewayFilter.java:332)
> at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(
> GatewayFilter.java:232)
> at org.apache.hadoop.gateway.filter.XForwardedHeaderFilter.doFilter(
> XForwardedHeaderFilter.java:30)
> at org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(
> AbstractGatewayFilter.java:61)
> at org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(
> GatewayFilter.java:332)
> at org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(
> GatewayFilter.java:232)
> at org.apache.hadoop.gateway.GatewayFilter.doFilter(
> GatewayFilter.java:139)
> at org.apache.hadoop.gateway.GatewayFilter.doFilter(
> GatewayFilter.java:91)
> at org.apache.hadoop.gateway.GatewayServlet.service(
> GatewayServlet.java:141)
> at org.eclipse.jetty.servlet.ServletHolder.handle(
> ServletHolder.java:812)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(
> ServletHandler.java:587)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(
> SecurityHandler.java:577)
> at org.eclipse.jetty.server.session.SessionHandler.
> doHandle(SessionHandler.java:223)
> at org.eclipse.jetty.server.handler.ContextHandler.
> doHandle(ContextHandler.java:1127)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(
> ServletHandler.java:515)
> at org.eclipse.jetty.server.session.SessionHandler.
> doScope(SessionHandler.java:185)
> at org.eclipse.jetty.server.handler.ContextHandler.
> doScope(ContextHandler.java:1061)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:141)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(
> ContextHandlerCollection.java:215)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:97)
> at org.apache.hadoop.gateway.trace.TraceHandler.handle(
> TraceHandler.java:51)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:97)
> at org.apache.hadoop.gateway.filter.CorrelationHandler.
> handle(CorrelationHandler.java:39)
> at org.eclipse.jetty.servlets.gzip.GzipHandler.handle(
> GzipHandler.java:529)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:97)
> at org.apache.hadoop.gateway.filter.PortMappingHelperHandler.handle(
> PortMappingHelperHandler.java:92)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:97)
> at org.eclipse.jetty.websocket.server.WebSocketHandler.
> handle(WebSocketHandler.java:112)
> at org.eclipse.jetty.server.handler.HandlerCollection.
> handle(HandlerCollection.java:110)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
> at org.eclipse.jetty.server.HttpConnection.onFillable(
> HttpConnection.java:257)
> at org.eclipse.jetty.io.AbstractConnection$2.run(
> AbstractConnection.java:544)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
> QueuedThreadPool.java:635)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(
> QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:745)
>
> On the "good" host:
> 2018-07-01 23:11:53,042 DEBUG authc.BasicHttpAuthenticationFilter (
> BasicHttpAuthenticationFilter.java:createToken(308)) - Attempting to
> execute login with headers [Basic Z3Vlc3Q6eUhDYmEpezxKPF05Ozw4TFpqOkU=]
> 2018-07-01 23:11:53,122 DEBUG realm.AuthenticatingRealm
> (AuthenticatingRealm.java:getAuthenticationInfo(569)) - Looked up
> AuthenticationInfo [guest] from doGetAuthenticationInfo
>
>
> I cannot see the difference of the two hosts. Both has the same os (linux
> 7.4), java version and pam lib:
>
> [opc@test-namenode ~]$ ls -l /usr/lib64/libpam.so.0
> lrwxrwxrwx. 1 root root 16 Jun 29 21:32 /usr/lib64/libpam.so.0 ->
> libpam.so.0.83.1
>
> [opc@test-namenode ~]$ java -version
> java version "1.8.0_112"
> Java(TM) SE Runtime Environment (build 1.8.0_112-b15)
> Java HotSpot(TM) 64-Bit Server VM (build 25.112-b15, mixed mode)
>
>
> Any idea about this exception? Thanks.
>
>
> On Sat, Jun 30, 2018 at 2:24 PM, larry mccay <lm...@apache.org> wrote:
>
>> Hmmm....
>>
>> You don't need to restart for topology changes. Glad It us working for
>> you now though!
>>
>>
>> On Sat, Jun 30, 2018, 4:05 PM Lian Jiang <ji...@gmail.com> wrote:
>>
>>> It worked now. I guess I missed knox restarting somewhere.
>>>
>>> On Sat, Jun 30, 2018 at 10:19 AM, Lian Jiang <ji...@gmail.com>
>>> wrote:
>>>
>>>> Furthermore, knoxcli.sh shows guest authentication is ok:
>>>>
>>>> sudo bin/knoxcli.sh user-auth-test --cluster ui --u guest --p
>>>> "{PASSWORD}"
>>>> LDAP authentication successful!
>>>>
>>>> The output shows LDAP but OS auth is used:
>>>>
>>>> <provider>
>>>> <role>authentication</role>
>>>> <name>ShiroProvider</name>
>>>> <enabled>true</enabled>
>>>> <param>
>>>> <name>sessionTimeout</name>
>>>> <value>30</value>
>>>> </param>
>>>> <param>
>>>> <name>main.pamRealm</name>
>>>> <value>org.apache.hadoop.gatew
>>>> ay.shirorealm.KnoxPamRealm</value>
>>>> </param>
>>>> <param>
>>>> <name>main.pamRealm.service</name>
>>>> <value>knox</value>
>>>> </param>
>>>> <param>
>>>> <name>urls./**</name>
>>>> <value>authcBasic</value>
>>>> </param>
>>>> </provider>
>>>> <provider>
>>>> <role>identity-assertion</role>
>>>> <name>Default</name>
>>>> <enabled>true</enabled>
>>>> </provider>
>>>> <provider>
>>>> <role>authorization</role>
>>>> <name>XASecurePDPKnox</name>
>>>> <enabled>true</enabled>
>>>> </provider>
>>>>
>>>> The knox pam service is:
>>>>
>>>> auth required pam_env.so
>>>> auth sufficient pam_unix.so nullok try_first_pass
>>>> auth requisite pam_succeed_if.so uid >= 1000 quiet_success
>>>> auth required pam_deny.so
>>>>
>>>> On Sat, Jun 30, 2018 at 9:21 AM, Lian Jiang <ji...@gmail.com>
>>>> wrote:
>>>>
>>>>> yes. I do both pamtester and curl on the knox host.
>>>>>
>>>>> On Sat, Jun 30, 2018 at 6:36 AM, larry mccay <lm...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Are you on the Knox host when testing with Pam tester? The accounts
>>>>>> will need to be on the Knox host.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sat, Jun 30, 2018, 2:22 AM Lian Jiang <ji...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I am using OS auth for knox and have verified the username and
>>>>>>> password work:
>>>>>>>
>>>>>>> sudo pamtester -v knox guest authenticate
>>>>>>> pamtester: invoking pam_start(knox, guest, ...)
>>>>>>> pamtester: performing operation - authenticate
>>>>>>> Password:
>>>>>>> pamtester: successfully authenticated
>>>>>>>
>>>>>>> However, my curl command failed:
>>>>>>>
>>>>>>> curl -ik -u guest:"{PASSWORD}" http://test-namenode.subnet1.h
>>>>>>> adoop.oraclevcn.com:8443/gateway/ui/webhdfs/v1/user/?op=LISTSTATUS
>>>>>>>
>>>>>>> The error is:
>>>>>>> Jun 30 06:16:03 test-namenode unix_chkpwd[37385]: check pass; user
>>>>>>> unknown
>>>>>>> Jun 30 06:16:03 test-namenode unix_chkpwd[37385]: password check
>>>>>>> failed for user (guest)
>>>>>>> Jun 30 06:16:03 test-namenode java: pam_unix(knox:auth):
>>>>>>> authentication failure; logname= uid=2018 euid=2018 tty= ruser= rhost=
>>>>>>> user=guest
>>>>>>>
>>>>>>>
>>>>>>> Any idea how I can debug? Appreciate any help.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>
Re: Knox OS authentication fail due to "password check failed for user"
Posted by Lian Jiang <ji...@gmail.com>.
Interestingly, on a different host, knox failed to authenticate a user due
to:
On the "bad" host:
2018-07-01 22:59:27,466 DEBUG authc.BasicHttpAuthenticationFilter
(BasicHttpAuthenticationFilter.java:createToken(308)) - Attempting to
execute login with headers [Basic Z3Vlc3Q6emhIQSVBQzIzKSg=]
2018-07-01 22:51:22,811 WARN authc.AbstractAuthenticator
(AbstractAuthenticator.java:authenticate(216)) - Authentication failed for
token submission [org.apache.shiro.authc.UsernamePasswordToken - admin,
rememberMe=false (10.0.21.117)]. Possible unexpected error? (Typical or
expected login exceptions should extend from AuthenticationException).
java.lang.NoClassDefFoundError: Could not initialize class
org.jvnet.libpam.impl.PAMLibrary$pam_conv
at org.jvnet.libpam.PAM.<init>(PAM.java:73)
at
org.apache.hadoop.gateway.shirorealm.KnoxPamRealm.doGetAuthenticationInfo(KnoxPamRealm.java:135)
at
org.apache.shiro.realm.AuthenticatingRealm.getAuthenticationInfo(AuthenticatingRealm.java:568)
at
org.apache.shiro.authc.pam.ModularRealmAuthenticator.doSingleRealmAuthentication(ModularRealmAuthenticator.java:180)
at
org.apache.shiro.authc.pam.ModularRealmAuthenticator.doAuthenticate(ModularRealmAuthenticator.java:267)
at
org.apache.shiro.authc.AbstractAuthenticator.authenticate(AbstractAuthenticator.java:198)
at
org.apache.shiro.mgt.AuthenticatingSecurityManager.authenticate(AuthenticatingSecurityManager.java:106)
at
org.apache.shiro.mgt.DefaultSecurityManager.login(DefaultSecurityManager.java:270)
at
org.apache.shiro.subject.support.DelegatingSubject.login(DelegatingSubject.java:256)
at
org.apache.shiro.web.filter.authc.AuthenticatingFilter.executeLogin(AuthenticatingFilter.java:53)
at
org.apache.shiro.web.filter.authc.BasicHttpAuthenticationFilter.onAccessDenied(BasicHttpAuthenticationFilter.java:190)
at
org.apache.shiro.web.filter.AccessControlFilter.onAccessDenied(AccessControlFilter.java:133)
at
org.apache.shiro.web.filter.AccessControlFilter.onPreHandle(AccessControlFilter.java:162)
at
org.apache.shiro.web.filter.PathMatchingFilter.isFilterChainContinued(PathMatchingFilter.java:203)
at
org.apache.shiro.web.filter.PathMatchingFilter.preHandle(PathMatchingFilter.java:178)
at
org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:131)
at
org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at
org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at
org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
at
org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
at
org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
at
org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
at
org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
at
org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
at
org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at
org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332)
at
org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232)
at
org.apache.hadoop.gateway.filter.ResponseCookieFilter.doFilter(ResponseCookieFilter.java:50)
at
org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61)
at
org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332)
at
org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232)
at
org.apache.hadoop.gateway.filter.XForwardedHeaderFilter.doFilter(XForwardedHeaderFilter.java:30)
at
org.apache.hadoop.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:61)
at
org.apache.hadoop.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:332)
at
org.apache.hadoop.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:232)
at
org.apache.hadoop.gateway.GatewayFilter.doFilter(GatewayFilter.java:139)
at
org.apache.hadoop.gateway.GatewayFilter.doFilter(GatewayFilter.java:91)
at
org.apache.hadoop.gateway.GatewayServlet.service(GatewayServlet.java:141)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at
org.apache.hadoop.gateway.trace.TraceHandler.handle(TraceHandler.java:51)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at
org.apache.hadoop.gateway.filter.CorrelationHandler.handle(CorrelationHandler.java:39)
at
org.eclipse.jetty.servlets.gzip.GzipHandler.handle(GzipHandler.java:529)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at
org.apache.hadoop.gateway.filter.PortMappingHelperHandler.handle(PortMappingHelperHandler.java:92)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at
org.eclipse.jetty.websocket.server.WebSocketHandler.handle(WebSocketHandler.java:112)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
On the "good" host:
2018-07-01 23:11:53,042 DEBUG authc.BasicHttpAuthenticationFilter
(BasicHttpAuthenticationFilter.java:createToken(308)) - Attempting to
execute login with headers [Basic Z3Vlc3Q6eUhDYmEpezxKPF05Ozw4TFpqOkU=]
2018-07-01 23:11:53,122 DEBUG realm.AuthenticatingRealm
(AuthenticatingRealm.java:getAuthenticationInfo(569)) - Looked up
AuthenticationInfo [guest] from doGetAuthenticationInfo
I cannot see the difference of the two hosts. Both has the same os (linux
7.4), java version and pam lib:
[opc@test-namenode ~]$ ls -l /usr/lib64/libpam.so.0
lrwxrwxrwx. 1 root root 16 Jun 29 21:32 /usr/lib64/libpam.so.0 ->
libpam.so.0.83.1
[opc@test-namenode ~]$ java -version
java version "1.8.0_112"
Java(TM) SE Runtime Environment (build 1.8.0_112-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.112-b15, mixed mode)
Any idea about this exception? Thanks.
On Sat, Jun 30, 2018 at 2:24 PM, larry mccay <lm...@apache.org> wrote:
> Hmmm....
>
> You don't need to restart for topology changes. Glad It us working for you
> now though!
>
>
> On Sat, Jun 30, 2018, 4:05 PM Lian Jiang <ji...@gmail.com> wrote:
>
>> It worked now. I guess I missed knox restarting somewhere.
>>
>> On Sat, Jun 30, 2018 at 10:19 AM, Lian Jiang <ji...@gmail.com>
>> wrote:
>>
>>> Furthermore, knoxcli.sh shows guest authentication is ok:
>>>
>>> sudo bin/knoxcli.sh user-auth-test --cluster ui --u guest --p
>>> "{PASSWORD}"
>>> LDAP authentication successful!
>>>
>>> The output shows LDAP but OS auth is used:
>>>
>>> <provider>
>>> <role>authentication</role>
>>> <name>ShiroProvider</name>
>>> <enabled>true</enabled>
>>> <param>
>>> <name>sessionTimeout</name>
>>> <value>30</value>
>>> </param>
>>> <param>
>>> <name>main.pamRealm</name>
>>> <value>org.apache.hadoop.gateway.shirorealm.
>>> KnoxPamRealm</value>
>>> </param>
>>> <param>
>>> <name>main.pamRealm.service</name>
>>> <value>knox</value>
>>> </param>
>>> <param>
>>> <name>urls./**</name>
>>> <value>authcBasic</value>
>>> </param>
>>> </provider>
>>> <provider>
>>> <role>identity-assertion</role>
>>> <name>Default</name>
>>> <enabled>true</enabled>
>>> </provider>
>>> <provider>
>>> <role>authorization</role>
>>> <name>XASecurePDPKnox</name>
>>> <enabled>true</enabled>
>>> </provider>
>>>
>>> The knox pam service is:
>>>
>>> auth required pam_env.so
>>> auth sufficient pam_unix.so nullok try_first_pass
>>> auth requisite pam_succeed_if.so uid >= 1000 quiet_success
>>> auth required pam_deny.so
>>>
>>> On Sat, Jun 30, 2018 at 9:21 AM, Lian Jiang <ji...@gmail.com>
>>> wrote:
>>>
>>>> yes. I do both pamtester and curl on the knox host.
>>>>
>>>> On Sat, Jun 30, 2018 at 6:36 AM, larry mccay <lm...@apache.org> wrote:
>>>>
>>>>> Are you on the Knox host when testing with Pam tester? The accounts
>>>>> will need to be on the Knox host.
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Jun 30, 2018, 2:22 AM Lian Jiang <ji...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I am using OS auth for knox and have verified the username and
>>>>>> password work:
>>>>>>
>>>>>> sudo pamtester -v knox guest authenticate
>>>>>> pamtester: invoking pam_start(knox, guest, ...)
>>>>>> pamtester: performing operation - authenticate
>>>>>> Password:
>>>>>> pamtester: successfully authenticated
>>>>>>
>>>>>> However, my curl command failed:
>>>>>>
>>>>>> curl -ik -u guest:"{PASSWORD}" http://test-namenode.subnet1.
>>>>>> hadoop.oraclevcn.com:8443/gateway/ui/webhdfs/v1/user/?op=LISTSTATUS
>>>>>>
>>>>>> The error is:
>>>>>> Jun 30 06:16:03 test-namenode unix_chkpwd[37385]: check pass; user
>>>>>> unknown
>>>>>> Jun 30 06:16:03 test-namenode unix_chkpwd[37385]: password check
>>>>>> failed for user (guest)
>>>>>> Jun 30 06:16:03 test-namenode java: pam_unix(knox:auth):
>>>>>> authentication failure; logname= uid=2018 euid=2018 tty= ruser= rhost=
>>>>>> user=guest
>>>>>>
>>>>>>
>>>>>> Any idea how I can debug? Appreciate any help.
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>
>>
Re: Knox OS authentication fail due to "password check failed for user"
Posted by larry mccay <lm...@apache.org>.
Hmmm....
You don't need to restart for topology changes. Glad It us working for you
now though!
On Sat, Jun 30, 2018, 4:05 PM Lian Jiang <ji...@gmail.com> wrote:
> It worked now. I guess I missed knox restarting somewhere.
>
> On Sat, Jun 30, 2018 at 10:19 AM, Lian Jiang <ji...@gmail.com>
> wrote:
>
>> Furthermore, knoxcli.sh shows guest authentication is ok:
>>
>> sudo bin/knoxcli.sh user-auth-test --cluster ui --u guest --p "{PASSWORD}"
>> LDAP authentication successful!
>>
>> The output shows LDAP but OS auth is used:
>>
>> <provider>
>> <role>authentication</role>
>> <name>ShiroProvider</name>
>> <enabled>true</enabled>
>> <param>
>> <name>sessionTimeout</name>
>> <value>30</value>
>> </param>
>> <param>
>> <name>main.pamRealm</name>
>>
>> <value>org.apache.hadoop.gateway.shirorealm.KnoxPamRealm</value>
>> </param>
>> <param>
>> <name>main.pamRealm.service</name>
>> <value>knox</value>
>> </param>
>> <param>
>> <name>urls./**</name>
>> <value>authcBasic</value>
>> </param>
>> </provider>
>> <provider>
>> <role>identity-assertion</role>
>> <name>Default</name>
>> <enabled>true</enabled>
>> </provider>
>> <provider>
>> <role>authorization</role>
>> <name>XASecurePDPKnox</name>
>> <enabled>true</enabled>
>> </provider>
>>
>> The knox pam service is:
>>
>> auth required pam_env.so
>> auth sufficient pam_unix.so nullok try_first_pass
>> auth requisite pam_succeed_if.so uid >= 1000 quiet_success
>> auth required pam_deny.so
>>
>> On Sat, Jun 30, 2018 at 9:21 AM, Lian Jiang <ji...@gmail.com>
>> wrote:
>>
>>> yes. I do both pamtester and curl on the knox host.
>>>
>>> On Sat, Jun 30, 2018 at 6:36 AM, larry mccay <lm...@apache.org> wrote:
>>>
>>>> Are you on the Knox host when testing with Pam tester? The accounts
>>>> will need to be on the Knox host.
>>>>
>>>>
>>>>
>>>> On Sat, Jun 30, 2018, 2:22 AM Lian Jiang <ji...@gmail.com> wrote:
>>>>
>>>>> I am using OS auth for knox and have verified the username and
>>>>> password work:
>>>>>
>>>>> sudo pamtester -v knox guest authenticate
>>>>> pamtester: invoking pam_start(knox, guest, ...)
>>>>> pamtester: performing operation - authenticate
>>>>> Password:
>>>>> pamtester: successfully authenticated
>>>>>
>>>>> However, my curl command failed:
>>>>>
>>>>> curl -ik -u guest:"{PASSWORD}"
>>>>> http://test-namenode.subnet1.hadoop.oraclevcn.com:8443/gateway/ui/webhdfs/v1/user/?op=LISTSTATUS
>>>>>
>>>>> The error is:
>>>>> Jun 30 06:16:03 test-namenode unix_chkpwd[37385]: check pass; user
>>>>> unknown
>>>>> Jun 30 06:16:03 test-namenode unix_chkpwd[37385]: password check
>>>>> failed for user (guest)
>>>>> Jun 30 06:16:03 test-namenode java: pam_unix(knox:auth):
>>>>> authentication failure; logname= uid=2018 euid=2018 tty= ruser= rhost=
>>>>> user=guest
>>>>>
>>>>>
>>>>> Any idea how I can debug? Appreciate any help.
>>>>>
>>>>>
>>>>>
>>>
>>
>
Re: Knox OS authentication fail due to "password check failed for user"
Posted by Lian Jiang <ji...@gmail.com>.
It worked now. I guess I missed knox restarting somewhere.
On Sat, Jun 30, 2018 at 10:19 AM, Lian Jiang <ji...@gmail.com> wrote:
> Furthermore, knoxcli.sh shows guest authentication is ok:
>
> sudo bin/knoxcli.sh user-auth-test --cluster ui --u guest --p "{PASSWORD}"
> LDAP authentication successful!
>
> The output shows LDAP but OS auth is used:
>
> <provider>
> <role>authentication</role>
> <name>ShiroProvider</name>
> <enabled>true</enabled>
> <param>
> <name>sessionTimeout</name>
> <value>30</value>
> </param>
> <param>
> <name>main.pamRealm</name>
> <value>org.apache.hadoop.gateway.shirorealm.
> KnoxPamRealm</value>
> </param>
> <param>
> <name>main.pamRealm.service</name>
> <value>knox</value>
> </param>
> <param>
> <name>urls./**</name>
> <value>authcBasic</value>
> </param>
> </provider>
> <provider>
> <role>identity-assertion</role>
> <name>Default</name>
> <enabled>true</enabled>
> </provider>
> <provider>
> <role>authorization</role>
> <name>XASecurePDPKnox</name>
> <enabled>true</enabled>
> </provider>
>
> The knox pam service is:
>
> auth required pam_env.so
> auth sufficient pam_unix.so nullok try_first_pass
> auth requisite pam_succeed_if.so uid >= 1000 quiet_success
> auth required pam_deny.so
>
> On Sat, Jun 30, 2018 at 9:21 AM, Lian Jiang <ji...@gmail.com> wrote:
>
>> yes. I do both pamtester and curl on the knox host.
>>
>> On Sat, Jun 30, 2018 at 6:36 AM, larry mccay <lm...@apache.org> wrote:
>>
>>> Are you on the Knox host when testing with Pam tester? The accounts will
>>> need to be on the Knox host.
>>>
>>>
>>>
>>> On Sat, Jun 30, 2018, 2:22 AM Lian Jiang <ji...@gmail.com> wrote:
>>>
>>>> I am using OS auth for knox and have verified the username and password
>>>> work:
>>>>
>>>> sudo pamtester -v knox guest authenticate
>>>> pamtester: invoking pam_start(knox, guest, ...)
>>>> pamtester: performing operation - authenticate
>>>> Password:
>>>> pamtester: successfully authenticated
>>>>
>>>> However, my curl command failed:
>>>>
>>>> curl -ik -u guest:"{PASSWORD}" http://test-namenode.subnet1.h
>>>> adoop.oraclevcn.com:8443/gateway/ui/webhdfs/v1/user/?op=LISTSTATUS
>>>>
>>>> The error is:
>>>> Jun 30 06:16:03 test-namenode unix_chkpwd[37385]: check pass; user
>>>> unknown
>>>> Jun 30 06:16:03 test-namenode unix_chkpwd[37385]: password check failed
>>>> for user (guest)
>>>> Jun 30 06:16:03 test-namenode java: pam_unix(knox:auth): authentication
>>>> failure; logname= uid=2018 euid=2018 tty= ruser= rhost= user=guest
>>>>
>>>>
>>>> Any idea how I can debug? Appreciate any help.
>>>>
>>>>
>>>>
>>
>
Re: Knox OS authentication fail due to "password check failed for user"
Posted by Lian Jiang <ji...@gmail.com>.
Furthermore, knoxcli.sh shows guest authentication is ok:
sudo bin/knoxcli.sh user-auth-test --cluster ui --u guest --p "{PASSWORD}"
LDAP authentication successful!
The output shows LDAP but OS auth is used:
<provider>
<role>authentication</role>
<name>ShiroProvider</name>
<enabled>true</enabled>
<param>
<name>sessionTimeout</name>
<value>30</value>
</param>
<param>
<name>main.pamRealm</name>
<value>org.apache.hadoop.gateway.shirorealm.KnoxPamRealm</value>
</param>
<param>
<name>main.pamRealm.service</name>
<value>knox</value>
</param>
<param>
<name>urls./**</name>
<value>authcBasic</value>
</param>
</provider>
<provider>
<role>identity-assertion</role>
<name>Default</name>
<enabled>true</enabled>
</provider>
<provider>
<role>authorization</role>
<name>XASecurePDPKnox</name>
<enabled>true</enabled>
</provider>
The knox pam service is:
auth required pam_env.so
auth sufficient pam_unix.so nullok try_first_pass
auth requisite pam_succeed_if.so uid >= 1000 quiet_success
auth required pam_deny.so
On Sat, Jun 30, 2018 at 9:21 AM, Lian Jiang <ji...@gmail.com> wrote:
> yes. I do both pamtester and curl on the knox host.
>
> On Sat, Jun 30, 2018 at 6:36 AM, larry mccay <lm...@apache.org> wrote:
>
>> Are you on the Knox host when testing with Pam tester? The accounts will
>> need to be on the Knox host.
>>
>>
>>
>> On Sat, Jun 30, 2018, 2:22 AM Lian Jiang <ji...@gmail.com> wrote:
>>
>>> I am using OS auth for knox and have verified the username and password
>>> work:
>>>
>>> sudo pamtester -v knox guest authenticate
>>> pamtester: invoking pam_start(knox, guest, ...)
>>> pamtester: performing operation - authenticate
>>> Password:
>>> pamtester: successfully authenticated
>>>
>>> However, my curl command failed:
>>>
>>> curl -ik -u guest:"{PASSWORD}" http://test-namenode.subnet1.h
>>> adoop.oraclevcn.com:8443/gateway/ui/webhdfs/v1/user/?op=LISTSTATUS
>>>
>>> The error is:
>>> Jun 30 06:16:03 test-namenode unix_chkpwd[37385]: check pass; user
>>> unknown
>>> Jun 30 06:16:03 test-namenode unix_chkpwd[37385]: password check failed
>>> for user (guest)
>>> Jun 30 06:16:03 test-namenode java: pam_unix(knox:auth): authentication
>>> failure; logname= uid=2018 euid=2018 tty= ruser= rhost= user=guest
>>>
>>>
>>> Any idea how I can debug? Appreciate any help.
>>>
>>>
>>>
>
Re: Knox OS authentication fail due to "password check failed for user"
Posted by Lian Jiang <ji...@gmail.com>.
yes. I do both pamtester and curl on the knox host.
On Sat, Jun 30, 2018 at 6:36 AM, larry mccay <lm...@apache.org> wrote:
> Are you on the Knox host when testing with Pam tester? The accounts will
> need to be on the Knox host.
>
>
>
> On Sat, Jun 30, 2018, 2:22 AM Lian Jiang <ji...@gmail.com> wrote:
>
>> I am using OS auth for knox and have verified the username and password
>> work:
>>
>> sudo pamtester -v knox guest authenticate
>> pamtester: invoking pam_start(knox, guest, ...)
>> pamtester: performing operation - authenticate
>> Password:
>> pamtester: successfully authenticated
>>
>> However, my curl command failed:
>>
>> curl -ik -u guest:"{PASSWORD}" http://test-namenode.subnet1.
>> hadoop.oraclevcn.com:8443/gateway/ui/webhdfs/v1/user/?op=LISTSTATUS
>>
>> The error is:
>> Jun 30 06:16:03 test-namenode unix_chkpwd[37385]: check pass; user unknown
>> Jun 30 06:16:03 test-namenode unix_chkpwd[37385]: password check failed
>> for user (guest)
>> Jun 30 06:16:03 test-namenode java: pam_unix(knox:auth): authentication
>> failure; logname= uid=2018 euid=2018 tty= ruser= rhost= user=guest
>>
>>
>> Any idea how I can debug? Appreciate any help.
>>
>>
>>
Re: Knox OS authentication fail due to "password check failed for user"
Posted by larry mccay <lm...@apache.org>.
Are you on the Knox host when testing with Pam tester? The accounts will
need to be on the Knox host.
On Sat, Jun 30, 2018, 2:22 AM Lian Jiang <ji...@gmail.com> wrote:
> I am using OS auth for knox and have verified the username and password
> work:
>
> sudo pamtester -v knox guest authenticate
> pamtester: invoking pam_start(knox, guest, ...)
> pamtester: performing operation - authenticate
> Password:
> pamtester: successfully authenticated
>
> However, my curl command failed:
>
> curl -ik -u guest:"{PASSWORD}"
> http://test-namenode.subnet1.hadoop.oraclevcn.com:8443/gateway/ui/webhdfs/v1/user/?op=LISTSTATUS
>
> The error is:
> Jun 30 06:16:03 test-namenode unix_chkpwd[37385]: check pass; user unknown
> Jun 30 06:16:03 test-namenode unix_chkpwd[37385]: password check failed
> for user (guest)
> Jun 30 06:16:03 test-namenode java: pam_unix(knox:auth): authentication
> failure; logname= uid=2018 euid=2018 tty= ruser= rhost= user=guest
>
>
> Any idea how I can debug? Appreciate any help.
>
>
>