You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@livy.apache.org by "Jiang, Yi J (CWM-NR)" <yi...@rbc.com> on 2018/10/10 21:12:38 UTC

Kerberos issue

Hello, Guys,
Who can help me about this issue.
I want to run Apache Livy in my dev box, I did not modify anything in the livy.conf except append the kerberos info in there

livy.impersonation.enabled = true
livy.server.auth.type = kerberos
livy.server.auth.kerberos.keytab = /home/username/hadoop/xxxxxxx.keytab
livy.server.auth.kerberos.principal = xxxxxxx@xxxxx.COM<ma...@xxxxx.COM>
livy.server.launch.kerberos.keytab = /home/admrave/hadoop/ xxxxxxx.keytab
livy.server.launch.kerberos.principal = xxxxxxx@xxxxxxx.COM<ma...@xxxxxxx.COM>

I can successfully use kinit -kt /home/username/hadoop/xxxxxxx.keytab xxxxxxx@xxxxx.COM<ma...@xxxxx.COM> to authenticate, I also cleaned up the kerberos cache, but once I start Livy in command line then it gives following error:

18/10/10 16:41:52 INFO server.LivyServer: SPNEGO auth enabled (principal = xxxxxxx@xxxxxxx.COM<ma...@xxxxxxx.COM>)
18/10/10 16:41:52 INFO server.Server: jetty-9.2.16.v20160414
18/10/10 16:41:52 INFO server.KerberosAuthenticationHandler: Login using keytab /home/admrave/hadoop/ xxxxxxx.keytab, for principal xxxxxxx@xxxxxxx.COM<ma...@xxxxxxx.COM>
18/10/10 16:41:52 WARN server.KerberosAuthenticationHandler: Failed to login as [xxxxxxx@xxxxxxx.COM]
javax.security.auth.login.LoginException: No key to store
        at com.sun.security.auth.module.Krb5LoginModule.commit(Krb5LoginModule.java:1119)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
        at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
        at java.security.AccessController.doPrivileged(Native Method)
       at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
        at javax.security.auth.login.LoginContext.login(LoginContext.java:588)
        at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:221)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.initializeAuthHandler(AuthenticationFilter.java:237)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:226)
        at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:138)
        at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:852)
        at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
        at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.server.Server.start(Server.java:387)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.server.Server.doStart(Server.java:354)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.apache.livy.server.WebServer.start(WebServer.scala:92)
        at org.apache.livy.server.LivyServer.start(LivyServer.scala:259)
        at org.apache.livy.server.LivyServer$.main(LivyServer.scala:339)
        at org.apache.livy.server.LivyServer.main(LivyServer.scala)
18/10/10 16:41:52 WARN component.AbstractLifeCycle: FAILED o.e.j.s.ServletContextHandler@1c3146bc{/,file:/tmp/livy-0.5.0-incubating-bin/src/main/org/apache/livy/server,STARTING}<mailto:o.e.j.s.ServletContextHandler@1c3146bc%7b/,file:/tmp/livy-0.5.0-incubating-bin/src/main/org/apache/livy/server,STARTING%7d>: javax.servlet.ServletException: org.apache.hadoop.security.authentication.client.AuthenticationException: javax.security.auth.login.Log  inException: No key to store
javax.servlet.ServletException: org.apache.hadoop.security.authentication.client.AuthenticationException: javax.security.auth.login.LoginException: No key to   store
        at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:240)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.initializeAuthHandler(AuthenticationFilter.java:237)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:226)
        at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:138)
        at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:852)
        at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
        at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.server.Server.start(Server.java:387)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.server.Server.doStart(Server.java:354)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.apache.livy.server.WebServer.start(WebServer.scala:92)
        at org.apache.livy.server.LivyServer.start(LivyServer.scala:259)
        at org.apache.livy.server.LivyServer$.main(LivyServer.scala:339)
        at org.apache.livy.server.LivyServer.main(LivyServer.scala)
Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: javax.security.auth.login.LoginException: No key to store
        at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:224)
        ... 21 more
Caused by: javax.security.auth.login.LoginException: No key to store

Please let me know how to fix that ASAP. Thank you
Jacky
_______________________________________________________________________
If you received this email in error, please advise the sender (by return email or otherwise) immediately. You have consented to receive the attached electronically at the above-noted email address; please retain a copy of this confirmation for future reference.  

Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur immédiatement, par retour de courriel ou par un autre moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de cette confirmation pour les fins de reference future.

RE: Kerberos issue

Posted by "Jiang, Yi J (CWM-NR)" <yi...@rbc.com>.
Revised
I have both exact same keytab and principal in livy.server.auth.kerberos and livy.server.launch.kerberos

From: Jiang, Yi J (CWM-NR) [mailto:yi.j.jiang@rbc.com]
Sent: October 10, 18 5:13 PM
To: user@livy.incubator.apache.org
Subject: Kerberos issue


Hello, Guys,
Who can help me about this issue.
I want to run Apache Livy in my dev box, I did not modify anything in the livy.conf except append the kerberos info in there

livy.impersonation.enabled = true
livy.server.auth.type = kerberos
livy.server.auth.kerberos.keytab = /home/username/hadoop/xxxxxxx.keytab
livy.server.auth.kerberos.principal = xxxxxxx@xxxxx.COM<ma...@xxxxx.COM>
livy.server.launch.kerberos.keytab = /home/ username/hadoop/ xxxxxxx.keytab
livy.server.launch.kerberos.principal = xxxxxxx@xxxxxxx.COM<ma...@xxxxxxx.COM>

I can successfully use kinit –kt /home/username/hadoop/xxxxxxx.keytab xxxxxxx@xxxxx.COM<ma...@xxxxx.COM> to authenticate, I also cleaned up the kerberos cache, but once I start Livy in command line then it gives following error:

18/10/10 16:41:52 INFO server.LivyServer: SPNEGO auth enabled (principal = xxxxxxx@xxxxxxx.COM<ma...@xxxxxxx.COM>)
18/10/10 16:41:52 INFO server.Server: jetty-9.2.16.v20160414
18/10/10 16:41:52 INFO server.KerberosAuthenticationHandler: Login using keytab /home/admrave/hadoop/ xxxxxxx.keytab, for principal xxxxxxx@xxxxxxx.COM<ma...@xxxxxxx.COM>
18/10/10 16:41:52 WARN server.KerberosAuthenticationHandler: Failed to login as [xxxxxxx@xxxxxxx.COM]
javax.security.auth.login.LoginException: No key to store
        at com.sun.security.auth.module.Krb5LoginModule.commit(Krb5LoginModule.java:1119)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
        at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
        at java.security.AccessController.doPrivileged(Native Method)
       at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
        at javax.security.auth.login.LoginContext.login(LoginContext.java:588)
        at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:221)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.initializeAuthHandler(AuthenticationFilter.java:237)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:226)
        at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:138)
        at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:852)
        at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
        at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.server.Server.start(Server.java:387)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.server.Server.doStart(Server.java:354)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.apache.livy.server.WebServer.start(WebServer.scala:92)
        at org.apache.livy.server.LivyServer.start(LivyServer.scala:259)
        at org.apache.livy.server.LivyServer$.main(LivyServer.scala:339)
        at org.apache.livy.server.LivyServer.main(LivyServer.scala)
18/10/10 16:41:52 WARN component.AbstractLifeCycle: FAILED o.e.j.s.ServletContextHandler@1c3146bc{/,file:/tmp/livy-0.5.0-incubating-bin/src/main/org/apache/livy/server,STARTING}<mailto:o.e.j.s.ServletContextHandler@1c3146bc%7b/,file:/tmp/livy-0.5.0-incubating-bin/src/main/org/apache/livy/server,STARTING%7d>: javax.servlet.ServletException: org.apache.hadoop.security.authentication.client.AuthenticationException: javax.security.auth.login.Log  inException: No key to store
javax.servlet.ServletException: org.apache.hadoop.security.authentication.client.AuthenticationException: javax.security.auth.login.LoginException: No key to   store
        at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:240)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.initializeAuthHandler(AuthenticationFilter.java:237)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:226)
        at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:138)
        at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:852)
        at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
        at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.server.Server.start(Server.java:387)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.server.Server.doStart(Server.java:354)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.apache.livy.server.WebServer.start(WebServer.scala:92)
        at org.apache.livy.server.LivyServer.start(LivyServer.scala:259)
        at org.apache.livy.server.LivyServer$.main(LivyServer.scala:339)
        at org.apache.livy.server.LivyServer.main(LivyServer.scala)
Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: javax.security.auth.login.LoginException: No key to store
        at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:224)
        ... 21 more
Caused by: javax.security.auth.login.LoginException: No key to store

Please let me know how to fix that ASAP. Thank you
Jacky

_______________________________________________________________________

If you received this email in error, please advise the sender (by return email or otherwise) immediately. You have consented to receive the attached electronically at the above-noted email address; please retain a copy of this confirmation for future reference.

Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur immédiatement, par retour de courriel ou par un autre moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de cette confirmation pour les fins de reference future.
_______________________________________________________________________
If you received this email in error, please advise the sender (by return email or otherwise) immediately. You have consented to receive the attached electronically at the above-noted email address; please retain a copy of this confirmation for future reference.  

Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur immédiatement, par retour de courriel ou par un autre moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de cette confirmation pour les fins de reference future.