You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@slider.apache.org by Manoj Samel <ma...@gmail.com> on 2016/02/09 02:54:36 UTC
slider 0.80 error on CDH 5.5.1 - "User is not based on a keytab in a
secure deployment"
Hi,
Slider 0.80 running on CDH 5.5.1 secured cluster fails to create slider-AM
with error "User is not based on a keytab in a secure deployment".
1) I had same slider version working without issue on CDH 5.4.2 without any
issues.
2) I thought the issue was in slider 0.81 (
https://issues.apache.org/jira/browse/SLIDER-1010) - is this affecting
slider 0.80 as well ?
Thanks for any feedback ....
Following is full trace
2016-02-09 00:45:25,932 [main] INFO appmaster.SliderAppMaster - Slider AM
Security Mode: KEYTAB
2016-02-09 00:45:25,932 [main] INFO appmaster.SliderAppMaster - Token
YARN_AM_RM_TOKEN
2016-02-09 00:45:25,932 [main] INFO appmaster.SliderAppMaster - Token
HDFS_DELEGATION_TOKEN
2016-02-09 00:45:25,932 [main] INFO appmaster.SliderAppMaster - Token
kms-dt
2016-02-09 00:45:25,951 [main] INFO security.SecurityConfiguration -
Leveraging host keytab file /etc/hadoop/conf/XXX.keytab for login
2016-02-09 00:45:25,968 [main] INFO security.UserGroupInformation - Login
successful for user XXX using keytab file /etc/hadoop/conf/XXX.keytab
2016-02-09 00:45:25,968 [main] ERROR main.ServiceLauncher - User is not
based on a keytab in a secure deployment.
2016-02-09 00:45:25,969 [main] INFO util.ExitUtil - Exiting with status 70
2016-02-09 00:45:25,974 [Thread-1] INFO ipc.Server - Stopping server on
1024
2016-02-09 00:45:25,975 [IPC Server listener on 1024] INFO ipc.Server -
Stopping IPC Server listener on 1024
2016-02-09 00:45:25,975 [IPC Server Responder] INFO ipc.Server - Stopping
IPC Server Responder
2016-02-09 00:45:25,975 [Thread-1] INFO appmaster.SliderAppMaster -
Process has exited with exit code 0 mapped to 0 -ignoring
2016-02-09 00:45:25,975 [AMRM Callback Handler Thread] INFO
impl.AMRMClientAsyncImpl - Interrupted while waiting for queue
java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at
org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:274)
Re: slider 0.80 error on CDH 5.5.1 - "User is not based on a keytab
in a secure deployment"
Posted by Manoj Samel <ma...@gmail.com>.
Hi Steve,
1. Regarding your comment <<<"From cloudera: "it showed up somewhere
around 5.3.0. It's not in any of the 5.2.x releases. It's definitely in
5.5.1.">>> -- sorry, I am not clean what you are referring to ? Are you
saying that HADOOP-10786 IS in cdh 5.5.1 ?
2. I had opened SLIDER-1067 for backport of slider fix to .82 release. Any
idea when 0.82 will be available ?
Thanks,
On Wed, Feb 10, 2016 at 3:45 AM, Steve Loughran <st...@hortonworks.com>
wrote:
>
> > On 9 Feb 2016, at 01:54, Manoj Samel <ma...@gmail.com> wrote:
> >
> > Hi,
> >
> > Slider 0.80 running on CDH 5.5.1 secured cluster fails to create
> slider-AM
> > with error "User is not based on a keytab in a secure deployment".
> >
> > 1) I had same slider version working without issue on CDH 5.4.2 without
> any
> > issues.
> > 2) I thought the issue was in slider 0.81 (
> > https://issues.apache.org/jira/browse/SLIDER-1010) - is this affecting
> > slider 0.80 as well ?
> >
>
> From cloudera: "it showed up somewhere around 5.3.0. It's not in any of
> the 5.2.x releases. It's definitely in 5.5.1."
>
>
>
>
Re: slider 0.80 error on CDH 5.5.1 - "User is not based on a keytab
in a secure deployment"
Posted by Steve Loughran <st...@hortonworks.com>.
> On 9 Feb 2016, at 01:54, Manoj Samel <ma...@gmail.com> wrote:
>
> Hi,
>
> Slider 0.80 running on CDH 5.5.1 secured cluster fails to create slider-AM
> with error "User is not based on a keytab in a secure deployment".
>
> 1) I had same slider version working without issue on CDH 5.4.2 without any
> issues.
> 2) I thought the issue was in slider 0.81 (
> https://issues.apache.org/jira/browse/SLIDER-1010) - is this affecting
> slider 0.80 as well ?
>
>From cloudera: "it showed up somewhere around 5.3.0. It's not in any of the 5.2.x releases. It's definitely in 5.5.1."
Re: slider 0.80 error on CDH 5.5.1 - "User is not based on a keytab
in a secure deployment"
Posted by Jon Maron <jm...@hortonworks.com>.
The slider fix was simply a logging improvement, I believe. The real issue is the modification of the way the JDK changed the kerberos logging module in latter versions of JDK 7 and JDK 8 (https://issues.apache.org/jira/browse/HADOOP-10786). I assume that your version of CDH is manifesting this latter issue. Can you verify whether it has the fix for the given hadoop JIRA?
— Jon
On Feb 9, 2016, at 12:22 PM, Manoj Samel <ma...@gmail.com>> wrote:
Any help on this will be appreciated !!!
Thanks,
Manoj
On Mon, Feb 8, 2016 at 5:54 PM, Manoj Samel <ma...@gmail.com>>
wrote:
Hi,
Slider 0.80 running on CDH 5.5.1 secured cluster fails to create slider-AM
with error "User is not based on a keytab in a secure deployment".
1) I had same slider version working without issue on CDH 5.4.2 without
any issues.
2) I thought the issue was in slider 0.81 (
https://issues.apache.org/jira/browse/SLIDER-1010) - is this affecting
slider 0.80 as well ?
Thanks for any feedback ....
Following is full trace
2016-02-09 00:45:25,932 [main] INFO appmaster.SliderAppMaster - Slider AM
Security Mode: KEYTAB
2016-02-09 00:45:25,932 [main] INFO appmaster.SliderAppMaster - Token
YARN_AM_RM_TOKEN
2016-02-09 00:45:25,932 [main] INFO appmaster.SliderAppMaster - Token
HDFS_DELEGATION_TOKEN
2016-02-09 00:45:25,932 [main] INFO appmaster.SliderAppMaster - Token
kms-dt
2016-02-09 00:45:25,951 [main] INFO security.SecurityConfiguration -
Leveraging host keytab file /etc/hadoop/conf/XXX.keytab for login
2016-02-09 00:45:25,968 [main] INFO security.UserGroupInformation - Login
successful for user XXX using keytab file /etc/hadoop/conf/XXX.keytab
2016-02-09 00:45:25,968 [main] ERROR main.ServiceLauncher - User is not
based on a keytab in a secure deployment.
2016-02-09 00:45:25,969 [main] INFO util.ExitUtil - Exiting with status 70
2016-02-09 00:45:25,974 [Thread-1] INFO ipc.Server - Stopping server on
1024
2016-02-09 00:45:25,975 [IPC Server listener on 1024] INFO ipc.Server -
Stopping IPC Server listener on 1024
2016-02-09 00:45:25,975 [IPC Server Responder] INFO ipc.Server - Stopping
IPC Server Responder
2016-02-09 00:45:25,975 [Thread-1] INFO appmaster.SliderAppMaster -
Process has exited with exit code 0 mapped to 0 -ignoring
2016-02-09 00:45:25,975 [AMRM Callback Handler Thread] INFO
impl.AMRMClientAsyncImpl - Interrupted while waiting for queue
java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at
org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:274)
Re: slider 0.80 error on CDH 5.5.1 - "User is not based on a keytab
in a secure deployment"
Posted by Manoj Samel <ma...@gmail.com>.
Any help on this will be appreciated !!!
Thanks,
Manoj
On Mon, Feb 8, 2016 at 5:54 PM, Manoj Samel <ma...@gmail.com>
wrote:
> Hi,
>
> Slider 0.80 running on CDH 5.5.1 secured cluster fails to create slider-AM
> with error "User is not based on a keytab in a secure deployment".
>
> 1) I had same slider version working without issue on CDH 5.4.2 without
> any issues.
> 2) I thought the issue was in slider 0.81 (
> https://issues.apache.org/jira/browse/SLIDER-1010) - is this affecting
> slider 0.80 as well ?
>
> Thanks for any feedback ....
>
>
>
> Following is full trace
>
> 2016-02-09 00:45:25,932 [main] INFO appmaster.SliderAppMaster - Slider AM
> Security Mode: KEYTAB
> 2016-02-09 00:45:25,932 [main] INFO appmaster.SliderAppMaster - Token
> YARN_AM_RM_TOKEN
> 2016-02-09 00:45:25,932 [main] INFO appmaster.SliderAppMaster - Token
> HDFS_DELEGATION_TOKEN
> 2016-02-09 00:45:25,932 [main] INFO appmaster.SliderAppMaster - Token
> kms-dt
> 2016-02-09 00:45:25,951 [main] INFO security.SecurityConfiguration -
> Leveraging host keytab file /etc/hadoop/conf/XXX.keytab for login
> 2016-02-09 00:45:25,968 [main] INFO security.UserGroupInformation - Login
> successful for user XXX using keytab file /etc/hadoop/conf/XXX.keytab
> 2016-02-09 00:45:25,968 [main] ERROR main.ServiceLauncher - User is not
> based on a keytab in a secure deployment.
> 2016-02-09 00:45:25,969 [main] INFO util.ExitUtil - Exiting with status 70
> 2016-02-09 00:45:25,974 [Thread-1] INFO ipc.Server - Stopping server on
> 1024
> 2016-02-09 00:45:25,975 [IPC Server listener on 1024] INFO ipc.Server -
> Stopping IPC Server listener on 1024
> 2016-02-09 00:45:25,975 [IPC Server Responder] INFO ipc.Server - Stopping
> IPC Server Responder
> 2016-02-09 00:45:25,975 [Thread-1] INFO appmaster.SliderAppMaster -
> Process has exited with exit code 0 mapped to 0 -ignoring
> 2016-02-09 00:45:25,975 [AMRM Callback Handler Thread] INFO
> impl.AMRMClientAsyncImpl - Interrupted while waiting for queue
> java.lang.InterruptedException
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:274)
>
Re: slider 0.80 error on CDH 5.5.1 - "User is not based on a keytab
in a secure deployment"
Posted by Steve Loughran <st...@hortonworks.com>.
> On 9 Feb 2016, at 01:54, Manoj Samel <ma...@gmail.com> wrote:
>
> Hi,
>
> Slider 0.80 running on CDH 5.5.1 secured cluster fails to create slider-AM
> with error "User is not based on a keytab in a secure deployment".
>
> 1) I had same slider version working without issue on CDH 5.4.2 without any
> issues.
> 2) I thought the issue was in slider 0.81 (
> https://issues.apache.org/jira/browse/SLIDER-1010) - is this affecting
> slider 0.80 as well ?
>
we haven't backported it, by the look of things.
I've been dealing with other kerberos grief on 0.90.2; addressed now, SLIDER-1026. You can't reliably use slider 0.90.2 on a secure cluster.
We need to do that new release we've been talking about...