You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@slider.apache.org by Yohan Bismuth <yo...@gmail.com> on 2015/05/28 18:59:45 UTC
Keytab issue with Hbase secure
Hi,
i'm facing an issue with Hbase in secure mode.
I followed the steps described on
http://slider.incubator.apache.org/docs/security.html
i created my headless keytab (and the associated principals), which i
deployed on hdfs and when i start an hbase application, the keytab is
correctly packaged in the SliderAppMaster container under the keytabs
folder, but here is the problem:
2015-05-28 16:03:07,037 [main] INFO appmaster.SliderAppMaster - Connecting
> to RM at 1024,address tracking URL=
> http://a4-5d-36-fd-a1-7c.hpc.criteo.preprod:1025
> 2015-05-28 16:03:07,065 [main] INFO appmaster.SliderAppMaster - Slider AM
> Security Mode: KEYTAB
> 2015-05-28 16:03:07,065 [main] INFO appmaster.SliderAppMaster - Token
> HDFS_DELEGATION_TOKEN
> 2015-05-28 16:03:07,065 [main] INFO appmaster.SliderAppMaster - Token
> YARN_AM_RM_TOKEN
> 2015-05-28 16:03:07,093 [main] INFO security.SecurityConfiguration - No
> host keytab file path specified. Will attempt to retrieve keytab file
> y.bismuth.keytab as a local resource for the container
> 2015-05-28 16:03:07,104 [main] INFO security.UserGroupInformation - Login
> successful for user y.bismuth using keytab file
> /hdfs/wwn/600508b1001c246eb94fcc5ff4d68b4e/yarn/data/usercache/y.bismuth/appcache/application_1432038882976_2039/container_e11_1432038882976_2039_01_000001/keytabs/y.bismuth.keytab
> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster - security
> enabled = true
>
> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
> <<<<<<SOME DEBUG
> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster - UGI =
> y.bismuth@HPC.CRITEO.PREPROD (auth:KERBEROS)
> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster - isKeytab
> = false
> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster - tokens =
> []
> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
> hasKerberosCredentials = true
> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
> credentials = org.apache.hadoop.security.Credentials@1cf2fed4
> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
> authentication method = KERBEROS
> 2015-05-28 16:03:07,111 [main] INFO appmaster.SliderAppMaster - config =
> Configuration: core-default.xml, core-site.xml, yarn-default.xml,
> yarn-site.xml, hdfs-default.xml, hdfs-site.xml,
> org/apache/slider/slider.xml, mapred-default.xml, mapred-site.xml
> 2015-05-28 16:03:07,111 [main] INFO appmaster.SliderAppMaster - SOME
> DEBUG>>>>>
>
> 2015-05-28 16:03:07,112 [main] ERROR main.ServiceLauncher - User is not
> based on a keytab in a secure deployment.
So as far as i can see, i'm logging in successfully using the keytab
packaged in the container, but the flag isKeytab, which should be set to
true in my UGI (i hope), is not, and i can't figure out why. Because of
that, my SliderAppMaster crash.
Any idea ?
Re: Keytab issue with Hbase secure
Posted by Jon Maron <jm...@hortonworks.com>.
That is strange - the login is from a keytab, but the UGI indicates it’s from a ticket. I’ll need to investigate some more. Can you send me the full log? I imagine the listserv will not allow it. Thanks!
> On May 29, 2015, at 5:24 AM, Yohan Bismuth <yo...@gmail.com> wrote:
>
> Btw, i've tried using java7 and java8: smae issue.
> I'm correctly logged as the principal of my keytab, and i can submit jobs
> (like a wordcount) using this keytab.
>
> If i remove this line from the code:
>
>> validateLoginUser(UserGroupInformation.getLoginUser());
>
> everything seems to work fine.
>
> hasKerberosCredentials returns true, so the login must be based on a
>> kerberos ticket. Perhaps it has expired?
>
> Well, this is the point here, i don't want the login to be based on a
> kerberos ticket (because it would mean the ugi has the wrong flag set to
> true). I want the login to be based on a kerberos key.
>
>
> 2015-05-29 8:49 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
>
>> C'est pas très grave pour nous c'est packagé à la cradoc pour l'instant.
>> C'est peut-etre aussi le fix ! Faudrait faire un git blame pour comprendre
>> pourquoi l'exception est là. Elle trigge peut-etre aussi dans un cas
>> légitime ?
>>
>> Question subsidiaire: tu soumets bien le job en tant que
>> y.bismuth@HPC.CRITEO.PREPROD et pas y.bismuth@CRITEOIS.LAN ? (c'est la
>> meme question que: tu as fait kinit en plus subtil)
>> Tu as essayé avec un krb5.conf qui mette bien HPC.CRITEO.PREPROD en
>> default realm sur la gateway.
>>
>> JB
>>
>> 2015-05-28 23:05 GMT+02:00 Yohan Bismuth <yo...@gmail.com>:
>>
>>> Bah ça implique de modifier et recompiler slider. Je sais pas si on veut
>>> ça...
>>>
>>> 2015-05-28 22:20 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
>>>
>>>> OK j'ai rien dit :)
>>>> Tu peux spécifier tout ça dans le thread, si vraiment ça fonctionne sans
>>>> l'exception, je vois même pas pourquoi on s'embête en fait ???
>>>>
>>>> JB
>>>>
>>>
>>>
>>
>>
>> --
>> Jean-Baptiste Note
>>
Re: Keytab issue with Hbase secure
Posted by Jon Maron <jm...@hortonworks.com>.
I’ll admit to being stumped. Looking for input from others…
> On May 29, 2015, at 9:46 AM, Yohan Bismuth <yo...@gmail.com> wrote:
>
> I just recompiled slider after i replaced the call to isFromKeytab()
> in validateLoginUser by isLoginKeytabBased(). Then i redeployed it on my
> gateway and launched my application.
> The error is the same as before.
>
> On Fri, May 29, 2015 at 3:36 PM, Jon Maron <jm...@hortonworks.com> wrote:
>
>> Please describe the full procedure you used to retest.
>>
>>> On May 29, 2015, at 9:28 AM, Yohan Bismuth <yo...@gmail.com>
>> wrote:
>>>
>>> ...but i got the same problem using this method instead
>>>
>>> On Fri, May 29, 2015 at 3:25 PM, Yohan Bismuth <yohan.bismuth1@gmail.com
>>>
>>> wrote:
>>>
>>>> oops my bad, was not looking the good thing
>>>>
>>>> On Fri, May 29, 2015 at 3:13 PM, Jon Maron <jm...@hortonworks.com>
>> wrote:
>>>>
>>>>> It’s declared as public static:
>>>>>
>>>>> public synchronized static boolean isLoginKeytabBased()
>>>>>
>>>>> at least the version I’m looking at
>>>>>
>>>>>> On May 29, 2015, at 8:52 AM, Yohan Bismuth <yo...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Mmh I can't compile this. I don't think you can use
>> isLoginKeytabBased()
>>>>>> since isKeytab is private in ugi.
>>>>>>
>>>>>> On Fri, May 29, 2015 at 1:56 PM, Jon Maron <jm...@hortonworks.com>
>>>>> wrote:
>>>>>>
>>>>>>> Looks like you’ve found a bug: validateLoginUser should be calling
>>>>>>> isLoginKeytabBased(), not isFromKeytab(). Would mind filing a JIRA?
>>>>>>>
>>>>>>> — Jon
>>>>>>>
>>>>>>>> On May 29, 2015, at 5:24 AM, Yohan Bismuth <
>> yohan.bismuth1@gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Btw, i've tried using java7 and java8: smae issue.
>>>>>>>> I'm correctly logged as the principal of my keytab, and i can submit
>>>>> jobs
>>>>>>>> (like a wordcount) using this keytab.
>>>>>>>>
>>>>>>>> If i remove this line from the code:
>>>>>>>>
>>>>>>>>> validateLoginUser(UserGroupInformation.getLoginUser());
>>>>>>>>
>>>>>>>> everything seems to work fine.
>>>>>>>>
>>>>>>>> hasKerberosCredentials returns true, so the login must be based on a
>>>>>>>>> kerberos ticket. Perhaps it has expired?
>>>>>>>>
>>>>>>>> Well, this is the point here, i don't want the login to be based on
>> a
>>>>>>>> kerberos ticket (because it would mean the ugi has the wrong flag
>> set
>>>>> to
>>>>>>>> true). I want the login to be based on a kerberos key.
>>>>>>>>
>>>>>>>>
>>>>>>>> 2015-05-29 8:49 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
>>>>>>>>
>>>>>>>>> C'est pas très grave pour nous c'est packagé à la cradoc pour
>>>>> l'instant.
>>>>>>>>> C'est peut-etre aussi le fix ! Faudrait faire un git blame pour
>>>>>>> comprendre
>>>>>>>>> pourquoi l'exception est là. Elle trigge peut-etre aussi dans un
>> cas
>>>>>>>>> légitime ?
>>>>>>>>>
>>>>>>>>> Question subsidiaire: tu soumets bien le job en tant que
>>>>>>>>> y.bismuth@HPC.CRITEO.PREPROD et pas y.bismuth@CRITEOIS.LAN ?
>> (c'est
>>>>> la
>>>>>>>>> meme question que: tu as fait kinit en plus subtil)
>>>>>>>>> Tu as essayé avec un krb5.conf qui mette bien HPC.CRITEO.PREPROD en
>>>>>>>>> default realm sur la gateway.
>>>>>>>>>
>>>>>>>>> JB
>>>>>>>>>
>>>>>>>>> 2015-05-28 23:05 GMT+02:00 Yohan Bismuth <yohan.bismuth1@gmail.com
>>> :
>>>>>>>>>
>>>>>>>>>> Bah ça implique de modifier et recompiler slider. Je sais pas si
>> on
>>>>>>> veut
>>>>>>>>>> ça...
>>>>>>>>>>
>>>>>>>>>> 2015-05-28 22:20 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
>>>>>>>>>>
>>>>>>>>>>> OK j'ai rien dit :)
>>>>>>>>>>> Tu peux spécifier tout ça dans le thread, si vraiment ça
>> fonctionne
>>>>>>> sans
>>>>>>>>>>> l'exception, je vois même pas pourquoi on s'embête en fait ???
>>>>>>>>>>>
>>>>>>>>>>> JB
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Jean-Baptiste Note
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>
>>
>>
Re: Keytab issue with Hbase secure
Posted by Yohan Bismuth <yo...@gmail.com>.
Hi, it seems to fix the issue :) However, like Jean Baptiste said, it
shouldn"t be a problem in cdh5.4.0.
I'll try to dig it a little more.
Anyway, thank you Jon
On Mon, Jun 1, 2015 at 10:53 AM, Jean-Baptiste Note <jb...@gmail.com>
wrote:
> Hi,
>
> Thanks for the heads up.
>
> We're running on JDK8, however, it looks like 10786 is in CDH5.4.0 which
> we're using (and compiled against); see:
>
> http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.4.0.releasenotes.html
>
> This is so close a problem however that we'll dig in this direction.
>
> Kind regards,
> JB
>
Re: Keytab issue with Hbase secure
Posted by Jean-Baptiste Note <jb...@gmail.com>.
Hi,
Thanks for the heads up.
We're running on JDK8, however, it looks like 10786 is in CDH5.4.0 which
we're using (and compiled against); see:
http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.4.0.releasenotes.html
This is so close a problem however that we'll dig in this direction.
Kind regards,
JB
Re: Keytab issue with Hbase secure
Posted by Jon Maron <jm...@hortonworks.com>.
Are you running on JDK 8? Could be this: https://issues.apache.org/jira/browse/HADOOP-10786
> On May 29, 2015, at 9:46 AM, Yohan Bismuth <yo...@gmail.com> wrote:
>
> I just recompiled slider after i replaced the call to isFromKeytab()
> in validateLoginUser by isLoginKeytabBased(). Then i redeployed it on my
> gateway and launched my application.
> The error is the same as before.
>
> On Fri, May 29, 2015 at 3:36 PM, Jon Maron <jm...@hortonworks.com> wrote:
>
>> Please describe the full procedure you used to retest.
>>
>>> On May 29, 2015, at 9:28 AM, Yohan Bismuth <yo...@gmail.com>
>> wrote:
>>>
>>> ...but i got the same problem using this method instead
>>>
>>> On Fri, May 29, 2015 at 3:25 PM, Yohan Bismuth <yohan.bismuth1@gmail.com
>>>
>>> wrote:
>>>
>>>> oops my bad, was not looking the good thing
>>>>
>>>> On Fri, May 29, 2015 at 3:13 PM, Jon Maron <jm...@hortonworks.com>
>> wrote:
>>>>
>>>>> It’s declared as public static:
>>>>>
>>>>> public synchronized static boolean isLoginKeytabBased()
>>>>>
>>>>> at least the version I’m looking at
>>>>>
>>>>>> On May 29, 2015, at 8:52 AM, Yohan Bismuth <yo...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Mmh I can't compile this. I don't think you can use
>> isLoginKeytabBased()
>>>>>> since isKeytab is private in ugi.
>>>>>>
>>>>>> On Fri, May 29, 2015 at 1:56 PM, Jon Maron <jm...@hortonworks.com>
>>>>> wrote:
>>>>>>
>>>>>>> Looks like you’ve found a bug: validateLoginUser should be calling
>>>>>>> isLoginKeytabBased(), not isFromKeytab(). Would mind filing a JIRA?
>>>>>>>
>>>>>>> — Jon
>>>>>>>
>>>>>>>> On May 29, 2015, at 5:24 AM, Yohan Bismuth <
>> yohan.bismuth1@gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Btw, i've tried using java7 and java8: smae issue.
>>>>>>>> I'm correctly logged as the principal of my keytab, and i can submit
>>>>> jobs
>>>>>>>> (like a wordcount) using this keytab.
>>>>>>>>
>>>>>>>> If i remove this line from the code:
>>>>>>>>
>>>>>>>>> validateLoginUser(UserGroupInformation.getLoginUser());
>>>>>>>>
>>>>>>>> everything seems to work fine.
>>>>>>>>
>>>>>>>> hasKerberosCredentials returns true, so the login must be based on a
>>>>>>>>> kerberos ticket. Perhaps it has expired?
>>>>>>>>
>>>>>>>> Well, this is the point here, i don't want the login to be based on
>> a
>>>>>>>> kerberos ticket (because it would mean the ugi has the wrong flag
>> set
>>>>> to
>>>>>>>> true). I want the login to be based on a kerberos key.
>>>>>>>>
>>>>>>>>
>>>>>>>> 2015-05-29 8:49 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
>>>>>>>>
>>>>>>>>> C'est pas très grave pour nous c'est packagé à la cradoc pour
>>>>> l'instant.
>>>>>>>>> C'est peut-etre aussi le fix ! Faudrait faire un git blame pour
>>>>>>> comprendre
>>>>>>>>> pourquoi l'exception est là. Elle trigge peut-etre aussi dans un
>> cas
>>>>>>>>> légitime ?
>>>>>>>>>
>>>>>>>>> Question subsidiaire: tu soumets bien le job en tant que
>>>>>>>>> y.bismuth@HPC.CRITEO.PREPROD et pas y.bismuth@CRITEOIS.LAN ?
>> (c'est
>>>>> la
>>>>>>>>> meme question que: tu as fait kinit en plus subtil)
>>>>>>>>> Tu as essayé avec un krb5.conf qui mette bien HPC.CRITEO.PREPROD en
>>>>>>>>> default realm sur la gateway.
>>>>>>>>>
>>>>>>>>> JB
>>>>>>>>>
>>>>>>>>> 2015-05-28 23:05 GMT+02:00 Yohan Bismuth <yohan.bismuth1@gmail.com
>>> :
>>>>>>>>>
>>>>>>>>>> Bah ça implique de modifier et recompiler slider. Je sais pas si
>> on
>>>>>>> veut
>>>>>>>>>> ça...
>>>>>>>>>>
>>>>>>>>>> 2015-05-28 22:20 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
>>>>>>>>>>
>>>>>>>>>>> OK j'ai rien dit :)
>>>>>>>>>>> Tu peux spécifier tout ça dans le thread, si vraiment ça
>> fonctionne
>>>>>>> sans
>>>>>>>>>>> l'exception, je vois même pas pourquoi on s'embête en fait ???
>>>>>>>>>>>
>>>>>>>>>>> JB
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Jean-Baptiste Note
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>
>>
>>
Re: Keytab issue with Hbase secure
Posted by Yohan Bismuth <yo...@gmail.com>.
I just recompiled slider after i replaced the call to isFromKeytab()
in validateLoginUser by isLoginKeytabBased(). Then i redeployed it on my
gateway and launched my application.
The error is the same as before.
On Fri, May 29, 2015 at 3:36 PM, Jon Maron <jm...@hortonworks.com> wrote:
> Please describe the full procedure you used to retest.
>
> > On May 29, 2015, at 9:28 AM, Yohan Bismuth <yo...@gmail.com>
> wrote:
> >
> > ...but i got the same problem using this method instead
> >
> > On Fri, May 29, 2015 at 3:25 PM, Yohan Bismuth <yohan.bismuth1@gmail.com
> >
> > wrote:
> >
> >> oops my bad, was not looking the good thing
> >>
> >> On Fri, May 29, 2015 at 3:13 PM, Jon Maron <jm...@hortonworks.com>
> wrote:
> >>
> >>> It’s declared as public static:
> >>>
> >>> public synchronized static boolean isLoginKeytabBased()
> >>>
> >>> at least the version I’m looking at
> >>>
> >>>> On May 29, 2015, at 8:52 AM, Yohan Bismuth <yo...@gmail.com>
> >>> wrote:
> >>>>
> >>>> Mmh I can't compile this. I don't think you can use
> isLoginKeytabBased()
> >>>> since isKeytab is private in ugi.
> >>>>
> >>>> On Fri, May 29, 2015 at 1:56 PM, Jon Maron <jm...@hortonworks.com>
> >>> wrote:
> >>>>
> >>>>> Looks like you’ve found a bug: validateLoginUser should be calling
> >>>>> isLoginKeytabBased(), not isFromKeytab(). Would mind filing a JIRA?
> >>>>>
> >>>>> — Jon
> >>>>>
> >>>>>> On May 29, 2015, at 5:24 AM, Yohan Bismuth <
> yohan.bismuth1@gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>> Btw, i've tried using java7 and java8: smae issue.
> >>>>>> I'm correctly logged as the principal of my keytab, and i can submit
> >>> jobs
> >>>>>> (like a wordcount) using this keytab.
> >>>>>>
> >>>>>> If i remove this line from the code:
> >>>>>>
> >>>>>>> validateLoginUser(UserGroupInformation.getLoginUser());
> >>>>>>
> >>>>>> everything seems to work fine.
> >>>>>>
> >>>>>> hasKerberosCredentials returns true, so the login must be based on a
> >>>>>>> kerberos ticket. Perhaps it has expired?
> >>>>>>
> >>>>>> Well, this is the point here, i don't want the login to be based on
> a
> >>>>>> kerberos ticket (because it would mean the ugi has the wrong flag
> set
> >>> to
> >>>>>> true). I want the login to be based on a kerberos key.
> >>>>>>
> >>>>>>
> >>>>>> 2015-05-29 8:49 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
> >>>>>>
> >>>>>>> C'est pas très grave pour nous c'est packagé à la cradoc pour
> >>> l'instant.
> >>>>>>> C'est peut-etre aussi le fix ! Faudrait faire un git blame pour
> >>>>> comprendre
> >>>>>>> pourquoi l'exception est là. Elle trigge peut-etre aussi dans un
> cas
> >>>>>>> légitime ?
> >>>>>>>
> >>>>>>> Question subsidiaire: tu soumets bien le job en tant que
> >>>>>>> y.bismuth@HPC.CRITEO.PREPROD et pas y.bismuth@CRITEOIS.LAN ?
> (c'est
> >>> la
> >>>>>>> meme question que: tu as fait kinit en plus subtil)
> >>>>>>> Tu as essayé avec un krb5.conf qui mette bien HPC.CRITEO.PREPROD en
> >>>>>>> default realm sur la gateway.
> >>>>>>>
> >>>>>>> JB
> >>>>>>>
> >>>>>>> 2015-05-28 23:05 GMT+02:00 Yohan Bismuth <yohan.bismuth1@gmail.com
> >:
> >>>>>>>
> >>>>>>>> Bah ça implique de modifier et recompiler slider. Je sais pas si
> on
> >>>>> veut
> >>>>>>>> ça...
> >>>>>>>>
> >>>>>>>> 2015-05-28 22:20 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
> >>>>>>>>
> >>>>>>>>> OK j'ai rien dit :)
> >>>>>>>>> Tu peux spécifier tout ça dans le thread, si vraiment ça
> fonctionne
> >>>>> sans
> >>>>>>>>> l'exception, je vois même pas pourquoi on s'embête en fait ???
> >>>>>>>>>
> >>>>>>>>> JB
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Jean-Baptiste Note
> >>>>>>>
> >>>>>
> >>>>>
> >>>
> >>>
> >>
>
>
Re: Keytab issue with Hbase secure
Posted by Jon Maron <jm...@hortonworks.com>.
Please describe the full procedure you used to retest.
> On May 29, 2015, at 9:28 AM, Yohan Bismuth <yo...@gmail.com> wrote:
>
> ...but i got the same problem using this method instead
>
> On Fri, May 29, 2015 at 3:25 PM, Yohan Bismuth <yo...@gmail.com>
> wrote:
>
>> oops my bad, was not looking the good thing
>>
>> On Fri, May 29, 2015 at 3:13 PM, Jon Maron <jm...@hortonworks.com> wrote:
>>
>>> It’s declared as public static:
>>>
>>> public synchronized static boolean isLoginKeytabBased()
>>>
>>> at least the version I’m looking at
>>>
>>>> On May 29, 2015, at 8:52 AM, Yohan Bismuth <yo...@gmail.com>
>>> wrote:
>>>>
>>>> Mmh I can't compile this. I don't think you can use isLoginKeytabBased()
>>>> since isKeytab is private in ugi.
>>>>
>>>> On Fri, May 29, 2015 at 1:56 PM, Jon Maron <jm...@hortonworks.com>
>>> wrote:
>>>>
>>>>> Looks like you’ve found a bug: validateLoginUser should be calling
>>>>> isLoginKeytabBased(), not isFromKeytab(). Would mind filing a JIRA?
>>>>>
>>>>> — Jon
>>>>>
>>>>>> On May 29, 2015, at 5:24 AM, Yohan Bismuth <yo...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Btw, i've tried using java7 and java8: smae issue.
>>>>>> I'm correctly logged as the principal of my keytab, and i can submit
>>> jobs
>>>>>> (like a wordcount) using this keytab.
>>>>>>
>>>>>> If i remove this line from the code:
>>>>>>
>>>>>>> validateLoginUser(UserGroupInformation.getLoginUser());
>>>>>>
>>>>>> everything seems to work fine.
>>>>>>
>>>>>> hasKerberosCredentials returns true, so the login must be based on a
>>>>>>> kerberos ticket. Perhaps it has expired?
>>>>>>
>>>>>> Well, this is the point here, i don't want the login to be based on a
>>>>>> kerberos ticket (because it would mean the ugi has the wrong flag set
>>> to
>>>>>> true). I want the login to be based on a kerberos key.
>>>>>>
>>>>>>
>>>>>> 2015-05-29 8:49 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
>>>>>>
>>>>>>> C'est pas très grave pour nous c'est packagé à la cradoc pour
>>> l'instant.
>>>>>>> C'est peut-etre aussi le fix ! Faudrait faire un git blame pour
>>>>> comprendre
>>>>>>> pourquoi l'exception est là. Elle trigge peut-etre aussi dans un cas
>>>>>>> légitime ?
>>>>>>>
>>>>>>> Question subsidiaire: tu soumets bien le job en tant que
>>>>>>> y.bismuth@HPC.CRITEO.PREPROD et pas y.bismuth@CRITEOIS.LAN ? (c'est
>>> la
>>>>>>> meme question que: tu as fait kinit en plus subtil)
>>>>>>> Tu as essayé avec un krb5.conf qui mette bien HPC.CRITEO.PREPROD en
>>>>>>> default realm sur la gateway.
>>>>>>>
>>>>>>> JB
>>>>>>>
>>>>>>> 2015-05-28 23:05 GMT+02:00 Yohan Bismuth <yo...@gmail.com>:
>>>>>>>
>>>>>>>> Bah ça implique de modifier et recompiler slider. Je sais pas si on
>>>>> veut
>>>>>>>> ça...
>>>>>>>>
>>>>>>>> 2015-05-28 22:20 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
>>>>>>>>
>>>>>>>>> OK j'ai rien dit :)
>>>>>>>>> Tu peux spécifier tout ça dans le thread, si vraiment ça fonctionne
>>>>> sans
>>>>>>>>> l'exception, je vois même pas pourquoi on s'embête en fait ???
>>>>>>>>>
>>>>>>>>> JB
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Jean-Baptiste Note
>>>>>>>
>>>>>
>>>>>
>>>
>>>
>>
Re: Keytab issue with Hbase secure
Posted by Yohan Bismuth <yo...@gmail.com>.
...but i got the same problem using this method instead
On Fri, May 29, 2015 at 3:25 PM, Yohan Bismuth <yo...@gmail.com>
wrote:
> oops my bad, was not looking the good thing
>
> On Fri, May 29, 2015 at 3:13 PM, Jon Maron <jm...@hortonworks.com> wrote:
>
>> It’s declared as public static:
>>
>> public synchronized static boolean isLoginKeytabBased()
>>
>> at least the version I’m looking at
>>
>> > On May 29, 2015, at 8:52 AM, Yohan Bismuth <yo...@gmail.com>
>> wrote:
>> >
>> > Mmh I can't compile this. I don't think you can use isLoginKeytabBased()
>> > since isKeytab is private in ugi.
>> >
>> > On Fri, May 29, 2015 at 1:56 PM, Jon Maron <jm...@hortonworks.com>
>> wrote:
>> >
>> >> Looks like you’ve found a bug: validateLoginUser should be calling
>> >> isLoginKeytabBased(), not isFromKeytab(). Would mind filing a JIRA?
>> >>
>> >> — Jon
>> >>
>> >>> On May 29, 2015, at 5:24 AM, Yohan Bismuth <yo...@gmail.com>
>> >> wrote:
>> >>>
>> >>> Btw, i've tried using java7 and java8: smae issue.
>> >>> I'm correctly logged as the principal of my keytab, and i can submit
>> jobs
>> >>> (like a wordcount) using this keytab.
>> >>>
>> >>> If i remove this line from the code:
>> >>>
>> >>>> validateLoginUser(UserGroupInformation.getLoginUser());
>> >>>
>> >>> everything seems to work fine.
>> >>>
>> >>> hasKerberosCredentials returns true, so the login must be based on a
>> >>>> kerberos ticket. Perhaps it has expired?
>> >>>
>> >>> Well, this is the point here, i don't want the login to be based on a
>> >>> kerberos ticket (because it would mean the ugi has the wrong flag set
>> to
>> >>> true). I want the login to be based on a kerberos key.
>> >>>
>> >>>
>> >>> 2015-05-29 8:49 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
>> >>>
>> >>>> C'est pas très grave pour nous c'est packagé à la cradoc pour
>> l'instant.
>> >>>> C'est peut-etre aussi le fix ! Faudrait faire un git blame pour
>> >> comprendre
>> >>>> pourquoi l'exception est là. Elle trigge peut-etre aussi dans un cas
>> >>>> légitime ?
>> >>>>
>> >>>> Question subsidiaire: tu soumets bien le job en tant que
>> >>>> y.bismuth@HPC.CRITEO.PREPROD et pas y.bismuth@CRITEOIS.LAN ? (c'est
>> la
>> >>>> meme question que: tu as fait kinit en plus subtil)
>> >>>> Tu as essayé avec un krb5.conf qui mette bien HPC.CRITEO.PREPROD en
>> >>>> default realm sur la gateway.
>> >>>>
>> >>>> JB
>> >>>>
>> >>>> 2015-05-28 23:05 GMT+02:00 Yohan Bismuth <yo...@gmail.com>:
>> >>>>
>> >>>>> Bah ça implique de modifier et recompiler slider. Je sais pas si on
>> >> veut
>> >>>>> ça...
>> >>>>>
>> >>>>> 2015-05-28 22:20 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
>> >>>>>
>> >>>>>> OK j'ai rien dit :)
>> >>>>>> Tu peux spécifier tout ça dans le thread, si vraiment ça fonctionne
>> >> sans
>> >>>>>> l'exception, je vois même pas pourquoi on s'embête en fait ???
>> >>>>>>
>> >>>>>> JB
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Jean-Baptiste Note
>> >>>>
>> >>
>> >>
>>
>>
>
Re: Keytab issue with Hbase secure
Posted by Yohan Bismuth <yo...@gmail.com>.
oops my bad, was not looking the good thing
On Fri, May 29, 2015 at 3:13 PM, Jon Maron <jm...@hortonworks.com> wrote:
> It’s declared as public static:
>
> public synchronized static boolean isLoginKeytabBased()
>
> at least the version I’m looking at
>
> > On May 29, 2015, at 8:52 AM, Yohan Bismuth <yo...@gmail.com>
> wrote:
> >
> > Mmh I can't compile this. I don't think you can use isLoginKeytabBased()
> > since isKeytab is private in ugi.
> >
> > On Fri, May 29, 2015 at 1:56 PM, Jon Maron <jm...@hortonworks.com>
> wrote:
> >
> >> Looks like you’ve found a bug: validateLoginUser should be calling
> >> isLoginKeytabBased(), not isFromKeytab(). Would mind filing a JIRA?
> >>
> >> — Jon
> >>
> >>> On May 29, 2015, at 5:24 AM, Yohan Bismuth <yo...@gmail.com>
> >> wrote:
> >>>
> >>> Btw, i've tried using java7 and java8: smae issue.
> >>> I'm correctly logged as the principal of my keytab, and i can submit
> jobs
> >>> (like a wordcount) using this keytab.
> >>>
> >>> If i remove this line from the code:
> >>>
> >>>> validateLoginUser(UserGroupInformation.getLoginUser());
> >>>
> >>> everything seems to work fine.
> >>>
> >>> hasKerberosCredentials returns true, so the login must be based on a
> >>>> kerberos ticket. Perhaps it has expired?
> >>>
> >>> Well, this is the point here, i don't want the login to be based on a
> >>> kerberos ticket (because it would mean the ugi has the wrong flag set
> to
> >>> true). I want the login to be based on a kerberos key.
> >>>
> >>>
> >>> 2015-05-29 8:49 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
> >>>
> >>>> C'est pas très grave pour nous c'est packagé à la cradoc pour
> l'instant.
> >>>> C'est peut-etre aussi le fix ! Faudrait faire un git blame pour
> >> comprendre
> >>>> pourquoi l'exception est là. Elle trigge peut-etre aussi dans un cas
> >>>> légitime ?
> >>>>
> >>>> Question subsidiaire: tu soumets bien le job en tant que
> >>>> y.bismuth@HPC.CRITEO.PREPROD et pas y.bismuth@CRITEOIS.LAN ? (c'est
> la
> >>>> meme question que: tu as fait kinit en plus subtil)
> >>>> Tu as essayé avec un krb5.conf qui mette bien HPC.CRITEO.PREPROD en
> >>>> default realm sur la gateway.
> >>>>
> >>>> JB
> >>>>
> >>>> 2015-05-28 23:05 GMT+02:00 Yohan Bismuth <yo...@gmail.com>:
> >>>>
> >>>>> Bah ça implique de modifier et recompiler slider. Je sais pas si on
> >> veut
> >>>>> ça...
> >>>>>
> >>>>> 2015-05-28 22:20 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
> >>>>>
> >>>>>> OK j'ai rien dit :)
> >>>>>> Tu peux spécifier tout ça dans le thread, si vraiment ça fonctionne
> >> sans
> >>>>>> l'exception, je vois même pas pourquoi on s'embête en fait ???
> >>>>>>
> >>>>>> JB
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Jean-Baptiste Note
> >>>>
> >>
> >>
>
>
Re: Keytab issue with Hbase secure
Posted by Jon Maron <jm...@hortonworks.com>.
It’s declared as public static:
public synchronized static boolean isLoginKeytabBased()
at least the version I’m looking at
> On May 29, 2015, at 8:52 AM, Yohan Bismuth <yo...@gmail.com> wrote:
>
> Mmh I can't compile this. I don't think you can use isLoginKeytabBased()
> since isKeytab is private in ugi.
>
> On Fri, May 29, 2015 at 1:56 PM, Jon Maron <jm...@hortonworks.com> wrote:
>
>> Looks like you’ve found a bug: validateLoginUser should be calling
>> isLoginKeytabBased(), not isFromKeytab(). Would mind filing a JIRA?
>>
>> — Jon
>>
>>> On May 29, 2015, at 5:24 AM, Yohan Bismuth <yo...@gmail.com>
>> wrote:
>>>
>>> Btw, i've tried using java7 and java8: smae issue.
>>> I'm correctly logged as the principal of my keytab, and i can submit jobs
>>> (like a wordcount) using this keytab.
>>>
>>> If i remove this line from the code:
>>>
>>>> validateLoginUser(UserGroupInformation.getLoginUser());
>>>
>>> everything seems to work fine.
>>>
>>> hasKerberosCredentials returns true, so the login must be based on a
>>>> kerberos ticket. Perhaps it has expired?
>>>
>>> Well, this is the point here, i don't want the login to be based on a
>>> kerberos ticket (because it would mean the ugi has the wrong flag set to
>>> true). I want the login to be based on a kerberos key.
>>>
>>>
>>> 2015-05-29 8:49 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
>>>
>>>> C'est pas très grave pour nous c'est packagé à la cradoc pour l'instant.
>>>> C'est peut-etre aussi le fix ! Faudrait faire un git blame pour
>> comprendre
>>>> pourquoi l'exception est là. Elle trigge peut-etre aussi dans un cas
>>>> légitime ?
>>>>
>>>> Question subsidiaire: tu soumets bien le job en tant que
>>>> y.bismuth@HPC.CRITEO.PREPROD et pas y.bismuth@CRITEOIS.LAN ? (c'est la
>>>> meme question que: tu as fait kinit en plus subtil)
>>>> Tu as essayé avec un krb5.conf qui mette bien HPC.CRITEO.PREPROD en
>>>> default realm sur la gateway.
>>>>
>>>> JB
>>>>
>>>> 2015-05-28 23:05 GMT+02:00 Yohan Bismuth <yo...@gmail.com>:
>>>>
>>>>> Bah ça implique de modifier et recompiler slider. Je sais pas si on
>> veut
>>>>> ça...
>>>>>
>>>>> 2015-05-28 22:20 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
>>>>>
>>>>>> OK j'ai rien dit :)
>>>>>> Tu peux spécifier tout ça dans le thread, si vraiment ça fonctionne
>> sans
>>>>>> l'exception, je vois même pas pourquoi on s'embête en fait ???
>>>>>>
>>>>>> JB
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Jean-Baptiste Note
>>>>
>>
>>
Re: Keytab issue with Hbase secure
Posted by Yohan Bismuth <yo...@gmail.com>.
Mmh I can't compile this. I don't think you can use isLoginKeytabBased()
since isKeytab is private in ugi.
On Fri, May 29, 2015 at 1:56 PM, Jon Maron <jm...@hortonworks.com> wrote:
> Looks like you’ve found a bug: validateLoginUser should be calling
> isLoginKeytabBased(), not isFromKeytab(). Would mind filing a JIRA?
>
> — Jon
>
> > On May 29, 2015, at 5:24 AM, Yohan Bismuth <yo...@gmail.com>
> wrote:
> >
> > Btw, i've tried using java7 and java8: smae issue.
> > I'm correctly logged as the principal of my keytab, and i can submit jobs
> > (like a wordcount) using this keytab.
> >
> > If i remove this line from the code:
> >
> >> validateLoginUser(UserGroupInformation.getLoginUser());
> >
> > everything seems to work fine.
> >
> > hasKerberosCredentials returns true, so the login must be based on a
> >> kerberos ticket. Perhaps it has expired?
> >
> > Well, this is the point here, i don't want the login to be based on a
> > kerberos ticket (because it would mean the ugi has the wrong flag set to
> > true). I want the login to be based on a kerberos key.
> >
> >
> > 2015-05-29 8:49 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
> >
> >> C'est pas très grave pour nous c'est packagé à la cradoc pour l'instant.
> >> C'est peut-etre aussi le fix ! Faudrait faire un git blame pour
> comprendre
> >> pourquoi l'exception est là. Elle trigge peut-etre aussi dans un cas
> >> légitime ?
> >>
> >> Question subsidiaire: tu soumets bien le job en tant que
> >> y.bismuth@HPC.CRITEO.PREPROD et pas y.bismuth@CRITEOIS.LAN ? (c'est la
> >> meme question que: tu as fait kinit en plus subtil)
> >> Tu as essayé avec un krb5.conf qui mette bien HPC.CRITEO.PREPROD en
> >> default realm sur la gateway.
> >>
> >> JB
> >>
> >> 2015-05-28 23:05 GMT+02:00 Yohan Bismuth <yo...@gmail.com>:
> >>
> >>> Bah ça implique de modifier et recompiler slider. Je sais pas si on
> veut
> >>> ça...
> >>>
> >>> 2015-05-28 22:20 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
> >>>
> >>>> OK j'ai rien dit :)
> >>>> Tu peux spécifier tout ça dans le thread, si vraiment ça fonctionne
> sans
> >>>> l'exception, je vois même pas pourquoi on s'embête en fait ???
> >>>>
> >>>> JB
> >>>>
> >>>
> >>>
> >>
> >>
> >> --
> >> Jean-Baptiste Note
> >>
>
>
Re: Keytab issue with Hbase secure
Posted by Jon Maron <jm...@hortonworks.com>.
Looks like you’ve found a bug: validateLoginUser should be calling isLoginKeytabBased(), not isFromKeytab(). Would mind filing a JIRA?
— Jon
> On May 29, 2015, at 5:24 AM, Yohan Bismuth <yo...@gmail.com> wrote:
>
> Btw, i've tried using java7 and java8: smae issue.
> I'm correctly logged as the principal of my keytab, and i can submit jobs
> (like a wordcount) using this keytab.
>
> If i remove this line from the code:
>
>> validateLoginUser(UserGroupInformation.getLoginUser());
>
> everything seems to work fine.
>
> hasKerberosCredentials returns true, so the login must be based on a
>> kerberos ticket. Perhaps it has expired?
>
> Well, this is the point here, i don't want the login to be based on a
> kerberos ticket (because it would mean the ugi has the wrong flag set to
> true). I want the login to be based on a kerberos key.
>
>
> 2015-05-29 8:49 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
>
>> C'est pas très grave pour nous c'est packagé à la cradoc pour l'instant.
>> C'est peut-etre aussi le fix ! Faudrait faire un git blame pour comprendre
>> pourquoi l'exception est là. Elle trigge peut-etre aussi dans un cas
>> légitime ?
>>
>> Question subsidiaire: tu soumets bien le job en tant que
>> y.bismuth@HPC.CRITEO.PREPROD et pas y.bismuth@CRITEOIS.LAN ? (c'est la
>> meme question que: tu as fait kinit en plus subtil)
>> Tu as essayé avec un krb5.conf qui mette bien HPC.CRITEO.PREPROD en
>> default realm sur la gateway.
>>
>> JB
>>
>> 2015-05-28 23:05 GMT+02:00 Yohan Bismuth <yo...@gmail.com>:
>>
>>> Bah ça implique de modifier et recompiler slider. Je sais pas si on veut
>>> ça...
>>>
>>> 2015-05-28 22:20 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
>>>
>>>> OK j'ai rien dit :)
>>>> Tu peux spécifier tout ça dans le thread, si vraiment ça fonctionne sans
>>>> l'exception, je vois même pas pourquoi on s'embête en fait ???
>>>>
>>>> JB
>>>>
>>>
>>>
>>
>>
>> --
>> Jean-Baptiste Note
>>
Re: Keytab issue with Hbase secure
Posted by Yohan Bismuth <yo...@gmail.com>.
Btw, i've tried using java7 and java8: smae issue.
I'm correctly logged as the principal of my keytab, and i can submit jobs
(like a wordcount) using this keytab.
If i remove this line from the code:
> validateLoginUser(UserGroupInformation.getLoginUser());
everything seems to work fine.
hasKerberosCredentials returns true, so the login must be based on a
> kerberos ticket. Perhaps it has expired?
Well, this is the point here, i don't want the login to be based on a
kerberos ticket (because it would mean the ugi has the wrong flag set to
true). I want the login to be based on a kerberos key.
2015-05-29 8:49 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
> C'est pas très grave pour nous c'est packagé à la cradoc pour l'instant.
> C'est peut-etre aussi le fix ! Faudrait faire un git blame pour comprendre
> pourquoi l'exception est là. Elle trigge peut-etre aussi dans un cas
> légitime ?
>
> Question subsidiaire: tu soumets bien le job en tant que
> y.bismuth@HPC.CRITEO.PREPROD et pas y.bismuth@CRITEOIS.LAN ? (c'est la
> meme question que: tu as fait kinit en plus subtil)
> Tu as essayé avec un krb5.conf qui mette bien HPC.CRITEO.PREPROD en
> default realm sur la gateway.
>
> JB
>
> 2015-05-28 23:05 GMT+02:00 Yohan Bismuth <yo...@gmail.com>:
>
>> Bah ça implique de modifier et recompiler slider. Je sais pas si on veut
>> ça...
>>
>> 2015-05-28 22:20 GMT+02:00 Jean-Baptiste Note <jb...@gmail.com>:
>>
>>> OK j'ai rien dit :)
>>> Tu peux spécifier tout ça dans le thread, si vraiment ça fonctionne sans
>>> l'exception, je vois même pas pourquoi on s'embête en fait ???
>>>
>>> JB
>>>
>>
>>
>
>
> --
> Jean-Baptiste Note
>
Re: Keytab issue with Hbase secure
Posted by Yohan Bismuth <yo...@gmail.com>.
>
> I'm using slider 0.80, this code is from here:
https://github.com/apache/incubator-slider/blob/releases/slider-0.80.0-incubating/slider-core/src/main/java/org/apache/slider/server/appmaster/SliderAppMaster.java
Re: Keytab issue with Hbase secure
Posted by Jon Maron <jm...@hortonworks.com>.
hasKerberosCredentials returns true, so the login must be based on a kerberos ticket. Perhaps it has expired? I guess you could a kdestroy followed by kinit…
Which slider version are you on? Is that error message at the bottom something you put in the code? I can’t find it in the codebase.
> On May 28, 2015, at 1:18 PM, Yohan Bismuth <yo...@gmail.com> wrote:
>
> Yes i did
> Le 28 mai 2015 19:15, "Jon Maron" <jm...@hortonworks.com> a écrit :
>
>> Did you actually log in (kinit) prior to invoking the slider client?
>> You’ll need to do that in order to establish an identity for the AM launch.
>>
>>> On May 28, 2015, at 12:59 PM, Yohan Bismuth <yo...@gmail.com>
>> wrote:
>>>
>>> Hi,
>>> i'm facing an issue with Hbase in secure mode.
>>> I followed the steps described on
>>> http://slider.incubator.apache.org/docs/security.html
>>>
>>> i created my headless keytab (and the associated principals), which i
>>> deployed on hdfs and when i start an hbase application, the keytab is
>>> correctly packaged in the SliderAppMaster container under the keytabs
>>> folder, but here is the problem:
>>>
>>> 2015-05-28 16:03:07,037 [main] INFO appmaster.SliderAppMaster -
>> Connecting
>>>> to RM at 1024,address tracking URL=
>>>> http://a4-5d-36-fd-a1-7c.hpc.criteo.preprod:1025
>>>> 2015-05-28 16:03:07,065 [main] INFO appmaster.SliderAppMaster - Slider
>> AM
>>>> Security Mode: KEYTAB
>>>> 2015-05-28 16:03:07,065 [main] INFO appmaster.SliderAppMaster - Token
>>>> HDFS_DELEGATION_TOKEN
>>>> 2015-05-28 16:03:07,065 [main] INFO appmaster.SliderAppMaster - Token
>>>> YARN_AM_RM_TOKEN
>>>> 2015-05-28 16:03:07,093 [main] INFO security.SecurityConfiguration - No
>>>> host keytab file path specified. Will attempt to retrieve keytab file
>>>> y.bismuth.keytab as a local resource for the container
>>>> 2015-05-28 16:03:07,104 [main] INFO security.UserGroupInformation -
>> Login
>>>> successful for user y.bismuth using keytab file
>>>>
>> /hdfs/wwn/600508b1001c246eb94fcc5ff4d68b4e/yarn/data/usercache/y.bismuth/appcache/application_1432038882976_2039/container_e11_1432038882976_2039_01_000001/keytabs/y.bismuth.keytab
>>>> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
>> security
>>>> enabled = true
>>>>
>>>
>>>
>>>> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
>>>> <<<<<<SOME DEBUG
>>>> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster - UGI =
>>>> y.bismuth@HPC.CRITEO.PREPROD (auth:KERBEROS)
>>>> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
>> isKeytab
>>>> = false
>>>> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster - tokens
>> =
>>>> []
>>>> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
>>>> hasKerberosCredentials = true
>>>> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
>>>> credentials = org.apache.hadoop.security.Credentials@1cf2fed4
>>>> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
>>>> authentication method = KERBEROS
>>>> 2015-05-28 16:03:07,111 [main] INFO appmaster.SliderAppMaster - config
>> =
>>>> Configuration: core-default.xml, core-site.xml, yarn-default.xml,
>>>> yarn-site.xml, hdfs-default.xml, hdfs-site.xml,
>>>> org/apache/slider/slider.xml, mapred-default.xml, mapred-site.xml
>>>> 2015-05-28 16:03:07,111 [main] INFO appmaster.SliderAppMaster - SOME
>>>> DEBUG>>>>>
>>>>
>>>
>>>
>>>> 2015-05-28 16:03:07,112 [main] ERROR main.ServiceLauncher - User is not
>>>> based on a keytab in a secure deployment.
>>>
>>>
>>> So as far as i can see, i'm logging in successfully using the keytab
>>> packaged in the container, but the flag isKeytab, which should be set to
>>> true in my UGI (i hope), is not, and i can't figure out why. Because of
>>> that, my SliderAppMaster crash.
>>>
>>> Any idea ?
>>
>>
Re: Keytab issue with Hbase secure
Posted by Yohan Bismuth <yo...@gmail.com>.
Yes i did
Le 28 mai 2015 19:15, "Jon Maron" <jm...@hortonworks.com> a écrit :
> Did you actually log in (kinit) prior to invoking the slider client?
> You’ll need to do that in order to establish an identity for the AM launch.
>
> > On May 28, 2015, at 12:59 PM, Yohan Bismuth <yo...@gmail.com>
> wrote:
> >
> > Hi,
> > i'm facing an issue with Hbase in secure mode.
> > I followed the steps described on
> > http://slider.incubator.apache.org/docs/security.html
> >
> > i created my headless keytab (and the associated principals), which i
> > deployed on hdfs and when i start an hbase application, the keytab is
> > correctly packaged in the SliderAppMaster container under the keytabs
> > folder, but here is the problem:
> >
> > 2015-05-28 16:03:07,037 [main] INFO appmaster.SliderAppMaster -
> Connecting
> >> to RM at 1024,address tracking URL=
> >> http://a4-5d-36-fd-a1-7c.hpc.criteo.preprod:1025
> >> 2015-05-28 16:03:07,065 [main] INFO appmaster.SliderAppMaster - Slider
> AM
> >> Security Mode: KEYTAB
> >> 2015-05-28 16:03:07,065 [main] INFO appmaster.SliderAppMaster - Token
> >> HDFS_DELEGATION_TOKEN
> >> 2015-05-28 16:03:07,065 [main] INFO appmaster.SliderAppMaster - Token
> >> YARN_AM_RM_TOKEN
> >> 2015-05-28 16:03:07,093 [main] INFO security.SecurityConfiguration - No
> >> host keytab file path specified. Will attempt to retrieve keytab file
> >> y.bismuth.keytab as a local resource for the container
> >> 2015-05-28 16:03:07,104 [main] INFO security.UserGroupInformation -
> Login
> >> successful for user y.bismuth using keytab file
> >>
> /hdfs/wwn/600508b1001c246eb94fcc5ff4d68b4e/yarn/data/usercache/y.bismuth/appcache/application_1432038882976_2039/container_e11_1432038882976_2039_01_000001/keytabs/y.bismuth.keytab
> >> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
> security
> >> enabled = true
> >>
> >
> >
> >> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
> >> <<<<<<SOME DEBUG
> >> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster - UGI =
> >> y.bismuth@HPC.CRITEO.PREPROD (auth:KERBEROS)
> >> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
> isKeytab
> >> = false
> >> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster - tokens
> =
> >> []
> >> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
> >> hasKerberosCredentials = true
> >> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
> >> credentials = org.apache.hadoop.security.Credentials@1cf2fed4
> >> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
> >> authentication method = KERBEROS
> >> 2015-05-28 16:03:07,111 [main] INFO appmaster.SliderAppMaster - config
> =
> >> Configuration: core-default.xml, core-site.xml, yarn-default.xml,
> >> yarn-site.xml, hdfs-default.xml, hdfs-site.xml,
> >> org/apache/slider/slider.xml, mapred-default.xml, mapred-site.xml
> >> 2015-05-28 16:03:07,111 [main] INFO appmaster.SliderAppMaster - SOME
> >> DEBUG>>>>>
> >>
> >
> >
> >> 2015-05-28 16:03:07,112 [main] ERROR main.ServiceLauncher - User is not
> >> based on a keytab in a secure deployment.
> >
> >
> > So as far as i can see, i'm logging in successfully using the keytab
> > packaged in the container, but the flag isKeytab, which should be set to
> > true in my UGI (i hope), is not, and i can't figure out why. Because of
> > that, my SliderAppMaster crash.
> >
> > Any idea ?
>
>
Re: Keytab issue with Hbase secure
Posted by Jon Maron <jm...@hortonworks.com>.
Did you actually log in (kinit) prior to invoking the slider client? You’ll need to do that in order to establish an identity for the AM launch.
> On May 28, 2015, at 12:59 PM, Yohan Bismuth <yo...@gmail.com> wrote:
>
> Hi,
> i'm facing an issue with Hbase in secure mode.
> I followed the steps described on
> http://slider.incubator.apache.org/docs/security.html
>
> i created my headless keytab (and the associated principals), which i
> deployed on hdfs and when i start an hbase application, the keytab is
> correctly packaged in the SliderAppMaster container under the keytabs
> folder, but here is the problem:
>
> 2015-05-28 16:03:07,037 [main] INFO appmaster.SliderAppMaster - Connecting
>> to RM at 1024,address tracking URL=
>> http://a4-5d-36-fd-a1-7c.hpc.criteo.preprod:1025
>> 2015-05-28 16:03:07,065 [main] INFO appmaster.SliderAppMaster - Slider AM
>> Security Mode: KEYTAB
>> 2015-05-28 16:03:07,065 [main] INFO appmaster.SliderAppMaster - Token
>> HDFS_DELEGATION_TOKEN
>> 2015-05-28 16:03:07,065 [main] INFO appmaster.SliderAppMaster - Token
>> YARN_AM_RM_TOKEN
>> 2015-05-28 16:03:07,093 [main] INFO security.SecurityConfiguration - No
>> host keytab file path specified. Will attempt to retrieve keytab file
>> y.bismuth.keytab as a local resource for the container
>> 2015-05-28 16:03:07,104 [main] INFO security.UserGroupInformation - Login
>> successful for user y.bismuth using keytab file
>> /hdfs/wwn/600508b1001c246eb94fcc5ff4d68b4e/yarn/data/usercache/y.bismuth/appcache/application_1432038882976_2039/container_e11_1432038882976_2039_01_000001/keytabs/y.bismuth.keytab
>> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster - security
>> enabled = true
>>
>
>
>> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
>> <<<<<<SOME DEBUG
>> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster - UGI =
>> y.bismuth@HPC.CRITEO.PREPROD (auth:KERBEROS)
>> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster - isKeytab
>> = false
>> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster - tokens =
>> []
>> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
>> hasKerberosCredentials = true
>> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
>> credentials = org.apache.hadoop.security.Credentials@1cf2fed4
>> 2015-05-28 16:03:07,104 [main] INFO appmaster.SliderAppMaster -
>> authentication method = KERBEROS
>> 2015-05-28 16:03:07,111 [main] INFO appmaster.SliderAppMaster - config =
>> Configuration: core-default.xml, core-site.xml, yarn-default.xml,
>> yarn-site.xml, hdfs-default.xml, hdfs-site.xml,
>> org/apache/slider/slider.xml, mapred-default.xml, mapred-site.xml
>> 2015-05-28 16:03:07,111 [main] INFO appmaster.SliderAppMaster - SOME
>> DEBUG>>>>>
>>
>
>
>> 2015-05-28 16:03:07,112 [main] ERROR main.ServiceLauncher - User is not
>> based on a keytab in a secure deployment.
>
>
> So as far as i can see, i'm logging in successfully using the keytab
> packaged in the container, but the flag isKeytab, which should be set to
> true in my UGI (i hope), is not, and i can't figure out why. Because of
> that, my SliderAppMaster crash.
>
> Any idea ?