You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@knox.apache.org by Christopher Jackson <ja...@gmail.com> on 2016/01/05 02:04:22 UTC
Re: Custom Ambari Service using Knox and Kerberos.
Hey guys,
Thanks for your feedback. Indeed you were correct the issue was that the guest user was not created on each node in the cluster. After doing so I was able to get webhdfs working through the gateway.
Before I continue discussing my issues with my custom service using knox and kerberos I would like to ask a more general question as I believe I may have gone about kerberos the wrong way. I know this is the knox user thread but I’m sure you guys are in general good with securing hadoop so I’ll ask, if I should redirect the question to another group please let me know.
Some brief background:
My web application interacts with HBase (using client and rest) and Oozie (using client) quite frequently and thus will need to be authenticated to kerberos to perform those actions. Since its a service that is always running I need a TGT pretty much indefinitely. How is this usually handled? Programmatically? Cron-job issuing kinit using keytab? Or is this done via proxy, using doAs java calls from the Hadoop Security classes?
My initial approach overlooked this issue, a TGT is generated at install time of the service, but after a given period of time it expires and the service starts throwing errors when it tries to access other services in the cluster.
I appreciated any best practices or common approaches anyone could share.
Regards,
Christopher Jackson
jackson.christopher.lee@gmail.com <ma...@gmail.com>
> On Dec 28, 2015, at 11:48 AM, Kevin Minder <ke...@hortonworks.com> wrote:
>
> In addition to Larry’s comments, this error:
> {"RemoteException":{"exception":"SecurityException","javaClassName":"java.lang.SecurityException","message":"Failed to obtain user group information: org.apache.hadoop.security.authorize.AuthorizationException: User: knox is not allowed to impersonate guest"}}
> Is usually caused by missing proxy user config:
> http://knox.apache.org/books/knox-0-6-0/user-guide.html#Grant+Proxy+privileges+for+Knox+user+in+`core-site.xml`+on+Hadoop+master+nodes <http://knox.apache.org/books/knox-0-6-0/user-guide.html#Grant+Proxy+privileges+for+Knox+user+in+%60core-site.xml%60+on+Hadoop+master+nodes>
>
> From: Christopher Jackson <jackson.christopher.lee@gmail.com <ma...@gmail.com>>
> Reply-To: "user@knox.apache.org <ma...@knox.apache.org>" <user@knox.apache.org <ma...@knox.apache.org>>
> Date: Thursday, December 24, 2015 at 1:24 PM
> To: "user@knox.apache.org <ma...@knox.apache.org>" <user@knox.apache.org <ma...@knox.apache.org>>
> Subject: Custom Ambari Service using Knox and Kerberos.
>
> {"RemoteException":{"exception":"SecurityException","javaClassName":"java.lang.SecurityException","message":"Failed to obtain user group information: org.apache.hadoop.security.authorize.AuthorizationException: User: knox is not allowed to impersonate guest"}}
>
Re: Custom Ambari Service using Knox and Kerberos.
Posted by larry mccay <lm...@apache.org>.
Hi Christopher -
I'm not sure that I have the proper context to advise you on your webapp
design and kerberos use.
I can tell you certain characteristics of what an app/service like yours
generally have in order to work properly with both kerberos clients and
consuming kerberized resources from within Hadoop.
* a service that is acting on behalf of end users generally implements the
trusted proxy pattern for interacting with the backend services. This means
that your service needs to authenticate to the other services via
kerberos/SPNEGO. Generally, such a service would login via its own keytab.
When calling REST APIs you will need to authenticate via SPNEGO and pass a
doAs=<endusername>
* if other services need to act on behalf of other users then your service
generally implements the server side of the trusted proxy pattern as well.
In which, clients will authenticate to your service via kerberos/SPNEGO and
provide a doAs parameter to represent the end user.
* keytab management is something that your ops folks should have some
provisions for and is not done in any hadoop prescribed manner. You
shouldn't need to refresh your keytabs that often. Generally, only when the
password of the user account changes.
You should be able to find various examples of these implementation
patterns across the hadoop ecosystem.
On the Knox lists, the expertise is going to be more around how to
configure Knox to work in a secure cluster for known Hadoop services and
services that behave in Hadoop compatible ways. It will also be able to
help you consume service APIs through Knox. Unfortunately, guidance on
service/app development itself is probably better directed to other
components that are more focused on the development model of Hadoop
applications themselves.
You may want to look into Slider for how to deploy a webapp to YARN. There
has been recent work to deploy tomcat servers on YARN through Slider. This
may be a good place to start to look at those details.
Hope this is helpful to you.
--larry
On Tue, Jan 5, 2016 at 4:01 AM, Rajesh Chandramohan <ra...@yahoo.com>
wrote:
> Hi ,
>
> If your Problem is just to have refreshed TGT at regular interval you can
> consider using k5start service. http://linux.die.net/man/1/k5start
> THis service daemon will keep your keytab refreshed in regular time.
>
> \Rajesh
>
>
> On Tuesday, 5 January 2016 6:34 AM, Christopher Jackson <
> jackson.christopher.lee@gmail.com> wrote:
>
>
> Hey guys,
>
> Thanks for your feedback. Indeed you were correct the issue was that the
> guest user was not created on each node in the cluster. After doing so I
> was able to get webhdfs working through the gateway.
>
> Before I continue discussing my issues with my custom service using knox
> and kerberos I would like to ask a more general question as I believe I may
> have gone about kerberos the wrong way. I know this is the knox user thread
> but I’m sure you guys are in general good with securing hadoop so I’ll ask,
> if I should redirect the question to another group please let me know.
>
> Some brief background:
>
> My web application interacts with HBase (using client and rest) and Oozie
> (using client) quite frequently and thus will need to be authenticated to
> kerberos to perform those actions. Since its a service that is always
> running I need a TGT pretty much indefinitely. How is this usually handled?
> Programmatically? Cron-job issuing kinit using keytab? Or is this done via
> proxy, using doAs java calls from the Hadoop Security classes?
>
> My initial approach overlooked this issue, a TGT is generated at install
> time of the service, but after a given period of time it expires and the
> service starts throwing errors when it tries to access other services in
> the cluster.
>
> I appreciated any best practices or common approaches anyone could share.
>
> Regards,
>
> Christopher Jackson
> jackson.christopher.lee@gmail.com
>
>
> On Dec 28, 2015, at 11:48 AM, Kevin Minder <ke...@hortonworks.com>
> wrote:
>
> In addition to Larry’s comments, this error:
>
> {"RemoteException":{"exception":"SecurityException","javaClassName":"java.lang.SecurityException","message":"Failed to obtain user group information: org.apache.hadoop.security.authorize.AuthorizationException: User: knox is not allowed to impersonate guest"}}
>
> Is usually caused by missing proxy user config:
>
> http://knox.apache.org/books/knox-0-6-0/user-guide.html#Grant+Proxy+privileges+for+Knox+user+in+`core-site.xml`+on+Hadoop+master+nodes
> <http://knox.apache.org/books/knox-0-6-0/user-guide.html#Grant+Proxy+privileges+for+Knox+user+in+%60core-site.xml%60+on+Hadoop+master+nodes>
>
> From: Christopher Jackson <ja...@gmail.com>
> Reply-To: "user@knox.apache.org" <us...@knox.apache.org>
> Date: Thursday, December 24, 2015 at 1:24 PM
> To: "user@knox.apache.org" <us...@knox.apache.org>
> Subject: Custom Ambari Service using Knox and Kerberos.
>
> {"RemoteException":{"exception":"SecurityException","javaClassName":"java.lang.SecurityException","message":"Failed to obtain user group information: org.apache.hadoop.security.authorize.AuthorizationException: User: knox is not allowed to impersonate guest"}}
>
>
>
>
>
>
Re: Custom Ambari Service using Knox and Kerberos.
Posted by Rajesh Chandramohan <ra...@yahoo.com>.
Hi ,
If your Problem is just to have refreshed TGT at regular interval you can consider using k5start service. http://linux.die.net/man/1/k5start THis service daemon will keep your keytab refreshed in regular time. \Rajesh
On Tuesday, 5 January 2016 6:34 AM, Christopher Jackson <ja...@gmail.com> wrote:
Hey guys,
Thanks for your feedback. Indeed you were correct the issue was that the guest user was not created on each node in the cluster. After doing so I was able to get webhdfs working through the gateway.
Before I continue discussing my issues with my custom service using knox and kerberos I would like to ask a more general question as I believe I may have gone about kerberos the wrong way. I know this is the knox user thread but I’m sure you guys are in general good with securing hadoop so I’ll ask, if I should redirect the question to another group please let me know.
Some brief background:
My web application interacts with HBase (using client and rest) and Oozie (using client) quite frequently and thus will need to be authenticated to kerberos to perform those actions. Since its a service that is always running I need a TGT pretty much indefinitely. How is this usually handled? Programmatically? Cron-job issuing kinit using keytab? Or is this done via proxy, using doAs java calls from the Hadoop Security classes?
My initial approach overlooked this issue, a TGT is generated at install time of the service, but after a given period of time it expires and the service starts throwing errors when it tries to access other services in the cluster.
I appreciated any best practices or common approaches anyone could share.
Regards,
Christopher Jacksonjackson.christopher.lee@gmail.com
On Dec 28, 2015, at 11:48 AM, Kevin Minder <ke...@hortonworks.com> wrote:
In addition to Larry’s comments, this error:{"RemoteException":{"exception":"SecurityException","javaClassName":"java.lang.SecurityException","message":"Failed to obtain user group information: org.apache.hadoop.security.authorize.AuthorizationException: User: knox is not allowed to impersonate guest"}}Is usually caused by missing proxy user config:http://knox.apache.org/books/knox-0-6-0/user-guide.html#Grant+Proxy+privileges+for+Knox+user+in+`core-site.xml`+on+Hadoop+master+nodes
From: Christopher Jackson <ja...@gmail.com>
Reply-To: "user@knox.apache.org" <us...@knox.apache.org>
Date: Thursday, December 24, 2015 at 1:24 PM
To: "user@knox.apache.org" <us...@knox.apache.org>
Subject: Custom Ambari Service using Knox and Kerberos.
{"RemoteException":{"exception":"SecurityException","javaClassName":"java.lang.SecurityException","message":"Failed to obtain user group information: org.apache.hadoop.security.authorize.AuthorizationException: User: knox is not allowed to impersonate guest"}}