Kerberos token to java actions

[Aleksei Udatšnõi <al...@softonic.com>]

Hi All,

I have noticed that after enabling Kerberos in our cluster, all Oozie java MR actions started failing with the delegation token error. Seems like it is a known issue documented in the Cookbook. As a solution, it is suggested to add a code snippet to all java MR jobs.
https://cwiki.apache.org/confluence/display/OOZIE/Java+Cookbook#JavaCookbook-AuthenticatingonaKerberosenabledcluster

At the same time, the delegation token is automatically propagated by Oozie to Hive, Pig, Sqoop and other actions.

The question is why can’t the token be propagated to Java actions by Oozie?

I have many custom Java MR jobs and also third party jobs and it would be pretty hard to patch all of them with this snippet.

Thank you
Aleksei

RE: Kerberos token to java actions

[Aleksei Udatšnõi <al...@softonic.com>]

FYI:
I have found a workaround for this issue. If you can't modify Java code of map-reduce jars, but need to execute them in a secure cluster, you can wrap these MR jobs into Oozie shell action and call it like:

hadoop jar third-party.jar MyJob -D mapreduce.job.credentials.binary=$HADOOP_TOKEN_FILE_LOCATION

It is not a clean solution, but at least it works. 


-----Original Message-----
From: Robert Kanter [mailto:rkanter@cloudera.com] 
Sent: viernes, 08 de noviembre de 2013 18:42
To: user@oozie.apache.org
Subject: Re: Kerberos token to java actions

Hi,

An action essentially consists of two classes: The ActionExecutor and the Main.  The ActionExecutor runs in Oozie and handles launching the action, checking its status, etc.  The Main runs in the single map-only task “launcher job” and handles running the actual action.  The delegation propagation code you pointed to has to go into the Main class.  In the case of, for example, the Pig action, we have the PigActionExecutor and the PigMain.  The PigMain has the propagation code, which is why its handled for you.  However, for the JavaAction, we provide the JavaActionExecutor, but you provide the “JavaMain", so we can’t include the propagation code.
I hope that explains it.


- Robert


On Fri, Nov 8, 2013 at 4:46 AM, Aleksei Udatšnõi < aleksei.udatsnoi@softonic.com> wrote:

> Hi All,
>
> I have noticed that after enabling Kerberos in our cluster, all Oozie 
> java MR actions started failing with the delegation token error. Seems 
> like it is a known issue documented in the Cookbook. As a solution, it 
> is suggested to add a code snippet to all java MR jobs.
>
> https://cwiki.apache.org/confluence/display/OOZIE/Java+Cookbook#JavaCo
> okbook-AuthenticatingonaKerberosenabledcluster
>
> At the same time, the delegation token is automatically propagated by 
> Oozie to Hive, Pig, Sqoop and other actions.
>
> The question is why can’t the token be propagated to Java actions by Oozie?
>
> I have many custom Java MR jobs and also third party jobs and it would 
> be pretty hard to patch all of them with this snippet.
>
> Thank you
> Aleksei
>

Re: Kerberos token to java actions

[Robert Kanter <rk...@cloudera.com>]

Hi,

An action essentially consists of two classes: The ActionExecutor and the
Main.  The ActionExecutor runs in Oozie and handles launching the action,
checking its status, etc.  The Main runs in the single map-only task
“launcher job” and handles running the actual action.  The delegation
propagation code you pointed to has to go into the Main class.  In the case
of, for example, the Pig action, we have the PigActionExecutor and the
PigMain.  The PigMain has the propagation code, which is why its handled
for you.  However, for the JavaAction, we provide the JavaActionExecutor,
but you provide the “JavaMain", so we can’t include the propagation code.
I hope that explains it.


- Robert


On Fri, Nov 8, 2013 at 4:46 AM, Aleksei Udatšnõi <
aleksei.udatsnoi@softonic.com> wrote:

> Hi All,
>
> I have noticed that after enabling Kerberos in our cluster, all Oozie java
> MR actions started failing with the delegation token error. Seems like it
> is a known issue documented in the Cookbook. As a solution, it is suggested
> to add a code snippet to all java MR jobs.
>
> https://cwiki.apache.org/confluence/display/OOZIE/Java+Cookbook#JavaCookbook-AuthenticatingonaKerberosenabledcluster
>
> At the same time, the delegation token is automatically propagated by
> Oozie to Hive, Pig, Sqoop and other actions.
>
> The question is why can’t the token be propagated to Java actions by Oozie?
>
> I have many custom Java MR jobs and also third party jobs and it would be
> pretty hard to patch all of them with this snippet.
>
> Thank you
> Aleksei
>