You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Morgrim Murdargent <mo...@gmail.com> on 2016/06/09 16:03:25 UTC
About shell action and HADOOP_TOKEN_FILE_LOCATION
Hello !
I have a question related to a shell action in an oozie workflow in a
kerberized cluster.
I found something about this in the oozie doc for java action :
*###IMPORTANT:* In order for a Java action to succeed on a secure cluster,
it must propagate the
Hadoop delegation token like in the following code snippet (this is benign
on non-secure clusters):
// propagate delegation related props from launcher job to MR job
if (System.getenv("HADOOP_TOKEN_FILE_LOCATION") != null) {
jobConf.set("mapreduce.job.credentials.binary",
System.getenv("HADOOP_TOKEN_FILE_LOCATION"));
}
###
https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html
But is it possible to use the HADOOP_TOKEN_FILE_LOCATION in a shell action
? In order to not do a kinit with a keytab ?
BR.
Morgrim.
Re: About shell action and HADOOP_TOKEN_FILE_LOCATION
Posted by Laurent H <la...@gmail.com>.
Oozie has already set the HADOOP_TOKEN_FILE_LOCATION variable when you
execute an action (you're right, it's an environment variable). If you use
some Hive stuff or something like that which use Kerberos - into a shell
action - it can be useful to launch your query like that :
*hive -hiveconf hadoopTokenKerberos=$HADOOP_TOKEN_FILE_LOCATION -f
yourFile.hql*
All other cases, you can set this variable directly into the action i think
(for hive, it's something like that cred=${hiveKerberosUri}
Regards,
--
Laurent HATIER - Consultant Big Data & Business Intelligence chez CapGemini
fr.linkedin.com/pub/laurent-hatier/25/36b/a86/
<http://fr.linkedin.com/pub/laurent-h/25/36b/a86/>
2016-06-09 19:40 GMT+02:00 Mohammad Islam <mi...@yahoo.com.invalid>:
> I think Oozie Shell action is already taken care of this. There should not
> be any issue.
> Please let us know if you find any.
> Regards,Mohammad
>
>
> On Thursday, June 9, 2016 10:32 AM, Morgrim Murdargent <
> morgrim.oozie@gmail.com> wrote:
>
>
> Hello Mohammad.
>
> Ty for your answer. There is no errors in fact. I'm just wondering this
> question.
>
> If I want to interact with hdfs or hbase or hive in a shell from an oozie
> shell action, how can I use the env variable HADOOP_TOKEN_FILE_LOCATION ?
> Is the usage of this env variable the solution ?
>
> BR.
>
> Morgrim
> Le 9 juin 2016 19:06, "Mohammad Islam" <mi...@yahoo.com.invalid> a
> écrit :
>
> > Do you need to kinit explicitly in your shell script to be use in Shell
> > Action?I think you don't need it. Any error message would help.
> >
> >
> > On Thursday, June 9, 2016 9:03 AM, Morgrim Murdargent <
> > morgrim.oozie@gmail.com> wrote:
> >
> >
> > Hello !
> >
> > I have a question related to a shell action in an oozie workflow in a
> > kerberized cluster.
> >
> > I found something about this in the oozie doc for java action :
> >
> > *###IMPORTANT:* In order for a Java action to succeed on a secure
> cluster,
> > it must propagate the
> > Hadoop delegation token like in the following code snippet (this is
> benign
> > on non-secure clusters):
> >
> > // propagate delegation related props from launcher job to MR job
> > if (System.getenv("HADOOP_TOKEN_FILE_LOCATION") != null) {
> > jobConf.set("mapreduce.job.credentials.binary",
> > System.getenv("HADOOP_TOKEN_FILE_LOCATION"));
> > }
> >
> > ###
> >
> > https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html
> >
> > But is it possible to use the HADOOP_TOKEN_FILE_LOCATION in a shell
> action
> > ? In order to not do a kinit with a keytab ?
> >
> > BR.
> >
> > Morgrim.
> >
> >
> >
>
>
>
Re: About shell action and HADOOP_TOKEN_FILE_LOCATION
Posted by Mohammad Islam <mi...@yahoo.com.INVALID>.
I think Oozie Shell action is already taken care of this. There should not be any issue.
Please let us know if you find any.
Regards,Mohammad
On Thursday, June 9, 2016 10:32 AM, Morgrim Murdargent <mo...@gmail.com> wrote:
Hello Mohammad.
Ty for your answer. There is no errors in fact. I'm just wondering this
question.
If I want to interact with hdfs or hbase or hive in a shell from an oozie
shell action, how can I use the env variable HADOOP_TOKEN_FILE_LOCATION ?
Is the usage of this env variable the solution ?
BR.
Morgrim
Le 9 juin 2016 19:06, "Mohammad Islam" <mi...@yahoo.com.invalid> a
écrit :
> Do you need to kinit explicitly in your shell script to be use in Shell
> Action?I think you don't need it. Any error message would help.
>
>
> On Thursday, June 9, 2016 9:03 AM, Morgrim Murdargent <
> morgrim.oozie@gmail.com> wrote:
>
>
> Hello !
>
> I have a question related to a shell action in an oozie workflow in a
> kerberized cluster.
>
> I found something about this in the oozie doc for java action :
>
> *###IMPORTANT:* In order for a Java action to succeed on a secure cluster,
> it must propagate the
> Hadoop delegation token like in the following code snippet (this is benign
> on non-secure clusters):
>
> // propagate delegation related props from launcher job to MR job
> if (System.getenv("HADOOP_TOKEN_FILE_LOCATION") != null) {
> jobConf.set("mapreduce.job.credentials.binary",
> System.getenv("HADOOP_TOKEN_FILE_LOCATION"));
> }
>
> ###
>
> https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html
>
> But is it possible to use the HADOOP_TOKEN_FILE_LOCATION in a shell action
> ? In order to not do a kinit with a keytab ?
>
> BR.
>
> Morgrim.
>
>
>
Re: About shell action and HADOOP_TOKEN_FILE_LOCATION
Posted by Morgrim Murdargent <mo...@gmail.com>.
Hello Mohammad.
Ty for your answer. There is no errors in fact. I'm just wondering this
question.
If I want to interact with hdfs or hbase or hive in a shell from an oozie
shell action, how can I use the env variable HADOOP_TOKEN_FILE_LOCATION ?
Is the usage of this env variable the solution ?
BR.
Morgrim
Le 9 juin 2016 19:06, "Mohammad Islam" <mi...@yahoo.com.invalid> a
écrit :
> Do you need to kinit explicitly in your shell script to be use in Shell
> Action?I think you don't need it. Any error message would help.
>
>
> On Thursday, June 9, 2016 9:03 AM, Morgrim Murdargent <
> morgrim.oozie@gmail.com> wrote:
>
>
> Hello !
>
> I have a question related to a shell action in an oozie workflow in a
> kerberized cluster.
>
> I found something about this in the oozie doc for java action :
>
> *###IMPORTANT:* In order for a Java action to succeed on a secure cluster,
> it must propagate the
> Hadoop delegation token like in the following code snippet (this is benign
> on non-secure clusters):
>
> // propagate delegation related props from launcher job to MR job
> if (System.getenv("HADOOP_TOKEN_FILE_LOCATION") != null) {
> jobConf.set("mapreduce.job.credentials.binary",
> System.getenv("HADOOP_TOKEN_FILE_LOCATION"));
> }
>
> ###
>
> https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html
>
> But is it possible to use the HADOOP_TOKEN_FILE_LOCATION in a shell action
> ? In order to not do a kinit with a keytab ?
>
> BR.
>
> Morgrim.
>
>
>
Re: About shell action and HADOOP_TOKEN_FILE_LOCATION
Posted by Mohammad Islam <mi...@yahoo.com.INVALID>.
Do you need to kinit explicitly in your shell script to be use in Shell Action?I think you don't need it. Any error message would help.
On Thursday, June 9, 2016 9:03 AM, Morgrim Murdargent <mo...@gmail.com> wrote:
Hello !
I have a question related to a shell action in an oozie workflow in a
kerberized cluster.
I found something about this in the oozie doc for java action :
*###IMPORTANT:* In order for a Java action to succeed on a secure cluster,
it must propagate the
Hadoop delegation token like in the following code snippet (this is benign
on non-secure clusters):
// propagate delegation related props from launcher job to MR job
if (System.getenv("HADOOP_TOKEN_FILE_LOCATION") != null) {
jobConf.set("mapreduce.job.credentials.binary",
System.getenv("HADOOP_TOKEN_FILE_LOCATION"));
}
###
https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html
But is it possible to use the HADOOP_TOKEN_FILE_LOCATION in a shell action
? In order to not do a kinit with a keytab ?
BR.
Morgrim.