You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Clément MATHIEU <cl...@unportant.info> on 2013/02/21 16:44:33 UTC
Submitting Hadoop jobs from a shell action leads to the lost of the
user identity
Hi all,
I'm currently troubleshooting an issue involving tasks ran under the
default mapred account rather the user account and would greatly
appreciate any help or suggestion.
The guilty action can be summarized like that:
- The workflow contains a shell action
- The shell action starts a Bash shell script
- The shell script executes several Pig scripts and starts some map
reduce tasks
The issue is that the Pig scripts and MR tasks are submitted to the
jobtracker and executed using the mapred Hadoop user rather the same
user that is used to run the shell action.
I believe that this issue can trivially be explained because Hadoop
relies on the Unix username of the process which invoked submit and, if
security is not enabled, Hadoop uses a shared mapred Unix user to
execute the tasks. How can I configure Oozie, or at least my action, to
avoid a such escape ?
PS: Using shell scripts to do Oozie's job can be seen as dumb, but the
explanation is quite rational ;)
Thanks,
-- Clément
Re: Submitting Hadoop jobs from a shell action leads to the lost of
the user identity
Posted by Clément MATHIEU <cl...@unportant.info>.
On Fri, 2013-02-22 at 08:25 -0800, Alejandro Abdelnur wrote:
Hi Alejandro,
> Clément, any luck?
Unfortunately I have not been able to find some free time to test your
proposal today. I will most likely not be able to do it before the end
of next week. I will report the outcome of the test on the mailing list
ASAP.
Regards,
-- Clément
Re: Submitting Hadoop jobs from a shell action leads to the lost of
the user identity
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Clément, any luck?
On Thu, Feb 21, 2013 at 9:35 AM, Alejandro Abdelnur <tu...@cloudera.com>wrote:
> Hi Clément,
>
> The HADOOP_USER_NAME trick will work only in a non-secure cluster where
> you can be whoever you want just by setting that property. it will not work
> in a secure setup.
>
> The JavaAction has the same limitation as the shell action (the shell
> action is a specialized java action that just launches a shell).
>
> Said this, would you try the following? passing the following option to
> all your hadoop tool invocations from your shell
>
> -Dmapreduce.job.credentials.binary=${HADOOP_TOKEN_FILE_LOCATION}
>
> If this work, it will work for both unsecure and secure clusters.
>
> Thx.
>
>
>
> On Thu, Feb 21, 2013 at 9:22 AM, Clément MATHIEU <cl...@unportant.info>wrote:
>
>> On 2013-02-21 18:01, Alejandro Abdelnur wrote:
>>
>> Thanks Alejandro,
>>
>> Oozie provides MR and Pig actions that propagate the user identity
>>> correctly. If you are using a shell action your shell action ends up
>>> running as the Unix user running the task, in an unsecure cluster is
>>> typically the mapred user, in a secure cluster is the same user that
>>> submitted the job to Oozie
>>>
>>
>> I fully understand that. My question was about how to make things work
>> for the shell action since I believe that my use case is legitimate. What
>> is the purpose of the shell action if even a simple operation like creating
>> a new file leads to a bad file ownership ?
>>
>> you want to run jobs against the cluster using the correct user, you
>>> should
>>> use the Oozie provided actions for that effect (MR, Pig, Sqoop, Hive,
>>> distcp, etc).
>>>
>>
>> In the meantime I did some experimentation with the HADOOP_USER_NAME
>> environment variable. Specifying it solve almost all my issues (Pig & MR
>> job are run using the right user, files have the right owner). You even
>> don't have to hard code the user name since you can use the user() EL
>> function.
>>
>> <exec>[SHELL-COMMAND]</exec>
>> <argument>[ARG-VALUE]</**argument>
>> ...
>> <env-var>HADOOP_USER_NAME=${**wf:user()}</env-var>
>> <file>[FILE-PATH]</file>
>> ...
>> </exec>
>>
>>
>> Wouldn't it be good to do something like this automatically rather the
>> put the burden on the user ? I don't see why an user would prefer the
>> current behavior.
>>
>> I'm not sure if this solution would work in secure mode too. But I
>> definitely believe that there is a better option than forcing the user to
>> always use Oozie actions even when another tool is more suitable for the
>> job.
>>
>>
>> BTW: Does the Java action suffers from the same limitation ?
>>
>>
>> Thanks,
>>
>>
>> -- Clément
>>
>
>
>
> --
> Alejandro
>
--
Alejandro
Re: Submitting Hadoop jobs from a shell action leads to the lost of
the user identity
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Hi Clément,
The HADOOP_USER_NAME trick will work only in a non-secure cluster where you
can be whoever you want just by setting that property. it will not work in
a secure setup.
The JavaAction has the same limitation as the shell action (the shell
action is a specialized java action that just launches a shell).
Said this, would you try the following? passing the following option to all
your hadoop tool invocations from your shell
-Dmapreduce.job.credentials.binary=${HADOOP_TOKEN_FILE_LOCATION}
If this work, it will work for both unsecure and secure clusters.
Thx.
On Thu, Feb 21, 2013 at 9:22 AM, Clément MATHIEU <cl...@unportant.info>wrote:
> On 2013-02-21 18:01, Alejandro Abdelnur wrote:
>
> Thanks Alejandro,
>
> Oozie provides MR and Pig actions that propagate the user identity
>> correctly. If you are using a shell action your shell action ends up
>> running as the Unix user running the task, in an unsecure cluster is
>> typically the mapred user, in a secure cluster is the same user that
>> submitted the job to Oozie
>>
>
> I fully understand that. My question was about how to make things work for
> the shell action since I believe that my use case is legitimate. What is
> the purpose of the shell action if even a simple operation like creating a
> new file leads to a bad file ownership ?
>
> you want to run jobs against the cluster using the correct user, you
>> should
>> use the Oozie provided actions for that effect (MR, Pig, Sqoop, Hive,
>> distcp, etc).
>>
>
> In the meantime I did some experimentation with the HADOOP_USER_NAME
> environment variable. Specifying it solve almost all my issues (Pig & MR
> job are run using the right user, files have the right owner). You even
> don't have to hard code the user name since you can use the user() EL
> function.
>
> <exec>[SHELL-COMMAND]</exec>
> <argument>[ARG-VALUE]</**argument>
> ...
> <env-var>HADOOP_USER_NAME=${**wf:user()}</env-var>
> <file>[FILE-PATH]</file>
> ...
> </exec>
>
>
> Wouldn't it be good to do something like this automatically rather the put
> the burden on the user ? I don't see why an user would prefer the current
> behavior.
>
> I'm not sure if this solution would work in secure mode too. But I
> definitely believe that there is a better option than forcing the user to
> always use Oozie actions even when another tool is more suitable for the
> job.
>
>
> BTW: Does the Java action suffers from the same limitation ?
>
>
> Thanks,
>
>
> -- Clément
>
--
Alejandro
Re: Submitting Hadoop jobs from a shell action leads to the lost of
the user identity
Posted by Clément MATHIEU <cl...@unportant.info>.
On 2013-02-21 18:01, Alejandro Abdelnur wrote:
Thanks Alejandro,
> Oozie provides MR and Pig actions that propagate the user identity
> correctly. If you are using a shell action your shell action ends up
> running as the Unix user running the task, in an unsecure cluster is
> typically the mapred user, in a secure cluster is the same user that
> submitted the job to Oozie
I fully understand that. My question was about how to make things work
for the shell action since I believe that my use case is legitimate.
What is the purpose of the shell action if even a simple operation like
creating a new file leads to a bad file ownership ?
> you want to run jobs against the cluster using the correct user, you
> should
> use the Oozie provided actions for that effect (MR, Pig, Sqoop, Hive,
> distcp, etc).
In the meantime I did some experimentation with the HADOOP_USER_NAME
environment variable. Specifying it solve almost all my issues (Pig & MR
job are run using the right user, files have the right owner). You even
don't have to hard code the user name since you can use the user() EL
function.
<exec>[SHELL-COMMAND]</exec>
<argument>[ARG-VALUE]</argument>
...
<env-var>HADOOP_USER_NAME=${wf:user()}</env-var>
<file>[FILE-PATH]</file>
...
</exec>
Wouldn't it be good to do something like this automatically rather the
put the burden on the user ? I don't see why an user would prefer the
current behavior.
I'm not sure if this solution would work in secure mode too. But I
definitely believe that there is a better option than forcing the user
to always use Oozie actions even when another tool is more suitable for
the job.
BTW: Does the Java action suffers from the same limitation ?
Thanks,
-- Clément
Re: Submitting Hadoop jobs from a shell action leads to the lost of
the user identity
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Clément,
Oozie provides MR and Pig actions that propagate the user identity
correctly. If you are using a shell action your shell action ends up
running as the Unix user running the task, in an unsecure cluster is
typically the mapred user, in a secure cluster is the same user that
submitted the job to Oozie (this requires user provisioning to all nodes in
the cluster), but then you'll face another problem, you user in the nodes
will not have a kerberos session. This means you are back to square one, if
you want to run jobs against the cluster using the correct user, you should
use the Oozie provided actions for that effect (MR, Pig, Sqoop, Hive,
distcp, etc).
Thx
On Thu, Feb 21, 2013 at 7:44 AM, Clément MATHIEU <cl...@unportant.info>wrote:
> Hi all,
>
> I'm currently troubleshooting an issue involving tasks ran under the
> default mapred account rather the user account and would greatly appreciate
> any help or suggestion.
>
>
> The guilty action can be summarized like that:
> - The workflow contains a shell action
> - The shell action starts a Bash shell script
> - The shell script executes several Pig scripts and starts some map
> reduce tasks
>
> The issue is that the Pig scripts and MR tasks are submitted to the
> jobtracker and executed using the mapred Hadoop user rather the same user
> that is used to run the shell action.
>
>
> I believe that this issue can trivially be explained because Hadoop relies
> on the Unix username of the process which invoked submit and, if security
> is not enabled, Hadoop uses a shared mapred Unix user to execute the tasks.
> How can I configure Oozie, or at least my action, to avoid a such escape ?
>
>
>
> PS: Using shell scripts to do Oozie's job can be seen as dumb, but the
> explanation is quite rational ;)
>
>
> Thanks,
>
> -- Clément
>
--
Alejandro