Setting permissions for slave nodes running mapper

[Joris Poort <gp...@gmail.com>]

Hi,

I'm trying to set permissions for the tasktracker and/or mapred user.
Basically I'm trying to execute and modify files from within the
mapper, but the code errors out stating that the mapred user on the
slave node doesn't have the right permissions to modify/execute files.

Any help or tips on this would be greatly appreciated!

Cheers,

Joris

Re: Setting permissions for slave nodes running mapper

[Joris Poort <gp...@gmail.com>]

Hi Harsh,

Thanks for the diagnosis.

I'm trying to execute some files already present on the OS.  I tried
to copy some files into the working directory for the mapred user
"/tmp" to use for execution, but don't even have permissions for that.

I tried using sudo but get
"org.apache.hadoop.util.Shell$ExitCodeException: sudo: no tty present
and no askpass program specified" since I'm using a java executed
process within the mapper.

Is there a way to give the mapred user root permissions?

Cheers,

Joris

On Thu, Sep 8, 2011 at 11:09 PM, Harsh J <ha...@cloudera.com> wrote:
> Hello Joris!
>
> What types of files are you trying to execute or modify? Distributed
> cache files? Your own files? Files already present on the OS?
>
> If its distributed cache stuff, or your own creations/etc., one thing
> you can try it to set "keep.failed.task.files" to "true" for your job
> and fail a task. Then head down to its specific TT's mapred.local.dir
> directory and see what the permissions of your distributed files look
> like?
>
> For the rest, can you ensure if simple tests (for permissions, etc.) like:
>
> $ sudo -u mapred <command you need to run via a task>
>
> Passes or not?
>
> On Fri, Sep 9, 2011 at 5:28 AM, Joris Poort <gp...@gmail.com> wrote:
>> Hi,
>>
>> I'm trying to set permissions for the tasktracker and/or mapred user.
>> Basically I'm trying to execute and modify files from within the
>> mapper, but the code errors out stating that the mapred user on the
>> slave node doesn't have the right permissions to modify/execute files.
>>
>> Any help or tips on this would be greatly appreciated!
>>
>> Cheers,
>>
>> Joris
>>
>
>
>
> --
> Harsh J
>

Re: Setting permissions for slave nodes running mapper

[Harsh J <ha...@cloudera.com>]

Hello Joris!

What types of files are you trying to execute or modify? Distributed
cache files? Your own files? Files already present on the OS?

If its distributed cache stuff, or your own creations/etc., one thing
you can try it to set "keep.failed.task.files" to "true" for your job
and fail a task. Then head down to its specific TT's mapred.local.dir
directory and see what the permissions of your distributed files look
like?

For the rest, can you ensure if simple tests (for permissions, etc.) like:

$ sudo -u mapred <command you need to run via a task>

Passes or not?

On Fri, Sep 9, 2011 at 5:28 AM, Joris Poort <gp...@gmail.com> wrote:
> Hi,
>
> I'm trying to set permissions for the tasktracker and/or mapred user.
> Basically I'm trying to execute and modify files from within the
> mapper, but the code errors out stating that the mapred user on the
> slave node doesn't have the right permissions to modify/execute files.
>
> Any help or tips on this would be greatly appreciated!
>
> Cheers,
>
> Joris
>



-- 
Harsh J