Getting Permission denied to run shell action

[rammohan ganapavarapu <ra...@gmail.com>]

Hi,

I have changed the yarn.nodemanager.local-dirs from default /tmp/ to
/data/tmp dir and from then i am getting this error



Cannot run program "test.sh" (in directory
"/data/tmp/nm-local-dir/usercache/hadoop/appcache/application_1476931300239_0003/container_1476931300239_0003_01_000002"):
error=13, Permission denied

Property in yarn-site.xml

${hadoop.tmp.dir}=/data/tmp/
<property>
 <name>yarn.nodemanager.local-dirs</name>
 <value>${hadoop.tmp.dir}/nm-local-dir</value>
</property>

Not sure why i am getting this error, i am submitting job wtih the same
user a directory owner,i have changed the ${hadoop.tmp.dir}=/tmp/ but oozie
still saying /data/* permission denied, i am not sure whats going on here,
can some one help me to figure it out?

Thanks,
Ram

Re: Getting Permission denied to run shell action

[rammohan ganapavarapu <ra...@gmail.com>]

So by changing disk mount options this issue disappeared not sure how mount
options causing permissions denied issue for oozie or yarn.

/dev/md0 on /data type ext4
(rw,nosuid,nodev,noexec,relatime,stripe=128,data=ordered)

I have changed mount options from above to below and it worked.

/dev/md0 on /data type ext4 (rw,relatime,stripe=128,data=ordered)

Do you guys know any disk/mount best practices?

Ram

On Fri, Oct 21, 2016 at 8:31 AM, rammohan ganapavarapu <
rammohanganap@gmail.com> wrote:

> I have rwx, I actually have 777 on that dir.
>
> On Oct 21, 2016 2:21 AM, <wg...@gmail.com> wrote:
>
>> Depends if /data is an directory or an mountpoint. Note, you need rwx for
>> directories
>>
>>
>>
>> --alex
>>
>>
>>
>> --
>> B: mapredit.blogspot.com
>>
>>
>>
>> *From: *rammohan ganapavarapu <ra...@gmail.com>
>> *Sent: *Friday, October 21, 2016 8:27 AM
>> *To: *user@oozie.apache.org
>> *Cc: *user@hadoop.apache.org
>> *Subject: *Re: Getting Permission denied to run shell action
>>
>>
>>
>> I am running oozie and yarn apps with same user and yes user can rw to
>> that directory.
>>
>>
>>
>> On Oct 20, 2016 11:17 PM, "Peter Cseh" <ge...@cloudera.com> wrote:
>>
>> Hi Ram,
>>
>> Can the yarn/oozie user write to /data/tmp?
>>
>> Check the permissions and ownership of /data/tmp.
>> Hope it helps
>> Gp
>>
>>
>> On Fri, Oct 21, 2016 at 1:32 AM, rammohan ganapavarapu <
>> rammohanganap@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > I have changed the yarn.nodemanager.local-dirs from default /tmp/ to
>> > /data/tmp dir and from then i am getting this error
>> >
>> >
>> >
>> > Cannot run program "test.sh" (in directory
>> > "/data/tmp/nm-local-dir/usercache/hadoop/appcache/
>> > application_1476931300239_0003/container_1476931300239_0003_
>> 01_000002"):
>> > error=13, Permission denied
>> >
>> > Property in yarn-site.xml
>> >
>> > ${hadoop.tmp.dir}=/data/tmp/
>> > <property>
>> >  <name>yarn.nodemanager.local-dirs</name>
>> >  <value>${hadoop.tmp.dir}/nm-local-dir</value>
>> > </property>
>> >
>> > Not sure why i am getting this error, i am submitting job wtih the same
>> > user a directory owner,i have changed the ${hadoop.tmp.dir}=/tmp/ but
>> oozie
>> > still saying /data/* permission denied, i am not sure whats going on
>> here,
>> > can some one help me to figure it out?
>> >
>> > Thanks,
>> > Ram
>> >
>>
>>
>>
>> --
>> Peter Cseh
>> Software Engineer
>> <http://www.cloudera.com>
>>
>>
>>
>

Re: Getting Permission denied to run shell action

[rammohan ganapavarapu <ra...@gmail.com>]

So by changing disk mount options this issue disappeared not sure how mount
options causing permissions denied issue for oozie or yarn.

/dev/md0 on /data type ext4
(rw,nosuid,nodev,noexec,relatime,stripe=128,data=ordered)

I have changed mount options from above to below and it worked.

/dev/md0 on /data type ext4 (rw,relatime,stripe=128,data=ordered)

Do you guys know any disk/mount best practices?

Ram

On Fri, Oct 21, 2016 at 8:31 AM, rammohan ganapavarapu <
rammohanganap@gmail.com> wrote:

> I have rwx, I actually have 777 on that dir.
>
> On Oct 21, 2016 2:21 AM, <wg...@gmail.com> wrote:
>
>> Depends if /data is an directory or an mountpoint. Note, you need rwx for
>> directories
>>
>>
>>
>> --alex
>>
>>
>>
>> --
>> B: mapredit.blogspot.com
>>
>>
>>
>> *From: *rammohan ganapavarapu <ra...@gmail.com>
>> *Sent: *Friday, October 21, 2016 8:27 AM
>> *To: *user@oozie.apache.org
>> *Cc: *user@hadoop.apache.org
>> *Subject: *Re: Getting Permission denied to run shell action
>>
>>
>>
>> I am running oozie and yarn apps with same user and yes user can rw to
>> that directory.
>>
>>
>>
>> On Oct 20, 2016 11:17 PM, "Peter Cseh" <ge...@cloudera.com> wrote:
>>
>> Hi Ram,
>>
>> Can the yarn/oozie user write to /data/tmp?
>>
>> Check the permissions and ownership of /data/tmp.
>> Hope it helps
>> Gp
>>
>>
>> On Fri, Oct 21, 2016 at 1:32 AM, rammohan ganapavarapu <
>> rammohanganap@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > I have changed the yarn.nodemanager.local-dirs from default /tmp/ to
>> > /data/tmp dir and from then i am getting this error
>> >
>> >
>> >
>> > Cannot run program "test.sh" (in directory
>> > "/data/tmp/nm-local-dir/usercache/hadoop/appcache/
>> > application_1476931300239_0003/container_1476931300239_0003_
>> 01_000002"):
>> > error=13, Permission denied
>> >
>> > Property in yarn-site.xml
>> >
>> > ${hadoop.tmp.dir}=/data/tmp/
>> > <property>
>> >  <name>yarn.nodemanager.local-dirs</name>
>> >  <value>${hadoop.tmp.dir}/nm-local-dir</value>
>> > </property>
>> >
>> > Not sure why i am getting this error, i am submitting job wtih the same
>> > user a directory owner,i have changed the ${hadoop.tmp.dir}=/tmp/ but
>> oozie
>> > still saying /data/* permission denied, i am not sure whats going on
>> here,
>> > can some one help me to figure it out?
>> >
>> > Thanks,
>> > Ram
>> >
>>
>>
>>
>> --
>> Peter Cseh
>> Software Engineer
>> <http://www.cloudera.com>
>>
>>
>>
>

RE: Getting Permission denied to run shell action

[rammohan ganapavarapu <ra...@gmail.com>]

I have rwx, I actually have 777 on that dir.

On Oct 21, 2016 2:21 AM, <wg...@gmail.com> wrote:

> Depends if /data is an directory or an mountpoint. Note, you need rwx for
> directories
>
>
>
> --alex
>
>
>
> --
> B: mapredit.blogspot.com
>
>
>
> *From: *rammohan ganapavarapu <ra...@gmail.com>
> *Sent: *Friday, October 21, 2016 8:27 AM
> *To: *user@oozie.apache.org
> *Cc: *user@hadoop.apache.org
> *Subject: *Re: Getting Permission denied to run shell action
>
>
>
> I am running oozie and yarn apps with same user and yes user can rw to
> that directory.
>
>
>
> On Oct 20, 2016 11:17 PM, "Peter Cseh" <ge...@cloudera.com> wrote:
>
> Hi Ram,
>
> Can the yarn/oozie user write to /data/tmp?
>
> Check the permissions and ownership of /data/tmp.
> Hope it helps
> Gp
>
>
> On Fri, Oct 21, 2016 at 1:32 AM, rammohan ganapavarapu <
> rammohanganap@gmail.com> wrote:
>
> > Hi,
> >
> > I have changed the yarn.nodemanager.local-dirs from default /tmp/ to
> > /data/tmp dir and from then i am getting this error
> >
> >
> >
> > Cannot run program "test.sh" (in directory
> > "/data/tmp/nm-local-dir/usercache/hadoop/appcache/
> > application_1476931300239_0003/container_1476931300239_0003_01_000002"):
> > error=13, Permission denied
> >
> > Property in yarn-site.xml
> >
> > ${hadoop.tmp.dir}=/data/tmp/
> > <property>
> >  <name>yarn.nodemanager.local-dirs</name>
> >  <value>${hadoop.tmp.dir}/nm-local-dir</value>
> > </property>
> >
> > Not sure why i am getting this error, i am submitting job wtih the same
> > user a directory owner,i have changed the ${hadoop.tmp.dir}=/tmp/ but
> oozie
> > still saying /data/* permission denied, i am not sure whats going on
> here,
> > can some one help me to figure it out?
> >
> > Thanks,
> > Ram
> >
>
>
>
> --
> Peter Cseh
> Software Engineer
> <http://www.cloudera.com>
>
>
>

RE: Getting Permission denied to run shell action

[rammohan ganapavarapu <ra...@gmail.com>]

I have rwx, I actually have 777 on that dir.

On Oct 21, 2016 2:21 AM, <wg...@gmail.com> wrote:

> Depends if /data is an directory or an mountpoint. Note, you need rwx for
> directories
>
>
>
> --alex
>
>
>
> --
> B: mapredit.blogspot.com
>
>
>
> *From: *rammohan ganapavarapu <ra...@gmail.com>
> *Sent: *Friday, October 21, 2016 8:27 AM
> *To: *user@oozie.apache.org
> *Cc: *user@hadoop.apache.org
> *Subject: *Re: Getting Permission denied to run shell action
>
>
>
> I am running oozie and yarn apps with same user and yes user can rw to
> that directory.
>
>
>
> On Oct 20, 2016 11:17 PM, "Peter Cseh" <ge...@cloudera.com> wrote:
>
> Hi Ram,
>
> Can the yarn/oozie user write to /data/tmp?
>
> Check the permissions and ownership of /data/tmp.
> Hope it helps
> Gp
>
>
> On Fri, Oct 21, 2016 at 1:32 AM, rammohan ganapavarapu <
> rammohanganap@gmail.com> wrote:
>
> > Hi,
> >
> > I have changed the yarn.nodemanager.local-dirs from default /tmp/ to
> > /data/tmp dir and from then i am getting this error
> >
> >
> >
> > Cannot run program "test.sh" (in directory
> > "/data/tmp/nm-local-dir/usercache/hadoop/appcache/
> > application_1476931300239_0003/container_1476931300239_0003_01_000002"):
> > error=13, Permission denied
> >
> > Property in yarn-site.xml
> >
> > ${hadoop.tmp.dir}=/data/tmp/
> > <property>
> >  <name>yarn.nodemanager.local-dirs</name>
> >  <value>${hadoop.tmp.dir}/nm-local-dir</value>
> > </property>
> >
> > Not sure why i am getting this error, i am submitting job wtih the same
> > user a directory owner,i have changed the ${hadoop.tmp.dir}=/tmp/ but
> oozie
> > still saying /data/* permission denied, i am not sure whats going on
> here,
> > can some one help me to figure it out?
> >
> > Thanks,
> > Ram
> >
>
>
>
> --
> Peter Cseh
> Software Engineer
> <http://www.cloudera.com>
>
>
>

RE: Getting Permission denied to run shell action

[wg...@gmail.com]

Depends if /data is an directory or an mountpoint. Note, you need rwx for directories

--alex

--
B: mapredit.blogspot.com

From: rammohan ganapavarapu
Sent: Friday, October 21, 2016 8:27 AM
To: user@oozie.apache.org
Cc: user@hadoop.apache.org
Subject: Re: Getting Permission denied to run shell action

I am running oozie and yarn apps with same user and yes user can rw to that directory.

On Oct 20, 2016 11:17 PM, "Peter Cseh" <ge...@cloudera.com> wrote:
Hi Ram,

Can the yarn/oozie user write to /data/tmp?

Check the permissions and ownership of /data/tmp.
Hope it helps
Gp


On Fri, Oct 21, 2016 at 1:32 AM, rammohan ganapavarapu <
rammohanganap@gmail.com> wrote:

> Hi,
>
> I have changed the yarn.nodemanager.local-dirs from default /tmp/ to
> /data/tmp dir and from then i am getting this error
>
>
>
> Cannot run program "test.sh" (in directory
> "/data/tmp/nm-local-dir/usercache/hadoop/appcache/
> application_1476931300239_0003/container_1476931300239_0003_01_000002"):
> error=13, Permission denied
>
> Property in yarn-site.xml
>
> ${hadoop.tmp.dir}=/data/tmp/
> <property>
>  <name>yarn.nodemanager.local-dirs</name>
>  <value>${hadoop.tmp.dir}/nm-local-dir</value>
> </property>
>
> Not sure why i am getting this error, i am submitting job wtih the same
> user a directory owner,i have changed the ${hadoop.tmp.dir}=/tmp/ but oozie
> still saying /data/* permission denied, i am not sure whats going on here,
> can some one help me to figure it out?
>
> Thanks,
> Ram
>



--
Peter Cseh
Software Engineer
<http://www.cloudera.com>

Re: Getting Permission denied to run shell action

[rammohan ganapavarapu <ra...@gmail.com>]

I am running oozie and yarn apps with same user and yes user can rw to that
directory.

On Oct 20, 2016 11:17 PM, "Peter Cseh" <ge...@cloudera.com> wrote:

> Hi Ram,
>
> Can the yarn/oozie user write to /data/tmp?
>
> Check the permissions and ownership of /data/tmp.
> Hope it helps
> Gp
>
>
> On Fri, Oct 21, 2016 at 1:32 AM, rammohan ganapavarapu <
> rammohanganap@gmail.com> wrote:
>
> > Hi,
> >
> > I have changed the yarn.nodemanager.local-dirs from default /tmp/ to
> > /data/tmp dir and from then i am getting this error
> >
> >
> >
> > Cannot run program "test.sh" (in directory
> > "/data/tmp/nm-local-dir/usercache/hadoop/appcache/
> > application_1476931300239_0003/container_1476931300239_0003_01_000002"):
> > error=13, Permission denied
> >
> > Property in yarn-site.xml
> >
> > ${hadoop.tmp.dir}=/data/tmp/
> > <property>
> >  <name>yarn.nodemanager.local-dirs</name>
> >  <value>${hadoop.tmp.dir}/nm-local-dir</value>
> > </property>
> >
> > Not sure why i am getting this error, i am submitting job wtih the same
> > user a directory owner,i have changed the ${hadoop.tmp.dir}=/tmp/ but
> oozie
> > still saying /data/* permission denied, i am not sure whats going on
> here,
> > can some one help me to figure it out?
> >
> > Thanks,
> > Ram
> >
>
>
>
> --
> Peter Cseh
> Software Engineer
> <http://www.cloudera.com>
>

Re: Getting Permission denied to run shell action

[rammohan ganapavarapu <ra...@gmail.com>]

I am running oozie and yarn apps with same user and yes user can rw to that
directory.

On Oct 20, 2016 11:17 PM, "Peter Cseh" <ge...@cloudera.com> wrote:

> Hi Ram,
>
> Can the yarn/oozie user write to /data/tmp?
>
> Check the permissions and ownership of /data/tmp.
> Hope it helps
> Gp
>
>
> On Fri, Oct 21, 2016 at 1:32 AM, rammohan ganapavarapu <
> rammohanganap@gmail.com> wrote:
>
> > Hi,
> >
> > I have changed the yarn.nodemanager.local-dirs from default /tmp/ to
> > /data/tmp dir and from then i am getting this error
> >
> >
> >
> > Cannot run program "test.sh" (in directory
> > "/data/tmp/nm-local-dir/usercache/hadoop/appcache/
> > application_1476931300239_0003/container_1476931300239_0003_01_000002"):
> > error=13, Permission denied
> >
> > Property in yarn-site.xml
> >
> > ${hadoop.tmp.dir}=/data/tmp/
> > <property>
> >  <name>yarn.nodemanager.local-dirs</name>
> >  <value>${hadoop.tmp.dir}/nm-local-dir</value>
> > </property>
> >
> > Not sure why i am getting this error, i am submitting job wtih the same
> > user a directory owner,i have changed the ${hadoop.tmp.dir}=/tmp/ but
> oozie
> > still saying /data/* permission denied, i am not sure whats going on
> here,
> > can some one help me to figure it out?
> >
> > Thanks,
> > Ram
> >
>
>
>
> --
> Peter Cseh
> Software Engineer
> <http://www.cloudera.com>
>

Re: Getting Permission denied to run shell action

[Peter Cseh <ge...@cloudera.com>]

Hi Ram,

Can the yarn/oozie user write to /data/tmp?

Check the permissions and ownership of /data/tmp.
Hope it helps
Gp


On Fri, Oct 21, 2016 at 1:32 AM, rammohan ganapavarapu <
rammohanganap@gmail.com> wrote:

> Hi,
>
> I have changed the yarn.nodemanager.local-dirs from default /tmp/ to
> /data/tmp dir and from then i am getting this error
>
>
>
> Cannot run program "test.sh" (in directory
> "/data/tmp/nm-local-dir/usercache/hadoop/appcache/
> application_1476931300239_0003/container_1476931300239_0003_01_000002"):
> error=13, Permission denied
>
> Property in yarn-site.xml
>
> ${hadoop.tmp.dir}=/data/tmp/
> <property>
>  <name>yarn.nodemanager.local-dirs</name>
>  <value>${hadoop.tmp.dir}/nm-local-dir</value>
> </property>
>
> Not sure why i am getting this error, i am submitting job wtih the same
> user a directory owner,i have changed the ${hadoop.tmp.dir}=/tmp/ but oozie
> still saying /data/* permission denied, i am not sure whats going on here,
> can some one help me to figure it out?
>
> Thanks,
> Ram
>



-- 
Peter Cseh
Software Engineer
<http://www.cloudera.com>

Re: Getting Permission denied to run shell action

[Peter Cseh <ge...@cloudera.com>]

Hi Ram,

Can the yarn/oozie user write to /data/tmp?

Check the permissions and ownership of /data/tmp.
Hope it helps
Gp


On Fri, Oct 21, 2016 at 1:32 AM, rammohan ganapavarapu <
rammohanganap@gmail.com> wrote:

> Hi,
>
> I have changed the yarn.nodemanager.local-dirs from default /tmp/ to
> /data/tmp dir and from then i am getting this error
>
>
>
> Cannot run program "test.sh" (in directory
> "/data/tmp/nm-local-dir/usercache/hadoop/appcache/
> application_1476931300239_0003/container_1476931300239_0003_01_000002"):
> error=13, Permission denied
>
> Property in yarn-site.xml
>
> ${hadoop.tmp.dir}=/data/tmp/
> <property>
>  <name>yarn.nodemanager.local-dirs</name>
>  <value>${hadoop.tmp.dir}/nm-local-dir</value>
> </property>
>
> Not sure why i am getting this error, i am submitting job wtih the same
> user a directory owner,i have changed the ${hadoop.tmp.dir}=/tmp/ but oozie
> still saying /data/* permission denied, i am not sure whats going on here,
> can some one help me to figure it out?
>
> Thanks,
> Ram
>



-- 
Peter Cseh
Software Engineer
<http://www.cloudera.com>