You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by "Peng, Wei" <We...@xerox.com> on 2011/04/24 08:40:40 UTC
HDFS permission denied
Hi,
I need a help very bad.
I got an HDFS permission error by starting to run hadoop job
org.apache.hadoop.security.AccessControlException: Permission denied:
user=wp, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x
I have the right permission to read and write files to my own hadoop
user directory.
It works fine when I use hadoop fs -put. The job input and output are
all from my own hadoop user directory.
It seems that when a job starts running, some data need to be written
into some directory, and I don't have the permission to that directory.
It is strange that the inode does not show which directory it is.
Why does hadoop write something to a directory with my name secretly? Do
I need to be set a particular user group?
Many Thanks..
Vivian
RE: HDFS permission denied
Posted by "Peng, Wei" <We...@xerox.com>.
Thanks for replying me!
I will check the name node log and tmp setting tomorrow.
Jame, do you mean that hadoop will write to tmp directory where I don't
have write permission?
Wei
-----Original Message-----
From: James Seigel [mailto:james@tynt.com]
Sent: Sunday, April 24, 2011 5:36 AM
To: common-user@hadoop.apache.org
Subject: Re: HDFS permission denied
Check where the hadoop tmp setting is pointing to.
James
Sent from my mobile. Please excuse the typos.
On 2011-04-24, at 12:41 AM, "Peng, Wei" <We...@xerox.com> wrote:
> Hi,
>
>
>
> I need a help very bad.
>
>
>
> I got an HDFS permission error by starting to run hadoop job
>
> org.apache.hadoop.security.AccessControlException: Permission denied:
>
> user=wp, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x
>
>
>
> I have the right permission to read and write files to my own hadoop
> user directory.
>
> It works fine when I use hadoop fs -put. The job input and output are
> all from my own hadoop user directory.
>
>
>
> It seems that when a job starts running, some data need to be written
> into some directory, and I don't have the permission to that
directory.
> It is strange that the inode does not show which directory it is.
>
>
>
> Why does hadoop write something to a directory with my name secretly?
Do
> I need to be set a particular user group?
>
>
>
> Many Thanks..
>
>
>
> Vivian
>
>
>
>
>
Re: HDFS permission denied
Posted by James Warren <ja...@stanfordalumni.org>.
At this point you should follow Mathias' advice - go to the logs and
determine which path has the permission issue. It's better to change the
settings for that path rather than disabling permissions (i.e. making
everything 777) randomly.
-jw
On Mon, Apr 25, 2011 at 10:04 AM, Peng, Wei <We...@xerox.com> wrote:
> James,
>
> Thanks for your replies.
> In this case, how can I set up the permission correctly in order to run
> a hadoop job?
> Do I need to set hadoop tmp directory (which is in the local directory
> instead of hdfs directory,right?) to be 777?
> Since the person who maintain the hadoop cluster has left, I have no
> idea what happened. =(
>
> Wei
>
> -----Original Message-----
> From: jameswarren3@gmail.com [mailto:jameswarren3@gmail.com] On Behalf
> Of James Warren
> Sent: Monday, April 25, 2011 9:56 AM
> To: common-user@hadoop.apache.org
> Subject: Re: HDFS permission denied
>
> Hi Wei -
>
> In general, settings changes aren't applied until the hadoop daemons are
> restarted. Sounds like someone enabled permissions previously, but they
> didn't take hold until you rebooted your cluster.
>
> cheers,
> -James
>
> On Mon, Apr 25, 2011 at 1:19 AM, Peng, Wei <We...@xerox.com> wrote:
>
> > I forgot to mention that the hadoop was running fine before.
> > However, after it crashed last week, the restarted hadoop cluster has
> > such permission issues.
> > So that means the settings are still as same as before.
> > Then what would be the cause?
> >
> > Wei
> >
> > -----Original Message-----
> > From: James Seigel [mailto:james@tynt.com]
> > Sent: Sunday, April 24, 2011 5:36 AM
> > To: common-user@hadoop.apache.org
> > Subject: Re: HDFS permission denied
> >
> > Check where the hadoop tmp setting is pointing to.
> >
> > James
> >
> > Sent from my mobile. Please excuse the typos.
> >
> > On 2011-04-24, at 12:41 AM, "Peng, Wei" <We...@xerox.com> wrote:
> >
> > > Hi,
> > >
> > >
> > >
> > > I need a help very bad.
> > >
> > >
> > >
> > > I got an HDFS permission error by starting to run hadoop job
> > >
> > > org.apache.hadoop.security.AccessControlException: Permission
> denied:
> > >
> > > user=wp, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x
> > >
> > >
> > >
> > > I have the right permission to read and write files to my own hadoop
> > > user directory.
> > >
> > > It works fine when I use hadoop fs -put. The job input and output
> are
> > > all from my own hadoop user directory.
> > >
> > >
> > >
> > > It seems that when a job starts running, some data need to be
> written
> > > into some directory, and I don't have the permission to that
> > directory.
> > > It is strange that the inode does not show which directory it is.
> > >
> > >
> > >
> > > Why does hadoop write something to a directory with my name
> secretly?
> > Do
> > > I need to be set a particular user group?
> > >
> > >
> > >
> > > Many Thanks..
> > >
> > >
> > >
> > > Vivian
> > >
> > >
> > >
> > >
> > >
> >
>
RE: HDFS permission denied
Posted by "Peng, Wei" <We...@xerox.com>.
James,
Thanks for your replies.
In this case, how can I set up the permission correctly in order to run
a hadoop job?
Do I need to set hadoop tmp directory (which is in the local directory
instead of hdfs directory,right?) to be 777?
Since the person who maintain the hadoop cluster has left, I have no
idea what happened. =(
Wei
-----Original Message-----
From: jameswarren3@gmail.com [mailto:jameswarren3@gmail.com] On Behalf
Of James Warren
Sent: Monday, April 25, 2011 9:56 AM
To: common-user@hadoop.apache.org
Subject: Re: HDFS permission denied
Hi Wei -
In general, settings changes aren't applied until the hadoop daemons are
restarted. Sounds like someone enabled permissions previously, but they
didn't take hold until you rebooted your cluster.
cheers,
-James
On Mon, Apr 25, 2011 at 1:19 AM, Peng, Wei <We...@xerox.com> wrote:
> I forgot to mention that the hadoop was running fine before.
> However, after it crashed last week, the restarted hadoop cluster has
> such permission issues.
> So that means the settings are still as same as before.
> Then what would be the cause?
>
> Wei
>
> -----Original Message-----
> From: James Seigel [mailto:james@tynt.com]
> Sent: Sunday, April 24, 2011 5:36 AM
> To: common-user@hadoop.apache.org
> Subject: Re: HDFS permission denied
>
> Check where the hadoop tmp setting is pointing to.
>
> James
>
> Sent from my mobile. Please excuse the typos.
>
> On 2011-04-24, at 12:41 AM, "Peng, Wei" <We...@xerox.com> wrote:
>
> > Hi,
> >
> >
> >
> > I need a help very bad.
> >
> >
> >
> > I got an HDFS permission error by starting to run hadoop job
> >
> > org.apache.hadoop.security.AccessControlException: Permission
denied:
> >
> > user=wp, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x
> >
> >
> >
> > I have the right permission to read and write files to my own hadoop
> > user directory.
> >
> > It works fine when I use hadoop fs -put. The job input and output
are
> > all from my own hadoop user directory.
> >
> >
> >
> > It seems that when a job starts running, some data need to be
written
> > into some directory, and I don't have the permission to that
> directory.
> > It is strange that the inode does not show which directory it is.
> >
> >
> >
> > Why does hadoop write something to a directory with my name
secretly?
> Do
> > I need to be set a particular user group?
> >
> >
> >
> > Many Thanks..
> >
> >
> >
> > Vivian
> >
> >
> >
> >
> >
>
Re: HDFS permission denied
Posted by James Warren <ja...@stanfordalumni.org>.
Hi Wei -
In general, settings changes aren't applied until the hadoop daemons are
restarted. Sounds like someone enabled permissions previously, but they
didn't take hold until you rebooted your cluster.
cheers,
-James
On Mon, Apr 25, 2011 at 1:19 AM, Peng, Wei <We...@xerox.com> wrote:
> I forgot to mention that the hadoop was running fine before.
> However, after it crashed last week, the restarted hadoop cluster has
> such permission issues.
> So that means the settings are still as same as before.
> Then what would be the cause?
>
> Wei
>
> -----Original Message-----
> From: James Seigel [mailto:james@tynt.com]
> Sent: Sunday, April 24, 2011 5:36 AM
> To: common-user@hadoop.apache.org
> Subject: Re: HDFS permission denied
>
> Check where the hadoop tmp setting is pointing to.
>
> James
>
> Sent from my mobile. Please excuse the typos.
>
> On 2011-04-24, at 12:41 AM, "Peng, Wei" <We...@xerox.com> wrote:
>
> > Hi,
> >
> >
> >
> > I need a help very bad.
> >
> >
> >
> > I got an HDFS permission error by starting to run hadoop job
> >
> > org.apache.hadoop.security.AccessControlException: Permission denied:
> >
> > user=wp, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x
> >
> >
> >
> > I have the right permission to read and write files to my own hadoop
> > user directory.
> >
> > It works fine when I use hadoop fs -put. The job input and output are
> > all from my own hadoop user directory.
> >
> >
> >
> > It seems that when a job starts running, some data need to be written
> > into some directory, and I don't have the permission to that
> directory.
> > It is strange that the inode does not show which directory it is.
> >
> >
> >
> > Why does hadoop write something to a directory with my name secretly?
> Do
> > I need to be set a particular user group?
> >
> >
> >
> > Many Thanks..
> >
> >
> >
> > Vivian
> >
> >
> >
> >
> >
>
RE: HDFS permission denied
Posted by "Peng, Wei" <We...@xerox.com>.
I forgot to mention that the hadoop was running fine before.
However, after it crashed last week, the restarted hadoop cluster has
such permission issues.
So that means the settings are still as same as before.
Then what would be the cause?
Wei
-----Original Message-----
From: James Seigel [mailto:james@tynt.com]
Sent: Sunday, April 24, 2011 5:36 AM
To: common-user@hadoop.apache.org
Subject: Re: HDFS permission denied
Check where the hadoop tmp setting is pointing to.
James
Sent from my mobile. Please excuse the typos.
On 2011-04-24, at 12:41 AM, "Peng, Wei" <We...@xerox.com> wrote:
> Hi,
>
>
>
> I need a help very bad.
>
>
>
> I got an HDFS permission error by starting to run hadoop job
>
> org.apache.hadoop.security.AccessControlException: Permission denied:
>
> user=wp, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x
>
>
>
> I have the right permission to read and write files to my own hadoop
> user directory.
>
> It works fine when I use hadoop fs -put. The job input and output are
> all from my own hadoop user directory.
>
>
>
> It seems that when a job starts running, some data need to be written
> into some directory, and I don't have the permission to that
directory.
> It is strange that the inode does not show which directory it is.
>
>
>
> Why does hadoop write something to a directory with my name secretly?
Do
> I need to be set a particular user group?
>
>
>
> Many Thanks..
>
>
>
> Vivian
>
>
>
>
>
Re: HDFS permission denied
Posted by James Seigel <ja...@tynt.com>.
Check where the hadoop tmp setting is pointing to.
James
Sent from my mobile. Please excuse the typos.
On 2011-04-24, at 12:41 AM, "Peng, Wei" <We...@xerox.com> wrote:
> Hi,
>
>
>
> I need a help very bad.
>
>
>
> I got an HDFS permission error by starting to run hadoop job
>
> org.apache.hadoop.security.AccessControlException: Permission denied:
>
> user=wp, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x
>
>
>
> I have the right permission to read and write files to my own hadoop
> user directory.
>
> It works fine when I use hadoop fs -put. The job input and output are
> all from my own hadoop user directory.
>
>
>
> It seems that when a job starts running, some data need to be written
> into some directory, and I don't have the permission to that directory.
> It is strange that the inode does not show which directory it is.
>
>
>
> Why does hadoop write something to a directory with my name secretly? Do
> I need to be set a particular user group?
>
>
>
> Many Thanks..
>
>
>
> Vivian
>
>
>
>
>
Re: HDFS permission denied
Posted by Mathias Herberts <ma...@gmail.com>.
Check the NN's logs to see the path which led to this.
On Apr 24, 2011 8:41 AM, "Peng, Wei" <We...@xerox.com> wrote:
> Hi,
>
>
>
> I need a help very bad.
>
>
>
> I got an HDFS permission error by starting to run hadoop job
>
> org.apache.hadoop.security.AccessControlException: Permission denied:
>
> user=wp, access=WRITE, inode="":hadoop:supergroup:rwxr-xr-x
>
>
>
> I have the right permission to read and write files to my own hadoop
> user directory.
>
> It works fine when I use hadoop fs -put. The job input and output are
> all from my own hadoop user directory.
>
>
>
> It seems that when a job starts running, some data need to be written
> into some directory, and I don't have the permission to that directory.
> It is strange that the inode does not show which directory it is.
>
>
>
> Why does hadoop write something to a directory with my name secretly? Do
> I need to be set a particular user group?
>
>
>
> Many Thanks..
>
>
>
> Vivian
>
>
>
>
>