[jira] [Commented] (HADOOP-9923) yarn staging area on hdfs has wrong permission and is created by the wrong user

["Roman Shaposhnik (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-9923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13754969#comment-13754969 ] 

Roman Shaposhnik commented on HADOOP-9923:
------------------------------------------

I've always thought that it is better to initialize HDFS before the first use just like we've come to expect to initialize a freshly formatted root Linux filesystem with something like 'base-files' package in Debian. In fact, Bigtop provides a specicial initialization script just for that purpose so that users of Bigtop-derived distributions of Hadoop can simply run init-hdfs and now worry about minute details of exact perms/ownerships.

You can take a look at our implementation over here: https://git-wip-us.apache.org/repos/asf?p=bigtop.git;a=blob;f=bigtop-packages/src/common/hadoop/init-hdfs.sh;h=bc96761cef604a6bb42fc09e7d439b8250993973;hb=HEAD

We also plan to improve it for Bigtop 0.7.0 release.
   
                
> yarn staging area on hdfs has wrong permission and is created by the wrong user
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-9923
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9923
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.1.0-beta
>            Reporter: André Kelpe
>
> I am setting up a cluster with hadoop 2.1-beta that consists of the following components:
> master: runs the namenode, the resourcemanager and the job history server.
> hadoop1, hadoop2, hadoop3: run datanodes and node managers
> I created 3 system users for the different components, like explained in the docs:
> hdfs: runs all things hdfs
> yarn: runs all things yarn
> mapred: runs the job history server
> If I now boot up the cluster, I cannot submit jobs since the yarn staging area permissions do not allow it.
> What I found out is, that the job-history-server is creating the staging directory while starting. This first of all causes it to be owned by the wrong user (mapred) and having the wrong permision (770). The docs are not really clear if I am supposed to start hdfs first, then create the staging area by hand and then start the job-history-server or if this is supposed to happen automatically by itself.
> In any case, either the docs should be updated or the job-history-server should not create the directory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira