[jira] [Commented] (HADOOP-9923) yarn staging area on hdfs has wrong permission and is created by the wrong user

["André Kelpe (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-9923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13756006#comment-13756006 ] 

André Kelpe commented on HADOOP-9923:
-------------------------------------

bigtop for sure can help, but shouldn't the hadoop distribution itself be deployable "as-is", even if it is not the most optimal way? 

The fact, that the history server creates a directory, that it is only supposed to read from, is IMO a bug. If it is not a bug, then it should at least be documented, how to initialize hdfs correctly, by just using the hadoop tarball. It is somewhat mentioned in the docs, but the fact that you have to do that in the middle of the daemon start-ups ins non-obvious. At least to me. 

It is also a regression from hadoop 1.x, where you could fire up a cluster with start-all.sh and it worked. (I know that start-all.sh is not the most ideal way, but it does work, for certain use cases.)
                
> yarn staging area on hdfs has wrong permission and is created by the wrong user
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-9923
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9923
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.1.0-beta
>            Reporter: André Kelpe
>
> I am setting up a cluster with hadoop 2.1-beta that consists of the following components:
> master: runs the namenode, the resourcemanager and the job history server.
> hadoop1, hadoop2, hadoop3: run datanodes and node managers
> I created 3 system users for the different components, like explained in the docs:
> hdfs: runs all things hdfs
> yarn: runs all things yarn
> mapred: runs the job history server
> If I now boot up the cluster, I cannot submit jobs since the yarn staging area permissions do not allow it.
> What I found out is, that the job-history-server is creating the staging directory while starting. This first of all causes it to be owned by the wrong user (mapred) and having the wrong permision (770). The docs are not really clear if I am supposed to start hdfs first, then create the staging area by hand and then start the job-history-server or if this is supposed to happen automatically by itself.
> In any case, either the docs should be updated or the job-history-server should not create the directory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira