[jira] [Created] (IGNITE-218) Wrong staging permissions while running MR job under hadoop accelerator

["Konstantin Boudnik (JIRA)" <ji...@apache.org>]

Konstantin Boudnik created IGNITE-218:
-----------------------------------------

             Summary: Wrong staging permissions while running MR job under hadoop accelerator
                 Key: IGNITE-218
                 URL: https://issues.apache.org/jira/browse/IGNITE-218
             Project: Ignite
          Issue Type: Bug
          Components: hadoop
    Affects Versions: sprint-1
            Reporter: Konstantin Boudnik


I am trying to run an MR example on gridgain-hadoop setup in Bigtop. The ggfs is configured to be backed by HDFS with
{code}
        <property name="secondaryFileSystem">
          <bean class="org.gridgain.grid.kernal.ggfs.hadoop.GridGgfsHadoopFileSystemWrapper">
            <constructor-arg value="hdfs://vmhost05-hbase1.bdva.wandisco.com:8020"/>
            <constructor-arg value="/etc/hadoop/conf/hdfs-site.xml"/>
          </bean>
        </property>
{code}
set for {{ggfsConfiguration}}
While all usual filesystem operations work properly, running an MR job produces the following exception in the beginning of the run
{noformat}
HADOOP_CONF_DIR=/etc/hadoop/gridgain.client.conf hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 2 2
Number of Maps  = 2
Samples per Map = 2
Wrote input for Map #0
Wrote input for Map #1
Starting Job
java.io.IOException: The ownership on the staging directory /tmp/hadoop-yarn/staging/root/.staging is not as expected. It is owned by gridgain. The directory must be owned by the submitter root or by root
        at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:112)
        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:348)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)
        at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:306)
        at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
        at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:145)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
However the permissions seem to be correct.
{noformat}
[root@vmhost05-hbase2 gridgain.client.conf]# sudo -u hdfs hadoop fs -ls /tmp/hadoop-yarn/staging/root                                             Found 1 items
drwx------   - root root          0 2015-02-10 19:31 /tmp/hadoop-yarn/staging/root/.staging
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)