You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ramya Sunil (JIRA)" <ji...@apache.org> on 2011/07/15 19:12:59 UTC
[jira] [Created] (MAPREDUCE-2687) Non superusers unable to launch
apps in a non-secure cluster
Non superusers unable to launch apps in a non-secure cluster
------------------------------------------------------------
Key: MAPREDUCE-2687
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Fix For: 0.23.0
Apps of non superuser fail to succeed in a non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2687) Non superusers unable to launch
apps in both secure and non-secure cluster
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2687:
-------------------------------------
Status: Patch Available (was: Open)
Looks like there is an issue with my secure env caused by random timing issues. This patch seems to work.
> Non superusers unable to launch apps in both secure and non-secure cluster
> --------------------------------------------------------------------------
>
> Key: MAPREDUCE-2687
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2687.patch, MAREDUCE-2687.patch
>
>
> Apps of non superuser fail to succeed in both secure and non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2687) Non superusers unable to launch
apps in both secure and non-secure cluster
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098918#comment-13098918 ]
Hudson commented on MAPREDUCE-2687:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #809 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/809/])
MAPREDUCE-2687. Fix NodeManager to use the right version of LocalDirAllocator.getLocalPathToWrite. Contributed by mahadev & acmurthy.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165890
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
> Non superusers unable to launch apps in both secure and non-secure cluster
> --------------------------------------------------------------------------
>
> Key: MAPREDUCE-2687
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2687.patch, MAREDUCE-2687.patch
>
>
> Apps of non superuser fail to succeed in both secure and non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2687) Non superusers unable to launch
apps in both secure and non-secure cluster
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2687:
-------------------------------------
Attachment: MAPREDUCE-2687.patch
With this updated patch I can run applications as other users than the mapred user.
OTOH, there is a different bug which causes apps from other users to fail due to an error in LTC:
{noformat}
2011-09-06 02:29:07,379 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception from container-launch :
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
at org.apache.hadoop.util.Shell.run(Shell.java:188)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:171)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:191)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:61)
{noformat}
Still tracking this down... I wonder if this causes permission issues.
I've committed HADOOP-7580 which should help ease debugging.
> Non superusers unable to launch apps in both secure and non-secure cluster
> --------------------------------------------------------------------------
>
> Key: MAPREDUCE-2687
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2687.patch, MAREDUCE-2687.patch
>
>
> Apps of non superuser fail to succeed in both secure and non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2687) Non superusers unable to launch
apps in a non-secure cluster
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar reassigned MAPREDUCE-2687:
----------------------------------------
Assignee: Mahadev konar
> Non superusers unable to launch apps in a non-secure cluster
> ------------------------------------------------------------
>
> Key: MAPREDUCE-2687
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Fix For: 0.23.0
>
>
> Apps of non superuser fail to succeed in a non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2687) Non superusers unable to launch
apps in both secure and non-secure cluster
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-2687:
-----------------------------------------------
Description: Apps of non superuser fail to succeed in both secure and non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails. (was: Apps of non superuser fail to succeed in a non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails.)
Priority: Blocker (was: Major)
Summary: Non superusers unable to launch apps in both secure and non-secure cluster (was: Non superusers unable to launch apps in a non-secure cluster)
[~karams] found another bug but in secure mode which I tracked down to be just another manifestation of this bug.
{code}
2011-08-30 15:22:55,997 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1314717753121_0001_01_000001 transitioned from LOCALIZING to LOCALIZED
2011-08-30 15:22:56,007 WARN org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext: Disk Error Exception: org.apache.hadoop.util.DiskChecker$DiskErrorException: directory is not writable: /tmp/hadoop/var/log/mapred/application_1314717753121_0001
at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:99)
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.createPath(LocalDirAllocator.java:260)
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:327)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:132)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:101)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:61)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
.... (4 times)
2011-08-30 15:22:56,009 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Failed to launch container
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find any valid local directory for application_1314717753121_0001/container_1314717753121_0001_01_000001
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:350)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:132)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:101)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:61)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
2011-08-30 15:22:56,010 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Processing container_1314717753121_0001_01_000001 of type CONTAINER_EXITED_WITH_FAILURE
{code}
Updating title, description and upping it to a blocker.
> Non superusers unable to launch apps in both secure and non-secure cluster
> --------------------------------------------------------------------------
>
> Key: MAPREDUCE-2687
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAREDUCE-2687.patch
>
>
> Apps of non superuser fail to succeed in both secure and non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2687) Non superusers unable to launch
apps in both secure and non-secure cluster
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098399#comment-13098399 ]
Hudson commented on MAPREDUCE-2687:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #838 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/838/])
MAPREDUCE-2687. Fix NodeManager to use the right version of LocalDirAllocator.getLocalPathToWrite. Contributed by mahadev & acmurthy.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165890
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
> Non superusers unable to launch apps in both secure and non-secure cluster
> --------------------------------------------------------------------------
>
> Key: MAPREDUCE-2687
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2687.patch, MAREDUCE-2687.patch
>
>
> Apps of non superuser fail to succeed in both secure and non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2687) Non superusers unable to launch
apps in both secure and non-secure cluster
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098895#comment-13098895 ]
Hudson commented on MAPREDUCE-2687:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #786 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/786/])
MAPREDUCE-2687. Fix NodeManager to use the right version of LocalDirAllocator.getLocalPathToWrite. Contributed by mahadev & acmurthy.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165890
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
> Non superusers unable to launch apps in both secure and non-secure cluster
> --------------------------------------------------------------------------
>
> Key: MAPREDUCE-2687
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2687.patch, MAREDUCE-2687.patch
>
>
> Apps of non superuser fail to succeed in both secure and non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2687) Non superusers unable to launch
apps in both secure and non-secure cluster
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2687:
-------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks Mahadev.
> Non superusers unable to launch apps in both secure and non-secure cluster
> --------------------------------------------------------------------------
>
> Key: MAPREDUCE-2687
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2687.patch, MAREDUCE-2687.patch
>
>
> Apps of non superuser fail to succeed in both secure and non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2687) Non superusers unable to launch
apps in both secure and non-secure cluster
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13097716#comment-13097716 ]
Arun C Murthy commented on MAPREDUCE-2687:
------------------------------------------
2011-09-06 02:29:07,378 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code from container is : 1
> Non superusers unable to launch apps in both secure and non-secure cluster
> --------------------------------------------------------------------------
>
> Key: MAPREDUCE-2687
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2687.patch, MAREDUCE-2687.patch
>
>
> Apps of non superuser fail to succeed in both secure and non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2687) Non superusers unable to launch
apps in a non-secure cluster
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-2687:
-------------------------------------
Attachment: MAREDUCE-2687.patch
Minor fix. This depends upon HADOOP-7580. This patch was there before the merge but due to some merging problems got lost.
> Non superusers unable to launch apps in a non-secure cluster
> ------------------------------------------------------------
>
> Key: MAPREDUCE-2687
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Fix For: 0.23.0
>
> Attachments: MAREDUCE-2687.patch
>
>
> Apps of non superuser fail to succeed in a non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2687) Non superusers unable to launch
apps in both secure and non-secure cluster
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098401#comment-13098401 ]
Hudson commented on MAPREDUCE-2687:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #915 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/915/])
MAPREDUCE-2687. Fix NodeManager to use the right version of LocalDirAllocator.getLocalPathToWrite. Contributed by mahadev & acmurthy.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165890
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
> Non superusers unable to launch apps in both secure and non-secure cluster
> --------------------------------------------------------------------------
>
> Key: MAPREDUCE-2687
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2687.patch, MAREDUCE-2687.patch
>
>
> Apps of non superuser fail to succeed in both secure and non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2687) Non superusers unable to launch
apps in a non-secure cluster
Posted by "Ramya Sunil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066085#comment-13066085 ]
Ramya Sunil commented on MAPREDUCE-2687:
----------------------------------------
The job is submitted as user ramya and mr_superuser is the owner of the mrv2 cluster. Below is the snippet of the logs:
{noformat}
INFO impl.JobImpl: DEBUG --- startJobs: parent=/user/mr_superuser/.staging child=jobID
INFO impl.JobImpl: Adding job token for jobID to jobTokenSecretManager
INFO impl.JobImpl: Writing back the job-token file on the remote file system:hdfs://<namenodehost>:<port>/user/mr_superuser/.staging/jobID/appTokens
WARN impl.JobImpl: Job init failed
org.apache.hadoop.yarn.YarnException: java.io.FileNotFoundException: File does not exist: hdfs://<namenodehost>:<port>/user/mr_superuser/.staging/jobID/job.splitmetainfo
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.createSplits(JobImpl.java:1071)
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transition(JobImpl.java:800)
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transition(JobImpl.java:771)
at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:376)
at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:294)
at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:39)
at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:439)
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:669)
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:116)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:508)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.init(MRAppMaster.java:240)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:558)
Caused by: java.io.FileNotFoundException: File does not exist: hdfs://<namenodehost>:<port>/user/mr_superuser/.staging/jobID/job.splitmetainfo
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:739)
at org.apache.hadoop.mapreduce.split.SplitMetaInfoReader.readSplitMetaInfo(SplitMetaInfoReader.java:50)
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.createSplits(JobImpl.java:1066)
... 11 more
11/07/14 18:32:17 ERROR app.MRAppMaster: Caught throwable. Exiting:
java.lang.NullPointerException
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.abortJob(JobImpl.java:1118)
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.access$2900(JobImpl.java:116)
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transition(JobImpl.java:947)
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transition(JobImpl.java:771)
at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:376)
at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:294)
at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:39)
at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:439)
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:669)
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:116)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:508)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.init(MRAppMaster.java:240)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:558)
{noformat}
> Non superusers unable to launch apps in a non-secure cluster
> ------------------------------------------------------------
>
> Key: MAPREDUCE-2687
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Fix For: 0.23.0
>
>
> Apps of non superuser fail to succeed in a non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2687) Non superusers unable to launch
apps in both secure and non-secure cluster
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098402#comment-13098402 ]
Hudson commented on MAPREDUCE-2687:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #849 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/849/])
MAPREDUCE-2687. Fix NodeManager to use the right version of LocalDirAllocator.getLocalPathToWrite. Contributed by mahadev & acmurthy.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165890
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
> Non superusers unable to launch apps in both secure and non-secure cluster
> --------------------------------------------------------------------------
>
> Key: MAPREDUCE-2687
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2687.patch, MAREDUCE-2687.patch
>
>
> Apps of non superuser fail to succeed in both secure and non-secure environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) is able to launch apps successfully. However, when a normal user submits a job, the job fails.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira