You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Prabhu Joseph (JIRA)" <ji...@apache.org> on 2019/04/15 08:54:00 UTC

[jira] [Created] (YARN-9482) DistributedShell job with localization fails in unsecure cluster

Prabhu Joseph created YARN-9482:
-----------------------------------

             Summary: DistributedShell job with localization fails in unsecure cluster
                 Key: YARN-9482
                 URL: https://issues.apache.org/jira/browse/YARN-9482
             Project: Hadoop YARN
          Issue Type: Bug
          Components: distributed-shell
    Affects Versions: 3.3.0
            Reporter: Prabhu Joseph
            Assignee: Prabhu Joseph


DistributedShell job with localization fails in unsecure cluster. The client localizes the input files to home directory (job user) whereas the AM runs as yarn user reads from it's home directory.

*Command:*

{code}
yarn jar /HADOOP/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.2.0.jar -shell_command ls  -shell_args / -jar  /HADOOP/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.2.0.jar -localize_files /tmp/prabhu
{code}

{code}
Exception in thread "Thread-4" java.io.UncheckedIOException: Error during localization setup
	at org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster$LaunchContainerRunnable.lambda$run$0(ApplicationMaster.java:1495)
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
	at org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster$LaunchContainerRunnable.run(ApplicationMaster.java:1481)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.FileNotFoundException: File does not exist: hdfs://yarn-ats-1:8020/user/yarn/DistributedShell/application_1554817981283_0003/prabhu
	at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1586)
	at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1579)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1594)
	at org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster$LaunchContainerRunnable.lambda$run$0(ApplicationMaster.java:1487)
{code}	





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org