yarn/hadoop error

[Manuel Sopena Ballesteros <ma...@garvan.org.au>]

Dear hadoop community,


I am getting this error when I submit a job to yarn after redeploying our hadoop cluster.


Application application_1586362992524_0015 failed 2 times due to AM Container for appattempt_1586362992524_0015_000002 exited with exitCode: 13
Failing this attempt.Diagnostics: [2020-04-09 13:31:22.190]Exception from container-launch.
Container id: container_1586362992524_0015_02_000001
Exit code: 13
Exception message: Launch container failed
Shell output: main : command provided 1
main : run as user is mansop
main : requested yarn user is mansop
Getting exit code file...
Creating script paths...
Writing pid file...
Writing to tmp file /d0/hadoop/yarn/local/nmPrivate/application_1586362992524_0015/container_1586362992524_0015_02_000001/container_1586362992524_0015_02_000001.pid.tmp
Writing to cgroup task files...
Creating local dirs...
Launching container...
Getting exit code file...
Creating script paths...
[2020-04-09 13:31:22.198]Container exited with a non-zero exit code 13. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/d1/hadoop/yarn/local/filecache/10/spark2-hdp-yarn-archive.tar.gz/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
[2020-04-09 13:31:22.199]Container exited with a non-zero exit code 13. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/d1/hadoop/yarn/local/filecache/10/spark2-hdp-yarn-archive.tar.gz/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
For more detailed output, check the application tracking page: http://gl-hdp-ctrl03-mlx.mlx:8088/cluster/app/application_1586362992524_0015 Then click on links to logs of each attempt.
. Failing the application.


This log does not says much to me and  I am stuck because don't know what do to next.


could someone please help me and point me which log files I should see?


thank you

NOTICE
Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed.

Yarn in docker swarm

[murat migdisoglu <mu...@gmail.com>]

Hello all,

I've setup a yarn cluster in a docker swarm environment and so far it seems
to be working well.

My biggest problem is with browsing through the links of the yarn
application manager ui.
Nodemanager hostnames are docker generated hostnames and they are
meaningless outside of the swarm environment.
Is there a way to configure a different hostname for the nodemanager ui? I
want them to use the internal hostnames for communicating between them but
show the provided hostname on yarn ui.

Can I use  yarn.nodemanager.webapp.address for something like this? Is this
configuration the web address that reflects on the yarn ui?
(particularly, I want to use the external hostname of my ec2 instance which
I can get before I start the nodemanager through some magic web api's on
ec2)

Thank you very much
Kind Regards