You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Kristoffer Sjögren <st...@gmail.com> on 2016/01/22 14:52:07 UTC
Yarn app: Cannot run "java -jar" container
Hi
I'm trying to run a yarn 2.7.1 application using a basic boilerplate
[1]. But I have trouble running the container with an executable jar
file using the following args list.
List<String> arg = Collections.singletonList(
"/usr/jdk64/jdk1.8.0_40/bin/java -jar app-1.0.0-SNAPSHOT.jar" +
" 1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout" +
" 2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
);
I'm not really sure why it doesn't work because the diagnostics
message on tells me exit code 1 (see below). Neither the stdout or
stderr files appear, and I have tried to pipe them to /tmp/stdout etc.
How do I debug this error? Is the diagnostics message the only way? I
have tried a gazillion different combinations of running the
container, and the process is very time consuming and frustrating when
there isn't any information to debug.
Any tips or pointers on how to trace this error down?
Cheers,
-Kristoffer
[1] https://github.com/hortonworks/simple-yarn-app/tree/master/src/main/java/com/hortonworks/simpleyarnapp
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
at org.apache.hadoop.util.Shell.run(Shell.java:487)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: Yarn app: Cannot run "java -jar" container
Posted by Kristoffer Sjögren <st...@gmail.com>.
I got the application work now, thanks Hitesh!
But why isn't the container stdout/stderr visible from the
ApplicationMaster UI. Is manual ssh the only way to look at them?
On Fri, Jan 22, 2016 at 9:19 PM, Kristoffer Sjögren <st...@gmail.com> wrote:
> Thanks for the tip Hitesh - that's really helpful.
>
> On Fri, Jan 22, 2016 at 7:47 PM, Hitesh Shah <hi...@apache.org> wrote:
>> Ideally, the “yarn logs -application” command should give you the logs for the container in question and the stdout/stderr there usually gives you a good indication on what is going wrong.
>>
>> Second more complex option:
>> - Set yarn.nodemanager.delete.debug-delay-sec to say 1200 or a large enough value. Restart all NMs.
>> - Run your application.
>> - Find the node on which your container failed.
>> - Search through the yarn nodemanager local-dirs to find the launch_container.sh for your container.
>> - Look at its contents to see if things are being setup correctly. Run it manually to debug.
>>
>> — Hitesh
>>
>> On Jan 22, 2016, at 5:52 AM, Kristoffer Sjögren <st...@gmail.com> wrote:
>>
>>> Hi
>>>
>>> I'm trying to run a yarn 2.7.1 application using a basic boilerplate
>>> [1]. But I have trouble running the container with an executable jar
>>> file using the following args list.
>>>
>>> List<String> arg = Collections.singletonList(
>>> "/usr/jdk64/jdk1.8.0_40/bin/java -jar app-1.0.0-SNAPSHOT.jar" +
>>> " 1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout" +
>>> " 2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
>>> );
>>>
>>> I'm not really sure why it doesn't work because the diagnostics
>>> message on tells me exit code 1 (see below). Neither the stdout or
>>> stderr files appear, and I have tried to pipe them to /tmp/stdout etc.
>>>
>>> How do I debug this error? Is the diagnostics message the only way? I
>>> have tried a gazillion different combinations of running the
>>> container, and the process is very time consuming and frustrating when
>>> there isn't any information to debug.
>>>
>>> Any tips or pointers on how to trace this error down?
>>>
>>> Cheers,
>>> -Kristoffer
>>>
>>> [1] https://github.com/hortonworks/simple-yarn-app/tree/master/src/main/java/com/hortonworks/simpleyarnapp
>>>
>>>
>>> Stack trace: ExitCodeException exitCode=1:
>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
>>> at org.apache.hadoop.util.Shell.run(Shell.java:487)
>>> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
>>> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
>>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
>>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:745)
>>>
>>>
>>> Container exited with a non-zero exit code 1
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: user-help@hadoop.apache.org
>>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: Yarn app: Cannot run "java -jar" container
Posted by Kristoffer Sjögren <st...@gmail.com>.
I got the application work now, thanks Hitesh!
But why isn't the container stdout/stderr visible from the
ApplicationMaster UI. Is manual ssh the only way to look at them?
On Fri, Jan 22, 2016 at 9:19 PM, Kristoffer Sjögren <st...@gmail.com> wrote:
> Thanks for the tip Hitesh - that's really helpful.
>
> On Fri, Jan 22, 2016 at 7:47 PM, Hitesh Shah <hi...@apache.org> wrote:
>> Ideally, the “yarn logs -application” command should give you the logs for the container in question and the stdout/stderr there usually gives you a good indication on what is going wrong.
>>
>> Second more complex option:
>> - Set yarn.nodemanager.delete.debug-delay-sec to say 1200 or a large enough value. Restart all NMs.
>> - Run your application.
>> - Find the node on which your container failed.
>> - Search through the yarn nodemanager local-dirs to find the launch_container.sh for your container.
>> - Look at its contents to see if things are being setup correctly. Run it manually to debug.
>>
>> — Hitesh
>>
>> On Jan 22, 2016, at 5:52 AM, Kristoffer Sjögren <st...@gmail.com> wrote:
>>
>>> Hi
>>>
>>> I'm trying to run a yarn 2.7.1 application using a basic boilerplate
>>> [1]. But I have trouble running the container with an executable jar
>>> file using the following args list.
>>>
>>> List<String> arg = Collections.singletonList(
>>> "/usr/jdk64/jdk1.8.0_40/bin/java -jar app-1.0.0-SNAPSHOT.jar" +
>>> " 1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout" +
>>> " 2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
>>> );
>>>
>>> I'm not really sure why it doesn't work because the diagnostics
>>> message on tells me exit code 1 (see below). Neither the stdout or
>>> stderr files appear, and I have tried to pipe them to /tmp/stdout etc.
>>>
>>> How do I debug this error? Is the diagnostics message the only way? I
>>> have tried a gazillion different combinations of running the
>>> container, and the process is very time consuming and frustrating when
>>> there isn't any information to debug.
>>>
>>> Any tips or pointers on how to trace this error down?
>>>
>>> Cheers,
>>> -Kristoffer
>>>
>>> [1] https://github.com/hortonworks/simple-yarn-app/tree/master/src/main/java/com/hortonworks/simpleyarnapp
>>>
>>>
>>> Stack trace: ExitCodeException exitCode=1:
>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
>>> at org.apache.hadoop.util.Shell.run(Shell.java:487)
>>> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
>>> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
>>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
>>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:745)
>>>
>>>
>>> Container exited with a non-zero exit code 1
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: user-help@hadoop.apache.org
>>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: Yarn app: Cannot run "java -jar" container
Posted by Kristoffer Sjögren <st...@gmail.com>.
I got the application work now, thanks Hitesh!
But why isn't the container stdout/stderr visible from the
ApplicationMaster UI. Is manual ssh the only way to look at them?
On Fri, Jan 22, 2016 at 9:19 PM, Kristoffer Sjögren <st...@gmail.com> wrote:
> Thanks for the tip Hitesh - that's really helpful.
>
> On Fri, Jan 22, 2016 at 7:47 PM, Hitesh Shah <hi...@apache.org> wrote:
>> Ideally, the “yarn logs -application” command should give you the logs for the container in question and the stdout/stderr there usually gives you a good indication on what is going wrong.
>>
>> Second more complex option:
>> - Set yarn.nodemanager.delete.debug-delay-sec to say 1200 or a large enough value. Restart all NMs.
>> - Run your application.
>> - Find the node on which your container failed.
>> - Search through the yarn nodemanager local-dirs to find the launch_container.sh for your container.
>> - Look at its contents to see if things are being setup correctly. Run it manually to debug.
>>
>> — Hitesh
>>
>> On Jan 22, 2016, at 5:52 AM, Kristoffer Sjögren <st...@gmail.com> wrote:
>>
>>> Hi
>>>
>>> I'm trying to run a yarn 2.7.1 application using a basic boilerplate
>>> [1]. But I have trouble running the container with an executable jar
>>> file using the following args list.
>>>
>>> List<String> arg = Collections.singletonList(
>>> "/usr/jdk64/jdk1.8.0_40/bin/java -jar app-1.0.0-SNAPSHOT.jar" +
>>> " 1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout" +
>>> " 2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
>>> );
>>>
>>> I'm not really sure why it doesn't work because the diagnostics
>>> message on tells me exit code 1 (see below). Neither the stdout or
>>> stderr files appear, and I have tried to pipe them to /tmp/stdout etc.
>>>
>>> How do I debug this error? Is the diagnostics message the only way? I
>>> have tried a gazillion different combinations of running the
>>> container, and the process is very time consuming and frustrating when
>>> there isn't any information to debug.
>>>
>>> Any tips or pointers on how to trace this error down?
>>>
>>> Cheers,
>>> -Kristoffer
>>>
>>> [1] https://github.com/hortonworks/simple-yarn-app/tree/master/src/main/java/com/hortonworks/simpleyarnapp
>>>
>>>
>>> Stack trace: ExitCodeException exitCode=1:
>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
>>> at org.apache.hadoop.util.Shell.run(Shell.java:487)
>>> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
>>> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
>>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
>>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:745)
>>>
>>>
>>> Container exited with a non-zero exit code 1
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: user-help@hadoop.apache.org
>>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: Yarn app: Cannot run "java -jar" container
Posted by Kristoffer Sjögren <st...@gmail.com>.
I got the application work now, thanks Hitesh!
But why isn't the container stdout/stderr visible from the
ApplicationMaster UI. Is manual ssh the only way to look at them?
On Fri, Jan 22, 2016 at 9:19 PM, Kristoffer Sjögren <st...@gmail.com> wrote:
> Thanks for the tip Hitesh - that's really helpful.
>
> On Fri, Jan 22, 2016 at 7:47 PM, Hitesh Shah <hi...@apache.org> wrote:
>> Ideally, the “yarn logs -application” command should give you the logs for the container in question and the stdout/stderr there usually gives you a good indication on what is going wrong.
>>
>> Second more complex option:
>> - Set yarn.nodemanager.delete.debug-delay-sec to say 1200 or a large enough value. Restart all NMs.
>> - Run your application.
>> - Find the node on which your container failed.
>> - Search through the yarn nodemanager local-dirs to find the launch_container.sh for your container.
>> - Look at its contents to see if things are being setup correctly. Run it manually to debug.
>>
>> — Hitesh
>>
>> On Jan 22, 2016, at 5:52 AM, Kristoffer Sjögren <st...@gmail.com> wrote:
>>
>>> Hi
>>>
>>> I'm trying to run a yarn 2.7.1 application using a basic boilerplate
>>> [1]. But I have trouble running the container with an executable jar
>>> file using the following args list.
>>>
>>> List<String> arg = Collections.singletonList(
>>> "/usr/jdk64/jdk1.8.0_40/bin/java -jar app-1.0.0-SNAPSHOT.jar" +
>>> " 1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout" +
>>> " 2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
>>> );
>>>
>>> I'm not really sure why it doesn't work because the diagnostics
>>> message on tells me exit code 1 (see below). Neither the stdout or
>>> stderr files appear, and I have tried to pipe them to /tmp/stdout etc.
>>>
>>> How do I debug this error? Is the diagnostics message the only way? I
>>> have tried a gazillion different combinations of running the
>>> container, and the process is very time consuming and frustrating when
>>> there isn't any information to debug.
>>>
>>> Any tips or pointers on how to trace this error down?
>>>
>>> Cheers,
>>> -Kristoffer
>>>
>>> [1] https://github.com/hortonworks/simple-yarn-app/tree/master/src/main/java/com/hortonworks/simpleyarnapp
>>>
>>>
>>> Stack trace: ExitCodeException exitCode=1:
>>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
>>> at org.apache.hadoop.util.Shell.run(Shell.java:487)
>>> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
>>> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
>>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
>>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:745)
>>>
>>>
>>> Container exited with a non-zero exit code 1
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: user-help@hadoop.apache.org
>>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: Yarn app: Cannot run "java -jar" container
Posted by Kristoffer Sjögren <st...@gmail.com>.
Thanks for the tip Hitesh - that's really helpful.
On Fri, Jan 22, 2016 at 7:47 PM, Hitesh Shah <hi...@apache.org> wrote:
> Ideally, the “yarn logs -application” command should give you the logs for the container in question and the stdout/stderr there usually gives you a good indication on what is going wrong.
>
> Second more complex option:
> - Set yarn.nodemanager.delete.debug-delay-sec to say 1200 or a large enough value. Restart all NMs.
> - Run your application.
> - Find the node on which your container failed.
> - Search through the yarn nodemanager local-dirs to find the launch_container.sh for your container.
> - Look at its contents to see if things are being setup correctly. Run it manually to debug.
>
> — Hitesh
>
> On Jan 22, 2016, at 5:52 AM, Kristoffer Sjögren <st...@gmail.com> wrote:
>
>> Hi
>>
>> I'm trying to run a yarn 2.7.1 application using a basic boilerplate
>> [1]. But I have trouble running the container with an executable jar
>> file using the following args list.
>>
>> List<String> arg = Collections.singletonList(
>> "/usr/jdk64/jdk1.8.0_40/bin/java -jar app-1.0.0-SNAPSHOT.jar" +
>> " 1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout" +
>> " 2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
>> );
>>
>> I'm not really sure why it doesn't work because the diagnostics
>> message on tells me exit code 1 (see below). Neither the stdout or
>> stderr files appear, and I have tried to pipe them to /tmp/stdout etc.
>>
>> How do I debug this error? Is the diagnostics message the only way? I
>> have tried a gazillion different combinations of running the
>> container, and the process is very time consuming and frustrating when
>> there isn't any information to debug.
>>
>> Any tips or pointers on how to trace this error down?
>>
>> Cheers,
>> -Kristoffer
>>
>> [1] https://github.com/hortonworks/simple-yarn-app/tree/master/src/main/java/com/hortonworks/simpleyarnapp
>>
>>
>> Stack trace: ExitCodeException exitCode=1:
>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
>> at org.apache.hadoop.util.Shell.run(Shell.java:487)
>> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
>> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>>
>>
>> Container exited with a non-zero exit code 1
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: user-help@hadoop.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: Yarn app: Cannot run "java -jar" container
Posted by Kristoffer Sjögren <st...@gmail.com>.
Thanks for the tip Hitesh - that's really helpful.
On Fri, Jan 22, 2016 at 7:47 PM, Hitesh Shah <hi...@apache.org> wrote:
> Ideally, the “yarn logs -application” command should give you the logs for the container in question and the stdout/stderr there usually gives you a good indication on what is going wrong.
>
> Second more complex option:
> - Set yarn.nodemanager.delete.debug-delay-sec to say 1200 or a large enough value. Restart all NMs.
> - Run your application.
> - Find the node on which your container failed.
> - Search through the yarn nodemanager local-dirs to find the launch_container.sh for your container.
> - Look at its contents to see if things are being setup correctly. Run it manually to debug.
>
> — Hitesh
>
> On Jan 22, 2016, at 5:52 AM, Kristoffer Sjögren <st...@gmail.com> wrote:
>
>> Hi
>>
>> I'm trying to run a yarn 2.7.1 application using a basic boilerplate
>> [1]. But I have trouble running the container with an executable jar
>> file using the following args list.
>>
>> List<String> arg = Collections.singletonList(
>> "/usr/jdk64/jdk1.8.0_40/bin/java -jar app-1.0.0-SNAPSHOT.jar" +
>> " 1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout" +
>> " 2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
>> );
>>
>> I'm not really sure why it doesn't work because the diagnostics
>> message on tells me exit code 1 (see below). Neither the stdout or
>> stderr files appear, and I have tried to pipe them to /tmp/stdout etc.
>>
>> How do I debug this error? Is the diagnostics message the only way? I
>> have tried a gazillion different combinations of running the
>> container, and the process is very time consuming and frustrating when
>> there isn't any information to debug.
>>
>> Any tips or pointers on how to trace this error down?
>>
>> Cheers,
>> -Kristoffer
>>
>> [1] https://github.com/hortonworks/simple-yarn-app/tree/master/src/main/java/com/hortonworks/simpleyarnapp
>>
>>
>> Stack trace: ExitCodeException exitCode=1:
>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
>> at org.apache.hadoop.util.Shell.run(Shell.java:487)
>> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
>> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>>
>>
>> Container exited with a non-zero exit code 1
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: user-help@hadoop.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: Yarn app: Cannot run "java -jar" container
Posted by Kristoffer Sjögren <st...@gmail.com>.
Thanks for the tip Hitesh - that's really helpful.
On Fri, Jan 22, 2016 at 7:47 PM, Hitesh Shah <hi...@apache.org> wrote:
> Ideally, the “yarn logs -application” command should give you the logs for the container in question and the stdout/stderr there usually gives you a good indication on what is going wrong.
>
> Second more complex option:
> - Set yarn.nodemanager.delete.debug-delay-sec to say 1200 or a large enough value. Restart all NMs.
> - Run your application.
> - Find the node on which your container failed.
> - Search through the yarn nodemanager local-dirs to find the launch_container.sh for your container.
> - Look at its contents to see if things are being setup correctly. Run it manually to debug.
>
> — Hitesh
>
> On Jan 22, 2016, at 5:52 AM, Kristoffer Sjögren <st...@gmail.com> wrote:
>
>> Hi
>>
>> I'm trying to run a yarn 2.7.1 application using a basic boilerplate
>> [1]. But I have trouble running the container with an executable jar
>> file using the following args list.
>>
>> List<String> arg = Collections.singletonList(
>> "/usr/jdk64/jdk1.8.0_40/bin/java -jar app-1.0.0-SNAPSHOT.jar" +
>> " 1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout" +
>> " 2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
>> );
>>
>> I'm not really sure why it doesn't work because the diagnostics
>> message on tells me exit code 1 (see below). Neither the stdout or
>> stderr files appear, and I have tried to pipe them to /tmp/stdout etc.
>>
>> How do I debug this error? Is the diagnostics message the only way? I
>> have tried a gazillion different combinations of running the
>> container, and the process is very time consuming and frustrating when
>> there isn't any information to debug.
>>
>> Any tips or pointers on how to trace this error down?
>>
>> Cheers,
>> -Kristoffer
>>
>> [1] https://github.com/hortonworks/simple-yarn-app/tree/master/src/main/java/com/hortonworks/simpleyarnapp
>>
>>
>> Stack trace: ExitCodeException exitCode=1:
>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
>> at org.apache.hadoop.util.Shell.run(Shell.java:487)
>> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
>> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>>
>>
>> Container exited with a non-zero exit code 1
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: user-help@hadoop.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: Yarn app: Cannot run "java -jar" container
Posted by Kristoffer Sjögren <st...@gmail.com>.
Thanks for the tip Hitesh - that's really helpful.
On Fri, Jan 22, 2016 at 7:47 PM, Hitesh Shah <hi...@apache.org> wrote:
> Ideally, the “yarn logs -application” command should give you the logs for the container in question and the stdout/stderr there usually gives you a good indication on what is going wrong.
>
> Second more complex option:
> - Set yarn.nodemanager.delete.debug-delay-sec to say 1200 or a large enough value. Restart all NMs.
> - Run your application.
> - Find the node on which your container failed.
> - Search through the yarn nodemanager local-dirs to find the launch_container.sh for your container.
> - Look at its contents to see if things are being setup correctly. Run it manually to debug.
>
> — Hitesh
>
> On Jan 22, 2016, at 5:52 AM, Kristoffer Sjögren <st...@gmail.com> wrote:
>
>> Hi
>>
>> I'm trying to run a yarn 2.7.1 application using a basic boilerplate
>> [1]. But I have trouble running the container with an executable jar
>> file using the following args list.
>>
>> List<String> arg = Collections.singletonList(
>> "/usr/jdk64/jdk1.8.0_40/bin/java -jar app-1.0.0-SNAPSHOT.jar" +
>> " 1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout" +
>> " 2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
>> );
>>
>> I'm not really sure why it doesn't work because the diagnostics
>> message on tells me exit code 1 (see below). Neither the stdout or
>> stderr files appear, and I have tried to pipe them to /tmp/stdout etc.
>>
>> How do I debug this error? Is the diagnostics message the only way? I
>> have tried a gazillion different combinations of running the
>> container, and the process is very time consuming and frustrating when
>> there isn't any information to debug.
>>
>> Any tips or pointers on how to trace this error down?
>>
>> Cheers,
>> -Kristoffer
>>
>> [1] https://github.com/hortonworks/simple-yarn-app/tree/master/src/main/java/com/hortonworks/simpleyarnapp
>>
>>
>> Stack trace: ExitCodeException exitCode=1:
>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
>> at org.apache.hadoop.util.Shell.run(Shell.java:487)
>> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
>> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
>> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>>
>>
>> Container exited with a non-zero exit code 1
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: user-help@hadoop.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: Yarn app: Cannot run "java -jar" container
Posted by Hitesh Shah <hi...@apache.org>.
Ideally, the “yarn logs -application” command should give you the logs for the container in question and the stdout/stderr there usually gives you a good indication on what is going wrong.
Second more complex option:
- Set yarn.nodemanager.delete.debug-delay-sec to say 1200 or a large enough value. Restart all NMs.
- Run your application.
- Find the node on which your container failed.
- Search through the yarn nodemanager local-dirs to find the launch_container.sh for your container.
- Look at its contents to see if things are being setup correctly. Run it manually to debug.
— Hitesh
On Jan 22, 2016, at 5:52 AM, Kristoffer Sjögren <st...@gmail.com> wrote:
> Hi
>
> I'm trying to run a yarn 2.7.1 application using a basic boilerplate
> [1]. But I have trouble running the container with an executable jar
> file using the following args list.
>
> List<String> arg = Collections.singletonList(
> "/usr/jdk64/jdk1.8.0_40/bin/java -jar app-1.0.0-SNAPSHOT.jar" +
> " 1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout" +
> " 2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
> );
>
> I'm not really sure why it doesn't work because the diagnostics
> message on tells me exit code 1 (see below). Neither the stdout or
> stderr files appear, and I have tried to pipe them to /tmp/stdout etc.
>
> How do I debug this error? Is the diagnostics message the only way? I
> have tried a gazillion different combinations of running the
> container, and the process is very time consuming and frustrating when
> there isn't any information to debug.
>
> Any tips or pointers on how to trace this error down?
>
> Cheers,
> -Kristoffer
>
> [1] https://github.com/hortonworks/simple-yarn-app/tree/master/src/main/java/com/hortonworks/simpleyarnapp
>
>
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
> at org.apache.hadoop.util.Shell.run(Shell.java:487)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
>
> Container exited with a non-zero exit code 1
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: user-help@hadoop.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: Yarn app: Cannot run "java -jar" container
Posted by Hitesh Shah <hi...@apache.org>.
Ideally, the “yarn logs -application” command should give you the logs for the container in question and the stdout/stderr there usually gives you a good indication on what is going wrong.
Second more complex option:
- Set yarn.nodemanager.delete.debug-delay-sec to say 1200 or a large enough value. Restart all NMs.
- Run your application.
- Find the node on which your container failed.
- Search through the yarn nodemanager local-dirs to find the launch_container.sh for your container.
- Look at its contents to see if things are being setup correctly. Run it manually to debug.
— Hitesh
On Jan 22, 2016, at 5:52 AM, Kristoffer Sjögren <st...@gmail.com> wrote:
> Hi
>
> I'm trying to run a yarn 2.7.1 application using a basic boilerplate
> [1]. But I have trouble running the container with an executable jar
> file using the following args list.
>
> List<String> arg = Collections.singletonList(
> "/usr/jdk64/jdk1.8.0_40/bin/java -jar app-1.0.0-SNAPSHOT.jar" +
> " 1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout" +
> " 2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
> );
>
> I'm not really sure why it doesn't work because the diagnostics
> message on tells me exit code 1 (see below). Neither the stdout or
> stderr files appear, and I have tried to pipe them to /tmp/stdout etc.
>
> How do I debug this error? Is the diagnostics message the only way? I
> have tried a gazillion different combinations of running the
> container, and the process is very time consuming and frustrating when
> there isn't any information to debug.
>
> Any tips or pointers on how to trace this error down?
>
> Cheers,
> -Kristoffer
>
> [1] https://github.com/hortonworks/simple-yarn-app/tree/master/src/main/java/com/hortonworks/simpleyarnapp
>
>
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
> at org.apache.hadoop.util.Shell.run(Shell.java:487)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
>
> Container exited with a non-zero exit code 1
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: user-help@hadoop.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: Yarn app: Cannot run "java -jar" container
Posted by Hitesh Shah <hi...@apache.org>.
Ideally, the “yarn logs -application” command should give you the logs for the container in question and the stdout/stderr there usually gives you a good indication on what is going wrong.
Second more complex option:
- Set yarn.nodemanager.delete.debug-delay-sec to say 1200 or a large enough value. Restart all NMs.
- Run your application.
- Find the node on which your container failed.
- Search through the yarn nodemanager local-dirs to find the launch_container.sh for your container.
- Look at its contents to see if things are being setup correctly. Run it manually to debug.
— Hitesh
On Jan 22, 2016, at 5:52 AM, Kristoffer Sjögren <st...@gmail.com> wrote:
> Hi
>
> I'm trying to run a yarn 2.7.1 application using a basic boilerplate
> [1]. But I have trouble running the container with an executable jar
> file using the following args list.
>
> List<String> arg = Collections.singletonList(
> "/usr/jdk64/jdk1.8.0_40/bin/java -jar app-1.0.0-SNAPSHOT.jar" +
> " 1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout" +
> " 2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
> );
>
> I'm not really sure why it doesn't work because the diagnostics
> message on tells me exit code 1 (see below). Neither the stdout or
> stderr files appear, and I have tried to pipe them to /tmp/stdout etc.
>
> How do I debug this error? Is the diagnostics message the only way? I
> have tried a gazillion different combinations of running the
> container, and the process is very time consuming and frustrating when
> there isn't any information to debug.
>
> Any tips or pointers on how to trace this error down?
>
> Cheers,
> -Kristoffer
>
> [1] https://github.com/hortonworks/simple-yarn-app/tree/master/src/main/java/com/hortonworks/simpleyarnapp
>
>
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
> at org.apache.hadoop.util.Shell.run(Shell.java:487)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
>
> Container exited with a non-zero exit code 1
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: user-help@hadoop.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
Re: Yarn app: Cannot run "java -jar" container
Posted by Hitesh Shah <hi...@apache.org>.
Ideally, the “yarn logs -application” command should give you the logs for the container in question and the stdout/stderr there usually gives you a good indication on what is going wrong.
Second more complex option:
- Set yarn.nodemanager.delete.debug-delay-sec to say 1200 or a large enough value. Restart all NMs.
- Run your application.
- Find the node on which your container failed.
- Search through the yarn nodemanager local-dirs to find the launch_container.sh for your container.
- Look at its contents to see if things are being setup correctly. Run it manually to debug.
— Hitesh
On Jan 22, 2016, at 5:52 AM, Kristoffer Sjögren <st...@gmail.com> wrote:
> Hi
>
> I'm trying to run a yarn 2.7.1 application using a basic boilerplate
> [1]. But I have trouble running the container with an executable jar
> file using the following args list.
>
> List<String> arg = Collections.singletonList(
> "/usr/jdk64/jdk1.8.0_40/bin/java -jar app-1.0.0-SNAPSHOT.jar" +
> " 1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout" +
> " 2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr"
> );
>
> I'm not really sure why it doesn't work because the diagnostics
> message on tells me exit code 1 (see below). Neither the stdout or
> stderr files appear, and I have tried to pipe them to /tmp/stdout etc.
>
> How do I debug this error? Is the diagnostics message the only way? I
> have tried a gazillion different combinations of running the
> container, and the process is very time consuming and frustrating when
> there isn't any information to debug.
>
> Any tips or pointers on how to trace this error down?
>
> Cheers,
> -Kristoffer
>
> [1] https://github.com/hortonworks/simple-yarn-app/tree/master/src/main/java/com/hortonworks/simpleyarnapp
>
>
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:576)
> at org.apache.hadoop.util.Shell.run(Shell.java:487)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753)
> at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
>
> Container exited with a non-zero exit code 1
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: user-help@hadoop.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org