Re: debugging class path issues with containers.

[Adam Kawa <ka...@gmail.com>]

You might need to set *yarn.application.classpath* in yarn-site.xml

*<property>*
* <name>yarn.application.classpath</name>*
*
<value>/etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*</value>*
* <description>Classpath for typical applications.</description>*
*</property>*

In <value></value> you can use configuration variables (e.g.
$HADOOP_COMMON_HOME or $HADOOP_YARN_HOME), assuming that they are set in
hadoop-env.sh (or other scripts).



2014-05-24 1:48 GMT+02:00 Kevin Burton <bu...@spinn3r.com>:

> What's the best way to debug yarn container issues?
>
> I was going to try to tweak the script but it gets deleted after the job
> fails.
>
> Looks like I'm having an issue with the classpath.. I'm getting a basic
> hadoop NCDFE on startup so I think it just has a broken class path.
>
> but of course I need to fix that…
>
> .. I'd like to just print the class path or maybe run a command before the
> container is launched to check the permissions, etc.
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/service/CompositeService
>         at java.lang.ClassLoader.defineClass1(Native Method)
>         at java.lang.ClassLoader.defineClass(ClassLoader.java:792)
>         at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>         at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>         at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>         at
> sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.service.CompositeService
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>         ... 13 more
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> Skype: *burtonator*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
> War is peace. Freedom is slavery. Ignorance is strength. Corporations are
> people.
>
>