You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by "Yu, Libo " <li...@citi.com> on 2013/07/10 22:00:01 UTC
NoClassDefFoundError:
org/apache/hadoop/yarn/service/CompositeService
Hi,
I tried to run the wordcount example with yarn. Here is the command line:
hadoop jar share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar wordcount /user/lyu/wordcount/input /user/lyu/wordcount/output
But I got this exception:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yar
n/service/CompositeService
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.service.CompositeService
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 12 more
Could not find the main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit
Here is conent of the CLASSPATH used by java:
/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*
Here is the console output:
13/07/09 20:43:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/07/09 20:43:07 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
13/07/09 20:43:07 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
13/07/09 20:43:08 INFO input.FileInputFormat: Total input paths to process : 0
13/07/09 20:43:08 INFO mapreduce.JobSubmitter: number of splits:0
13/07/09 20:43:08 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
13/07/09 20:43:08 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
13/07/09 20:43:09 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1373416798858_0001
13/07/09 20:43:09 INFO client.YarnClientImpl: Submitted application application_1373416798858_0001 to ResourceManager at host1.libo.com/192.168.10.1:8032
13/07/09 20:43:09 INFO mapreduce.Job: The url to track the job: http://host1.libo.com:8088/proxy/application_1373416798858_0001/
13/07/09 20:43:09 INFO mapreduce.Job: Running job: job_1373416798858_0001
13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 running in uber mode : false
13/07/09 20:43:12 INFO mapreduce.Job: map 0% reduce 0%
13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 failed with state FAILED due to: Application application_1373416798858_0001 failed 1 times due to AM Container for appattempt_1373416798858_0001_000001 exited with exitCode: 1 due to:
.Failing this attempt.. Failing the application.
13/07/09 20:43:12 INFO mapreduce.Job: Counters: 0
The paths for CompositeService and MRAppMaster are in the CLASSPATH and I checked the jar files and did
find the class files. Anybody knows why? Thanks.
Regards,
Libo
Re: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService
Posted by 闫昆 <ya...@gmail.com>.
wow! I have the same question and that is terrible,I speed three day
times.your ask question is also my resolve way
Thanks
2013/7/11 Devaraj k <de...@huawei.com>
> Hi Libo,****
>
> ** **
>
> MRAppMaster is not able to load the yarn related jar files. ****
>
> ** **
>
> Is this the classpath using by MRAppMaster or any other process? ****
>
> ** **
>
>
> “/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*”
> ****
>
> ** **
>
> If this is not the MRAppMaster process classpath, can you try to get the
> MRAppMaster java process classpath.****
>
> ** **
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> *From:* Yu, Libo [mailto:libo.yu@citi.com]
> *Sent:* 11 July 2013 01:30
> *To:* 'user@hadoop.apache.org'
> *Subject:* NoClassDefFoundError:
> org/apache/hadoop/yarn/service/CompositeService****
>
> ** **
>
> Hi,****
>
> ** **
>
> I tried to run the wordcount example with yarn. Here is the command line:*
> ***
>
> hadoop jar
> share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar
> wordcount /user/lyu/wordcount/input /user/lyu/wordcount/output****
>
> ** **
>
> But I got this exception:****
>
> ** **
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/yar
> n/service/CompositeService
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.service.CompositeService
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> ... 12 more
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit****
>
> ** **
>
> Here is conent of the CLASSPATH used by java:****
>
> ** **
>
>
> /opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*
> ****
>
> ** **
>
> Here is the console output:****
>
> ** **
>
> 13/07/09 20:43:07 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 13/07/09 20:43:07 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
> 13/07/09 20:43:07 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
> 13/07/09 20:43:08 INFO input.FileInputFormat: Total input paths to process
> : 0
> 13/07/09 20:43:08 INFO mapreduce.JobSubmitter: number of splits:0
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.value.class is
> deprecated. Instead, use mapreduce.job.output.value.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.combine.class is
> deprecated. Instead, use mapreduce.job.combine.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.job.name is deprecated.
> Instead, use mapreduce.job.name
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.reduce.class is
> deprecated. Instead, use mapreduce.job.reduce.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.input.dir is deprecated.
> Instead, use mapreduce.input.fileinputformat.inputdir
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.dir is
> deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.map.tasks is deprecated.
> Instead, use mapreduce.job.maps
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.key.class is
> deprecated. Instead, use mapreduce.job.output.key.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/07/09 20:43:09 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1373416798858_0001
> 13/07/09 20:43:09 INFO client.YarnClientImpl: Submitted application
> application_1373416798858_0001 to ResourceManager at
> host1.libo.com/192.168.10.1:8032
> 13/07/09 20:43:09 INFO mapreduce.Job: The url to track the job:
> http://host1.libo.com:8088/proxy/application_1373416798858_0001/
> 13/07/09 20:43:09 INFO mapreduce.Job: Running job: job_1373416798858_0001
> 13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 running
> in uber mode : false
> 13/07/09 20:43:12 INFO mapreduce.Job: map 0% reduce 0%
> 13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 failed
> with state FAILED due to: Application application_1373416798858_0001 failed
> 1 times due to AM Container for appattempt_1373416798858_0001_000001 exited
> with exitCode: 1 due to:
> .Failing this attempt.. Failing the application.
> 13/07/09 20:43:12 INFO mapreduce.Job: Counters: 0****
>
> ** **
>
> The paths for CompositeService and MRAppMaster are in the CLASSPATH and I
> checked the jar files and did****
>
> find the class files. Anybody knows why? Thanks.****
>
> ** **
>
> ** **
>
> Regards,****
>
> ** **
>
> Libo****
>
> ** **
>
Re: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService
Posted by 闫昆 <ya...@gmail.com>.
wow! I have the same question and that is terrible,I speed three day
times.your ask question is also my resolve way,thanks Devaraj k
2013/7/11 Devaraj k <de...@huawei.com>
> Hi Libo,****
>
> ** **
>
> MRAppMaster is not able to load the yarn related jar files. ****
>
> ** **
>
> Is this the classpath using by MRAppMaster or any other process? ****
>
> ** **
>
>
> “/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*”
> ****
>
> ** **
>
> If this is not the MRAppMaster process classpath, can you try to get the
> MRAppMaster java process classpath.****
>
> ** **
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> *From:* Yu, Libo [mailto:libo.yu@citi.com]
> *Sent:* 11 July 2013 01:30
> *To:* 'user@hadoop.apache.org'
> *Subject:* NoClassDefFoundError:
> org/apache/hadoop/yarn/service/CompositeService****
>
> ** **
>
> Hi,****
>
> ** **
>
> I tried to run the wordcount example with yarn. Here is the command line:*
> ***
>
> hadoop jar
> share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar
> wordcount /user/lyu/wordcount/input /user/lyu/wordcount/output****
>
> ** **
>
> But I got this exception:****
>
> ** **
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/yar
> n/service/CompositeService
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.service.CompositeService
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> ... 12 more
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit****
>
> ** **
>
> Here is conent of the CLASSPATH used by java:****
>
> ** **
>
>
> /opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*
> ****
>
> ** **
>
> Here is the console output:****
>
> ** **
>
> 13/07/09 20:43:07 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 13/07/09 20:43:07 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
> 13/07/09 20:43:07 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
> 13/07/09 20:43:08 INFO input.FileInputFormat: Total input paths to process
> : 0
> 13/07/09 20:43:08 INFO mapreduce.JobSubmitter: number of splits:0
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.value.class is
> deprecated. Instead, use mapreduce.job.output.value.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.combine.class is
> deprecated. Instead, use mapreduce.job.combine.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.job.name is deprecated.
> Instead, use mapreduce.job.name
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.reduce.class is
> deprecated. Instead, use mapreduce.job.reduce.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.input.dir is deprecated.
> Instead, use mapreduce.input.fileinputformat.inputdir
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.dir is
> deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.map.tasks is deprecated.
> Instead, use mapreduce.job.maps
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.key.class is
> deprecated. Instead, use mapreduce.job.output.key.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/07/09 20:43:09 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1373416798858_0001
> 13/07/09 20:43:09 INFO client.YarnClientImpl: Submitted application
> application_1373416798858_0001 to ResourceManager at
> host1.libo.com/192.168.10.1:8032
> 13/07/09 20:43:09 INFO mapreduce.Job: The url to track the job:
> http://host1.libo.com:8088/proxy/application_1373416798858_0001/
> 13/07/09 20:43:09 INFO mapreduce.Job: Running job: job_1373416798858_0001
> 13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 running
> in uber mode : false
> 13/07/09 20:43:12 INFO mapreduce.Job: map 0% reduce 0%
> 13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 failed
> with state FAILED due to: Application application_1373416798858_0001 failed
> 1 times due to AM Container for appattempt_1373416798858_0001_000001 exited
> with exitCode: 1 due to:
> .Failing this attempt.. Failing the application.
> 13/07/09 20:43:12 INFO mapreduce.Job: Counters: 0****
>
> ** **
>
> The paths for CompositeService and MRAppMaster are in the CLASSPATH and I
> checked the jar files and did****
>
> find the class files. Anybody knows why? Thanks.****
>
> ** **
>
> ** **
>
> Regards,****
>
> ** **
>
> Libo****
>
> ** **
>
Re: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService
Posted by 闫昆 <ya...@gmail.com>.
wow! I have the same question and that is terrible,I speed three day
times.your ask question is also my resolve way
Thanks
2013/7/11 Devaraj k <de...@huawei.com>
> Hi Libo,****
>
> ** **
>
> MRAppMaster is not able to load the yarn related jar files. ****
>
> ** **
>
> Is this the classpath using by MRAppMaster or any other process? ****
>
> ** **
>
>
> “/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*”
> ****
>
> ** **
>
> If this is not the MRAppMaster process classpath, can you try to get the
> MRAppMaster java process classpath.****
>
> ** **
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> *From:* Yu, Libo [mailto:libo.yu@citi.com]
> *Sent:* 11 July 2013 01:30
> *To:* 'user@hadoop.apache.org'
> *Subject:* NoClassDefFoundError:
> org/apache/hadoop/yarn/service/CompositeService****
>
> ** **
>
> Hi,****
>
> ** **
>
> I tried to run the wordcount example with yarn. Here is the command line:*
> ***
>
> hadoop jar
> share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar
> wordcount /user/lyu/wordcount/input /user/lyu/wordcount/output****
>
> ** **
>
> But I got this exception:****
>
> ** **
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/yar
> n/service/CompositeService
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.service.CompositeService
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> ... 12 more
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit****
>
> ** **
>
> Here is conent of the CLASSPATH used by java:****
>
> ** **
>
>
> /opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*
> ****
>
> ** **
>
> Here is the console output:****
>
> ** **
>
> 13/07/09 20:43:07 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 13/07/09 20:43:07 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
> 13/07/09 20:43:07 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
> 13/07/09 20:43:08 INFO input.FileInputFormat: Total input paths to process
> : 0
> 13/07/09 20:43:08 INFO mapreduce.JobSubmitter: number of splits:0
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.value.class is
> deprecated. Instead, use mapreduce.job.output.value.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.combine.class is
> deprecated. Instead, use mapreduce.job.combine.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.job.name is deprecated.
> Instead, use mapreduce.job.name
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.reduce.class is
> deprecated. Instead, use mapreduce.job.reduce.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.input.dir is deprecated.
> Instead, use mapreduce.input.fileinputformat.inputdir
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.dir is
> deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.map.tasks is deprecated.
> Instead, use mapreduce.job.maps
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.key.class is
> deprecated. Instead, use mapreduce.job.output.key.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/07/09 20:43:09 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1373416798858_0001
> 13/07/09 20:43:09 INFO client.YarnClientImpl: Submitted application
> application_1373416798858_0001 to ResourceManager at
> host1.libo.com/192.168.10.1:8032
> 13/07/09 20:43:09 INFO mapreduce.Job: The url to track the job:
> http://host1.libo.com:8088/proxy/application_1373416798858_0001/
> 13/07/09 20:43:09 INFO mapreduce.Job: Running job: job_1373416798858_0001
> 13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 running
> in uber mode : false
> 13/07/09 20:43:12 INFO mapreduce.Job: map 0% reduce 0%
> 13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 failed
> with state FAILED due to: Application application_1373416798858_0001 failed
> 1 times due to AM Container for appattempt_1373416798858_0001_000001 exited
> with exitCode: 1 due to:
> .Failing this attempt.. Failing the application.
> 13/07/09 20:43:12 INFO mapreduce.Job: Counters: 0****
>
> ** **
>
> The paths for CompositeService and MRAppMaster are in the CLASSPATH and I
> checked the jar files and did****
>
> find the class files. Anybody knows why? Thanks.****
>
> ** **
>
> ** **
>
> Regards,****
>
> ** **
>
> Libo****
>
> ** **
>
Re: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService
Posted by 闫昆 <ya...@gmail.com>.
wow! I have the same question and that is terrible,I speed three day
times.your ask question is also my resolve way
Thanks
2013/7/11 Devaraj k <de...@huawei.com>
> Hi Libo,****
>
> ** **
>
> MRAppMaster is not able to load the yarn related jar files. ****
>
> ** **
>
> Is this the classpath using by MRAppMaster or any other process? ****
>
> ** **
>
>
> “/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*”
> ****
>
> ** **
>
> If this is not the MRAppMaster process classpath, can you try to get the
> MRAppMaster java process classpath.****
>
> ** **
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> *From:* Yu, Libo [mailto:libo.yu@citi.com]
> *Sent:* 11 July 2013 01:30
> *To:* 'user@hadoop.apache.org'
> *Subject:* NoClassDefFoundError:
> org/apache/hadoop/yarn/service/CompositeService****
>
> ** **
>
> Hi,****
>
> ** **
>
> I tried to run the wordcount example with yarn. Here is the command line:*
> ***
>
> hadoop jar
> share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar
> wordcount /user/lyu/wordcount/input /user/lyu/wordcount/output****
>
> ** **
>
> But I got this exception:****
>
> ** **
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/yar
> n/service/CompositeService
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.service.CompositeService
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> ... 12 more
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit****
>
> ** **
>
> Here is conent of the CLASSPATH used by java:****
>
> ** **
>
>
> /opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*
> ****
>
> ** **
>
> Here is the console output:****
>
> ** **
>
> 13/07/09 20:43:07 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 13/07/09 20:43:07 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
> 13/07/09 20:43:07 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
> 13/07/09 20:43:08 INFO input.FileInputFormat: Total input paths to process
> : 0
> 13/07/09 20:43:08 INFO mapreduce.JobSubmitter: number of splits:0
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.value.class is
> deprecated. Instead, use mapreduce.job.output.value.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.combine.class is
> deprecated. Instead, use mapreduce.job.combine.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.job.name is deprecated.
> Instead, use mapreduce.job.name
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.reduce.class is
> deprecated. Instead, use mapreduce.job.reduce.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.input.dir is deprecated.
> Instead, use mapreduce.input.fileinputformat.inputdir
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.dir is
> deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.map.tasks is deprecated.
> Instead, use mapreduce.job.maps
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.key.class is
> deprecated. Instead, use mapreduce.job.output.key.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/07/09 20:43:09 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1373416798858_0001
> 13/07/09 20:43:09 INFO client.YarnClientImpl: Submitted application
> application_1373416798858_0001 to ResourceManager at
> host1.libo.com/192.168.10.1:8032
> 13/07/09 20:43:09 INFO mapreduce.Job: The url to track the job:
> http://host1.libo.com:8088/proxy/application_1373416798858_0001/
> 13/07/09 20:43:09 INFO mapreduce.Job: Running job: job_1373416798858_0001
> 13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 running
> in uber mode : false
> 13/07/09 20:43:12 INFO mapreduce.Job: map 0% reduce 0%
> 13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 failed
> with state FAILED due to: Application application_1373416798858_0001 failed
> 1 times due to AM Container for appattempt_1373416798858_0001_000001 exited
> with exitCode: 1 due to:
> .Failing this attempt.. Failing the application.
> 13/07/09 20:43:12 INFO mapreduce.Job: Counters: 0****
>
> ** **
>
> The paths for CompositeService and MRAppMaster are in the CLASSPATH and I
> checked the jar files and did****
>
> find the class files. Anybody knows why? Thanks.****
>
> ** **
>
> ** **
>
> Regards,****
>
> ** **
>
> Libo****
>
> ** **
>
Re: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService
Posted by 闫昆 <ya...@gmail.com>.
wow! I have the same question and that is terrible,I speed three day
times.your ask question is also my resolve way
Thanks
2013/7/11 Devaraj k <de...@huawei.com>
> Hi Libo,****
>
> ** **
>
> MRAppMaster is not able to load the yarn related jar files. ****
>
> ** **
>
> Is this the classpath using by MRAppMaster or any other process? ****
>
> ** **
>
>
> “/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*”
> ****
>
> ** **
>
> If this is not the MRAppMaster process classpath, can you try to get the
> MRAppMaster java process classpath.****
>
> ** **
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> *From:* Yu, Libo [mailto:libo.yu@citi.com]
> *Sent:* 11 July 2013 01:30
> *To:* 'user@hadoop.apache.org'
> *Subject:* NoClassDefFoundError:
> org/apache/hadoop/yarn/service/CompositeService****
>
> ** **
>
> Hi,****
>
> ** **
>
> I tried to run the wordcount example with yarn. Here is the command line:*
> ***
>
> hadoop jar
> share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar
> wordcount /user/lyu/wordcount/input /user/lyu/wordcount/output****
>
> ** **
>
> But I got this exception:****
>
> ** **
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/yar
> n/service/CompositeService
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.service.CompositeService
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> ... 12 more
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit****
>
> ** **
>
> Here is conent of the CLASSPATH used by java:****
>
> ** **
>
>
> /opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*
> ****
>
> ** **
>
> Here is the console output:****
>
> ** **
>
> 13/07/09 20:43:07 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 13/07/09 20:43:07 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
> 13/07/09 20:43:07 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
> 13/07/09 20:43:08 INFO input.FileInputFormat: Total input paths to process
> : 0
> 13/07/09 20:43:08 INFO mapreduce.JobSubmitter: number of splits:0
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.value.class is
> deprecated. Instead, use mapreduce.job.output.value.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.combine.class is
> deprecated. Instead, use mapreduce.job.combine.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.job.name is deprecated.
> Instead, use mapreduce.job.name
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.reduce.class is
> deprecated. Instead, use mapreduce.job.reduce.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.input.dir is deprecated.
> Instead, use mapreduce.input.fileinputformat.inputdir
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.dir is
> deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.map.tasks is deprecated.
> Instead, use mapreduce.job.maps
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.key.class is
> deprecated. Instead, use mapreduce.job.output.key.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/07/09 20:43:09 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1373416798858_0001
> 13/07/09 20:43:09 INFO client.YarnClientImpl: Submitted application
> application_1373416798858_0001 to ResourceManager at
> host1.libo.com/192.168.10.1:8032
> 13/07/09 20:43:09 INFO mapreduce.Job: The url to track the job:
> http://host1.libo.com:8088/proxy/application_1373416798858_0001/
> 13/07/09 20:43:09 INFO mapreduce.Job: Running job: job_1373416798858_0001
> 13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 running
> in uber mode : false
> 13/07/09 20:43:12 INFO mapreduce.Job: map 0% reduce 0%
> 13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 failed
> with state FAILED due to: Application application_1373416798858_0001 failed
> 1 times due to AM Container for appattempt_1373416798858_0001_000001 exited
> with exitCode: 1 due to:
> .Failing this attempt.. Failing the application.
> 13/07/09 20:43:12 INFO mapreduce.Job: Counters: 0****
>
> ** **
>
> The paths for CompositeService and MRAppMaster are in the CLASSPATH and I
> checked the jar files and did****
>
> find the class files. Anybody knows why? Thanks.****
>
> ** **
>
> ** **
>
> Regards,****
>
> ** **
>
> Libo****
>
> ** **
>
Re: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService
Posted by 闫昆 <ya...@gmail.com>.
wow! I have the same question and that is terrible,I speed three day
times.your ask question is also my resolve way,thanks Devaraj k
2013/7/11 Devaraj k <de...@huawei.com>
> Hi Libo,****
>
> ** **
>
> MRAppMaster is not able to load the yarn related jar files. ****
>
> ** **
>
> Is this the classpath using by MRAppMaster or any other process? ****
>
> ** **
>
>
> “/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*”
> ****
>
> ** **
>
> If this is not the MRAppMaster process classpath, can you try to get the
> MRAppMaster java process classpath.****
>
> ** **
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> *From:* Yu, Libo [mailto:libo.yu@citi.com]
> *Sent:* 11 July 2013 01:30
> *To:* 'user@hadoop.apache.org'
> *Subject:* NoClassDefFoundError:
> org/apache/hadoop/yarn/service/CompositeService****
>
> ** **
>
> Hi,****
>
> ** **
>
> I tried to run the wordcount example with yarn. Here is the command line:*
> ***
>
> hadoop jar
> share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar
> wordcount /user/lyu/wordcount/input /user/lyu/wordcount/output****
>
> ** **
>
> But I got this exception:****
>
> ** **
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/yar
> n/service/CompositeService
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.service.CompositeService
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> ... 12 more
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit****
>
> ** **
>
> Here is conent of the CLASSPATH used by java:****
>
> ** **
>
>
> /opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*
> ****
>
> ** **
>
> Here is the console output:****
>
> ** **
>
> 13/07/09 20:43:07 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 13/07/09 20:43:07 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
> 13/07/09 20:43:07 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
> 13/07/09 20:43:08 INFO input.FileInputFormat: Total input paths to process
> : 0
> 13/07/09 20:43:08 INFO mapreduce.JobSubmitter: number of splits:0
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.value.class is
> deprecated. Instead, use mapreduce.job.output.value.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.combine.class is
> deprecated. Instead, use mapreduce.job.combine.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.job.name is deprecated.
> Instead, use mapreduce.job.name
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.reduce.class is
> deprecated. Instead, use mapreduce.job.reduce.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.input.dir is deprecated.
> Instead, use mapreduce.input.fileinputformat.inputdir
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.dir is
> deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.map.tasks is deprecated.
> Instead, use mapreduce.job.maps
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.key.class is
> deprecated. Instead, use mapreduce.job.output.key.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/07/09 20:43:09 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1373416798858_0001
> 13/07/09 20:43:09 INFO client.YarnClientImpl: Submitted application
> application_1373416798858_0001 to ResourceManager at
> host1.libo.com/192.168.10.1:8032
> 13/07/09 20:43:09 INFO mapreduce.Job: The url to track the job:
> http://host1.libo.com:8088/proxy/application_1373416798858_0001/
> 13/07/09 20:43:09 INFO mapreduce.Job: Running job: job_1373416798858_0001
> 13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 running
> in uber mode : false
> 13/07/09 20:43:12 INFO mapreduce.Job: map 0% reduce 0%
> 13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 failed
> with state FAILED due to: Application application_1373416798858_0001 failed
> 1 times due to AM Container for appattempt_1373416798858_0001_000001 exited
> with exitCode: 1 due to:
> .Failing this attempt.. Failing the application.
> 13/07/09 20:43:12 INFO mapreduce.Job: Counters: 0****
>
> ** **
>
> The paths for CompositeService and MRAppMaster are in the CLASSPATH and I
> checked the jar files and did****
>
> find the class files. Anybody knows why? Thanks.****
>
> ** **
>
> ** **
>
> Regards,****
>
> ** **
>
> Libo****
>
> ** **
>
Re: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService
Posted by 闫昆 <ya...@gmail.com>.
wow! I have the same question and that is terrible,I speed three day
times.your ask question is also my resolve way,thanks Devaraj k
2013/7/11 Devaraj k <de...@huawei.com>
> Hi Libo,****
>
> ** **
>
> MRAppMaster is not able to load the yarn related jar files. ****
>
> ** **
>
> Is this the classpath using by MRAppMaster or any other process? ****
>
> ** **
>
>
> “/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*”
> ****
>
> ** **
>
> If this is not the MRAppMaster process classpath, can you try to get the
> MRAppMaster java process classpath.****
>
> ** **
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> *From:* Yu, Libo [mailto:libo.yu@citi.com]
> *Sent:* 11 July 2013 01:30
> *To:* 'user@hadoop.apache.org'
> *Subject:* NoClassDefFoundError:
> org/apache/hadoop/yarn/service/CompositeService****
>
> ** **
>
> Hi,****
>
> ** **
>
> I tried to run the wordcount example with yarn. Here is the command line:*
> ***
>
> hadoop jar
> share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar
> wordcount /user/lyu/wordcount/input /user/lyu/wordcount/output****
>
> ** **
>
> But I got this exception:****
>
> ** **
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/yar
> n/service/CompositeService
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.service.CompositeService
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> ... 12 more
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit****
>
> ** **
>
> Here is conent of the CLASSPATH used by java:****
>
> ** **
>
>
> /opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*
> ****
>
> ** **
>
> Here is the console output:****
>
> ** **
>
> 13/07/09 20:43:07 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 13/07/09 20:43:07 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
> 13/07/09 20:43:07 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
> 13/07/09 20:43:08 INFO input.FileInputFormat: Total input paths to process
> : 0
> 13/07/09 20:43:08 INFO mapreduce.JobSubmitter: number of splits:0
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.value.class is
> deprecated. Instead, use mapreduce.job.output.value.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.combine.class is
> deprecated. Instead, use mapreduce.job.combine.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.job.name is deprecated.
> Instead, use mapreduce.job.name
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.reduce.class is
> deprecated. Instead, use mapreduce.job.reduce.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.input.dir is deprecated.
> Instead, use mapreduce.input.fileinputformat.inputdir
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.dir is
> deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.map.tasks is deprecated.
> Instead, use mapreduce.job.maps
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.key.class is
> deprecated. Instead, use mapreduce.job.output.key.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/07/09 20:43:09 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1373416798858_0001
> 13/07/09 20:43:09 INFO client.YarnClientImpl: Submitted application
> application_1373416798858_0001 to ResourceManager at
> host1.libo.com/192.168.10.1:8032
> 13/07/09 20:43:09 INFO mapreduce.Job: The url to track the job:
> http://host1.libo.com:8088/proxy/application_1373416798858_0001/
> 13/07/09 20:43:09 INFO mapreduce.Job: Running job: job_1373416798858_0001
> 13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 running
> in uber mode : false
> 13/07/09 20:43:12 INFO mapreduce.Job: map 0% reduce 0%
> 13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 failed
> with state FAILED due to: Application application_1373416798858_0001 failed
> 1 times due to AM Container for appattempt_1373416798858_0001_000001 exited
> with exitCode: 1 due to:
> .Failing this attempt.. Failing the application.
> 13/07/09 20:43:12 INFO mapreduce.Job: Counters: 0****
>
> ** **
>
> The paths for CompositeService and MRAppMaster are in the CLASSPATH and I
> checked the jar files and did****
>
> find the class files. Anybody knows why? Thanks.****
>
> ** **
>
> ** **
>
> Regards,****
>
> ** **
>
> Libo****
>
> ** **
>
Re: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService
Posted by 闫昆 <ya...@gmail.com>.
wow! I have the same question and that is terrible,I speed three day
times.your ask question is also my resolve way,thanks Devaraj k
2013/7/11 Devaraj k <de...@huawei.com>
> Hi Libo,****
>
> ** **
>
> MRAppMaster is not able to load the yarn related jar files. ****
>
> ** **
>
> Is this the classpath using by MRAppMaster or any other process? ****
>
> ** **
>
>
> “/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*”
> ****
>
> ** **
>
> If this is not the MRAppMaster process classpath, can you try to get the
> MRAppMaster java process classpath.****
>
> ** **
>
> ** **
>
> Thanks****
>
> Devaraj k****
>
> *From:* Yu, Libo [mailto:libo.yu@citi.com]
> *Sent:* 11 July 2013 01:30
> *To:* 'user@hadoop.apache.org'
> *Subject:* NoClassDefFoundError:
> org/apache/hadoop/yarn/service/CompositeService****
>
> ** **
>
> Hi,****
>
> ** **
>
> I tried to run the wordcount example with yarn. Here is the command line:*
> ***
>
> hadoop jar
> share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar
> wordcount /user/lyu/wordcount/input /user/lyu/wordcount/output****
>
> ** **
>
> But I got this exception:****
>
> ** **
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/yar
> n/service/CompositeService
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.service.CompositeService
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> ... 12 more
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit****
>
> ** **
>
> Here is conent of the CLASSPATH used by java:****
>
> ** **
>
>
> /opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*
> ****
>
> ** **
>
> Here is the console output:****
>
> ** **
>
> 13/07/09 20:43:07 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 13/07/09 20:43:07 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
> 13/07/09 20:43:07 INFO service.AbstractService:
> Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
> 13/07/09 20:43:08 INFO input.FileInputFormat: Total input paths to process
> : 0
> 13/07/09 20:43:08 INFO mapreduce.JobSubmitter: number of splits:0
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.jar is deprecated.
> Instead, use mapreduce.job.jar
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.value.class is
> deprecated. Instead, use mapreduce.job.output.value.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.combine.class is
> deprecated. Instead, use mapreduce.job.combine.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.job.name is deprecated.
> Instead, use mapreduce.job.name
> 13/07/09 20:43:08 WARN conf.Configuration: mapreduce.reduce.class is
> deprecated. Instead, use mapreduce.job.reduce.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.input.dir is deprecated.
> Instead, use mapreduce.input.fileinputformat.inputdir
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.dir is
> deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.map.tasks is deprecated.
> Instead, use mapreduce.job.maps
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.output.key.class is
> deprecated. Instead, use mapreduce.job.output.key.class
> 13/07/09 20:43:08 WARN conf.Configuration: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/07/09 20:43:09 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1373416798858_0001
> 13/07/09 20:43:09 INFO client.YarnClientImpl: Submitted application
> application_1373416798858_0001 to ResourceManager at
> host1.libo.com/192.168.10.1:8032
> 13/07/09 20:43:09 INFO mapreduce.Job: The url to track the job:
> http://host1.libo.com:8088/proxy/application_1373416798858_0001/
> 13/07/09 20:43:09 INFO mapreduce.Job: Running job: job_1373416798858_0001
> 13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 running
> in uber mode : false
> 13/07/09 20:43:12 INFO mapreduce.Job: map 0% reduce 0%
> 13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 failed
> with state FAILED due to: Application application_1373416798858_0001 failed
> 1 times due to AM Container for appattempt_1373416798858_0001_000001 exited
> with exitCode: 1 due to:
> .Failing this attempt.. Failing the application.
> 13/07/09 20:43:12 INFO mapreduce.Job: Counters: 0****
>
> ** **
>
> The paths for CompositeService and MRAppMaster are in the CLASSPATH and I
> checked the jar files and did****
>
> find the class files. Anybody knows why? Thanks.****
>
> ** **
>
> ** **
>
> Regards,****
>
> ** **
>
> Libo****
>
> ** **
>
RE: NoClassDefFoundError:
org/apache/hadoop/yarn/service/CompositeService
Posted by Devaraj k <de...@huawei.com>.
Hi Libo,
MRAppMaster is not able to load the yarn related jar files.
Is this the classpath using by MRAppMaster or any other process?
"/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*"
If this is not the MRAppMaster process classpath, can you try to get the MRAppMaster java process classpath.
Thanks
Devaraj k
From: Yu, Libo [mailto:libo.yu@citi.com]
Sent: 11 July 2013 01:30
To: 'user@hadoop.apache.org'
Subject: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService
Hi,
I tried to run the wordcount example with yarn. Here is the command line:
hadoop jar share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar wordcount /user/lyu/wordcount/input /user/lyu/wordcount/output
But I got this exception:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yar
n/service/CompositeService
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.service.CompositeService
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 12 more
Could not find the main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit
Here is conent of the CLASSPATH used by java:
/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*
Here is the console output:
13/07/09 20:43:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/07/09 20:43:07 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
13/07/09 20:43:07 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
13/07/09 20:43:08 INFO input.FileInputFormat: Total input paths to process : 0
13/07/09 20:43:08 INFO mapreduce.JobSubmitter: number of splits:0
13/07/09 20:43:08 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
13/07/09 20:43:08 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
13/07/09 20:43:09 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1373416798858_0001
13/07/09 20:43:09 INFO client.YarnClientImpl: Submitted application application_1373416798858_0001 to ResourceManager at host1.libo.com/192.168.10.1:8032
13/07/09 20:43:09 INFO mapreduce.Job: The url to track the job: http://host1.libo.com:8088/proxy/application_1373416798858_0001/
13/07/09 20:43:09 INFO mapreduce.Job: Running job: job_1373416798858_0001
13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 running in uber mode : false
13/07/09 20:43:12 INFO mapreduce.Job: map 0% reduce 0%
13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 failed with state FAILED due to: Application application_1373416798858_0001 failed 1 times due to AM Container for appattempt_1373416798858_0001_000001 exited with exitCode: 1 due to:
.Failing this attempt.. Failing the application.
13/07/09 20:43:12 INFO mapreduce.Job: Counters: 0
The paths for CompositeService and MRAppMaster are in the CLASSPATH and I checked the jar files and did
find the class files. Anybody knows why? Thanks.
Regards,
Libo
RE: NoClassDefFoundError:
org/apache/hadoop/yarn/service/CompositeService
Posted by Devaraj k <de...@huawei.com>.
Hi Libo,
MRAppMaster is not able to load the yarn related jar files.
Is this the classpath using by MRAppMaster or any other process?
"/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*"
If this is not the MRAppMaster process classpath, can you try to get the MRAppMaster java process classpath.
Thanks
Devaraj k
From: Yu, Libo [mailto:libo.yu@citi.com]
Sent: 11 July 2013 01:30
To: 'user@hadoop.apache.org'
Subject: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService
Hi,
I tried to run the wordcount example with yarn. Here is the command line:
hadoop jar share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar wordcount /user/lyu/wordcount/input /user/lyu/wordcount/output
But I got this exception:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yar
n/service/CompositeService
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.service.CompositeService
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 12 more
Could not find the main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit
Here is conent of the CLASSPATH used by java:
/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*
Here is the console output:
13/07/09 20:43:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/07/09 20:43:07 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
13/07/09 20:43:07 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
13/07/09 20:43:08 INFO input.FileInputFormat: Total input paths to process : 0
13/07/09 20:43:08 INFO mapreduce.JobSubmitter: number of splits:0
13/07/09 20:43:08 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
13/07/09 20:43:08 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
13/07/09 20:43:09 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1373416798858_0001
13/07/09 20:43:09 INFO client.YarnClientImpl: Submitted application application_1373416798858_0001 to ResourceManager at host1.libo.com/192.168.10.1:8032
13/07/09 20:43:09 INFO mapreduce.Job: The url to track the job: http://host1.libo.com:8088/proxy/application_1373416798858_0001/
13/07/09 20:43:09 INFO mapreduce.Job: Running job: job_1373416798858_0001
13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 running in uber mode : false
13/07/09 20:43:12 INFO mapreduce.Job: map 0% reduce 0%
13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 failed with state FAILED due to: Application application_1373416798858_0001 failed 1 times due to AM Container for appattempt_1373416798858_0001_000001 exited with exitCode: 1 due to:
.Failing this attempt.. Failing the application.
13/07/09 20:43:12 INFO mapreduce.Job: Counters: 0
The paths for CompositeService and MRAppMaster are in the CLASSPATH and I checked the jar files and did
find the class files. Anybody knows why? Thanks.
Regards,
Libo
RE: NoClassDefFoundError:
org/apache/hadoop/yarn/service/CompositeService
Posted by Devaraj k <de...@huawei.com>.
Hi Libo,
MRAppMaster is not able to load the yarn related jar files.
Is this the classpath using by MRAppMaster or any other process?
"/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*"
If this is not the MRAppMaster process classpath, can you try to get the MRAppMaster java process classpath.
Thanks
Devaraj k
From: Yu, Libo [mailto:libo.yu@citi.com]
Sent: 11 July 2013 01:30
To: 'user@hadoop.apache.org'
Subject: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService
Hi,
I tried to run the wordcount example with yarn. Here is the command line:
hadoop jar share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar wordcount /user/lyu/wordcount/input /user/lyu/wordcount/output
But I got this exception:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yar
n/service/CompositeService
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.service.CompositeService
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 12 more
Could not find the main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit
Here is conent of the CLASSPATH used by java:
/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*
Here is the console output:
13/07/09 20:43:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/07/09 20:43:07 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
13/07/09 20:43:07 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
13/07/09 20:43:08 INFO input.FileInputFormat: Total input paths to process : 0
13/07/09 20:43:08 INFO mapreduce.JobSubmitter: number of splits:0
13/07/09 20:43:08 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
13/07/09 20:43:08 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
13/07/09 20:43:09 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1373416798858_0001
13/07/09 20:43:09 INFO client.YarnClientImpl: Submitted application application_1373416798858_0001 to ResourceManager at host1.libo.com/192.168.10.1:8032
13/07/09 20:43:09 INFO mapreduce.Job: The url to track the job: http://host1.libo.com:8088/proxy/application_1373416798858_0001/
13/07/09 20:43:09 INFO mapreduce.Job: Running job: job_1373416798858_0001
13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 running in uber mode : false
13/07/09 20:43:12 INFO mapreduce.Job: map 0% reduce 0%
13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 failed with state FAILED due to: Application application_1373416798858_0001 failed 1 times due to AM Container for appattempt_1373416798858_0001_000001 exited with exitCode: 1 due to:
.Failing this attempt.. Failing the application.
13/07/09 20:43:12 INFO mapreduce.Job: Counters: 0
The paths for CompositeService and MRAppMaster are in the CLASSPATH and I checked the jar files and did
find the class files. Anybody knows why? Thanks.
Regards,
Libo
RE: NoClassDefFoundError:
org/apache/hadoop/yarn/service/CompositeService
Posted by Devaraj k <de...@huawei.com>.
Hi Libo,
MRAppMaster is not able to load the yarn related jar files.
Is this the classpath using by MRAppMaster or any other process?
"/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*"
If this is not the MRAppMaster process classpath, can you try to get the MRAppMaster java process classpath.
Thanks
Devaraj k
From: Yu, Libo [mailto:libo.yu@citi.com]
Sent: 11 July 2013 01:30
To: 'user@hadoop.apache.org'
Subject: NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService
Hi,
I tried to run the wordcount example with yarn. Here is the command line:
hadoop jar share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar wordcount /user/lyu/wordcount/input /user/lyu/wordcount/output
But I got this exception:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yar
n/service/CompositeService
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.service.CompositeService
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 12 more
Could not find the main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit
Here is conent of the CLASSPATH used by java:
/opt/hadoop/hadoop-2.0.0-cdh4.3.0/conf:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/common/*:/contrib/capacity-scheduler/*.jar:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/hdfs/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/yarn/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce2/lib/*:/opt/hadoop/hadoop-2.0.0-cdh4.3.0/share/hadoop/mapreduce/*
Here is the console output:
13/07/09 20:43:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/07/09 20:43:07 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
13/07/09 20:43:07 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
13/07/09 20:43:08 INFO input.FileInputFormat: Total input paths to process : 0
13/07/09 20:43:08 INFO mapreduce.JobSubmitter: number of splits:0
13/07/09 20:43:08 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
13/07/09 20:43:08 WARN conf.Configuration: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
13/07/09 20:43:08 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
13/07/09 20:43:08 WARN conf.Configuration: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
13/07/09 20:43:08 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
13/07/09 20:43:09 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1373416798858_0001
13/07/09 20:43:09 INFO client.YarnClientImpl: Submitted application application_1373416798858_0001 to ResourceManager at host1.libo.com/192.168.10.1:8032
13/07/09 20:43:09 INFO mapreduce.Job: The url to track the job: http://host1.libo.com:8088/proxy/application_1373416798858_0001/
13/07/09 20:43:09 INFO mapreduce.Job: Running job: job_1373416798858_0001
13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 running in uber mode : false
13/07/09 20:43:12 INFO mapreduce.Job: map 0% reduce 0%
13/07/09 20:43:12 INFO mapreduce.Job: Job job_1373416798858_0001 failed with state FAILED due to: Application application_1373416798858_0001 failed 1 times due to AM Container for appattempt_1373416798858_0001_000001 exited with exitCode: 1 due to:
.Failing this attempt.. Failing the application.
13/07/09 20:43:12 INFO mapreduce.Job: Counters: 0
The paths for CompositeService and MRAppMaster are in the CLASSPATH and I checked the jar files and did
find the class files. Anybody knows why? Thanks.
Regards,
Libo