You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by siscia <si...@yahoo.com.INVALID> on 2015/12/14 10:56:46 UTC
GridMix doesn't run ClassNotFoundException Rumen
Hello folks,
I am trying to run a simulation with GridMix but the simulation fails
every time.
If I inspect the log I find the error with looks like this:
"Error:
*java.lang.ClassNotFoundException:
org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
java.security.AccessController.doPrivileged(Native Method)\n\tat
java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat
java.lang.Class.getDeclaredConstructors0(Native Method)\n\tat
java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat java.lang.Class.getConstructor0(Class.java:2895)\n\tat
java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat
java.security.AccessController.doPrivileged(Native Method)\n\tat
javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
The error is pretty clear, but I have no idea how to fix it.
I run GridMix by command line with:
bin/hadoop jar share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar
iopath trace.json
Any hits ?
Cheers :)
Simone
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by siscia <si...@yahoo.com.INVALID>.
Hi :)
It seems perfectly fine:
simo@simo:~/hadoop-2.6.1$ jar -tvf
share/hadoop/tools/lib/hadoop-rumen-2.6.1.jar | grep
ResourceUsageMetrics
3696 Wed Sep 16 21:17:24 CEST 2015
org/apache/hadoop/tools/rumen/ResourceUsageMetrics.class
I honestly believe that the problem is something way more trivial...
But I have no idea what to check out...
Cheers
Simone
On 16/12/2015 13:20, Wellington Chevreuil wrote:
> Hi,
>
> It seems the classpath definition is ok. Can you inspect the rumen jar
> to make sure the file is not corrupt somehow?
>
> jar -tvf share/hadoop/tools/lib/hadoop-rumen-2.6.1.jar | grep
> ResourceUsageMetrics
>
>> On 15 Dec 2015, at 17:14, siscia <sisciamirror@yahoo.com
>> <ma...@yahoo.com>> wrote:
>>
>> Hi,
>>
>> It looks like everything is in place but still doesn't work...
>>
>> Let me recap, I am running in pseudo distributed mode.
>>
>> I start the system with this sequence of command.
>>
>> cd hadoop-2.6.1
>> sbin/start-dfs.sh
>> sbin/start-yarn.sh
>> sbin/hadoop-daemon.sh start datanode
>> sbin/yarn-daemon.sh start nodemanager
>>
>>
>> Now the single node of the system is healthy and everything looks cool.
>>
>> jps give me this:
>>
>> simo@simo:~/hadoop-2.6.1$ jps
>> 10107 Jps
>> 4114 NodeManager
>> 3533 SecondaryNameNode
>> 3344 NameNode
>> 4014 DataNode
>> 3737 ResourceManager
>>
>>
>> I check which user is running the NodeManager
>>
>> simo@simo:~/hadoop-2.6.1$ ps u 4114
>> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
>> COMMAND
>> simo 4114 1.9 8.7 1764708 346188 pts/3 Sl 17:37 0:39
>> /usr/bin/java -Dproc_nodemanager -Xmx1000m -server
>> -Dhadoop.log.dir=/home/simo/hadoop
>>
>>
>> I am running the nodemanager.
>>
>> Let me check the permission of the rumen jar
>>
>> simo@simo:~/hadoop-2.6.1$ ls -l share/hadoop/tools/lib/ | grep rumen
>> -rw-r--r-- 1 simo simo 278018 set 16 23:18 hadoop-rumen-2.6.1.jar
>>
>>
>> I can read the rumen jar.
>>
>> I don't have a lot of ideas of what I could be doing wrong...
>>
>> You have any thoughts ?
>>
>> Thanks for your time,
>>
>> Simone
>>
>>
>> On 15/12/2015 16:27, Wellington Chevreuil wrote:
>>>
>>> Hi,
>>>
>>> Since it's on yarn class path, it should already be on Nodemanager
>>> or the jobs path. You can check the Nodemanager process itself, with
>>> a ps command.
>>>
>>> Can you also check the file permissions for this jar? Make sure it's
>>> readable by the user running Nodemanager process.
>>>
>>> Em 15/12/2015 12:53, "siscia" <si...@yahoo.com> escreveu:
>>>
>>> Hi Wellington,
>>>
>>> I am missing a passage, if I check the classpath of both hadoop
>>> and yarn they should include hadoop-rumen-2.6.1.jar
>>>
>>> simo@simo:~/hadoop-2.6.1$
>>> <mailto:simo@simo:%7E/hadoop-2.6.1$> bin/hadoop classpath
>>> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
>>>
>>> simo@simo:~/hadoop-2.6.1$
>>> <mailto:simo@simo:%7E/hadoop-2.6.1$> bin/yarn classpath
>>> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/*home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
>>>
>>>
>>> Given that
>>>
>>> simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
>>> hadoop-extras-2.6.1.jar servlet-api-2.5.jar
>>> *hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
>>> hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
>>> hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
>>> hadoop-sls-2.6.1.jar xz-1.0.jar
>>> hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
>>>
>>> Am I doing something wrong ? How do I check the classpath of the
>>> NodeManager ?
>>>
>>> Greets
>>>
>>> Simone
>>>
>>> On 15/12/2015 12:45, Wellington Chevreuil wrote:
>>>> Hi Simone,
>>>>
>>>> You should make sure to include hadoop-rumen-2.6.0.jar on the
>>>> classpath for the Nodemanagers, or include it on the classpath
>>>> of your job.
>>>>
>>>>> On 14 Dec 2015, at 09:56, siscia
>>>>> <si...@yahoo.com.INVALID> wrote:
>>>>>
>>>>> Hello folks,
>>>>>
>>>>> I am trying to run a simulation with GridMix but the
>>>>> simulation fails every time.
>>>>>
>>>>> If I inspect the log I find the error with looks like this:
>>>>>
>>>>> "Error:
>>>>> *java.lang.ClassNotFoundException:
>>>>> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>>>>>
>>>>> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
>>>>> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
>>>>> java.security.AccessController.doPrivileged(Native
>>>>> Method)\n\tat
>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat java.lang.Class.getDeclaredConstructors0(Native
>>>>> Method)\n\tat
>>>>> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
>>>>> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
>>>>> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat
>>>>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
>>>>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
>>>>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
>>>>> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
>>>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
>>>>> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
>>>>> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat
>>>>> java.security.AccessController.doPrivileged(Native
>>>>> Method)\n\tat
>>>>> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
>>>>> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>>>>>
>>>>> The error is pretty clear, but I have no idea how to fix it.
>>>>>
>>>>> I run GridMix by command line with:
>>>>>
>>>>> bin/hadoop jar
>>>>> share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath
>>>>> trace.json
>>>>>
>>>>> Any hits ?
>>>>>
>>>>> Cheers :)
>>>>>
>>>>> Simone
>>>>
>>>
>>
>
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by siscia <si...@yahoo.com.INVALID>.
Hi :)
It seems perfectly fine:
simo@simo:~/hadoop-2.6.1$ jar -tvf
share/hadoop/tools/lib/hadoop-rumen-2.6.1.jar | grep
ResourceUsageMetrics
3696 Wed Sep 16 21:17:24 CEST 2015
org/apache/hadoop/tools/rumen/ResourceUsageMetrics.class
I honestly believe that the problem is something way more trivial...
But I have no idea what to check out...
Cheers
Simone
On 16/12/2015 13:20, Wellington Chevreuil wrote:
> Hi,
>
> It seems the classpath definition is ok. Can you inspect the rumen jar
> to make sure the file is not corrupt somehow?
>
> jar -tvf share/hadoop/tools/lib/hadoop-rumen-2.6.1.jar | grep
> ResourceUsageMetrics
>
>> On 15 Dec 2015, at 17:14, siscia <sisciamirror@yahoo.com
>> <ma...@yahoo.com>> wrote:
>>
>> Hi,
>>
>> It looks like everything is in place but still doesn't work...
>>
>> Let me recap, I am running in pseudo distributed mode.
>>
>> I start the system with this sequence of command.
>>
>> cd hadoop-2.6.1
>> sbin/start-dfs.sh
>> sbin/start-yarn.sh
>> sbin/hadoop-daemon.sh start datanode
>> sbin/yarn-daemon.sh start nodemanager
>>
>>
>> Now the single node of the system is healthy and everything looks cool.
>>
>> jps give me this:
>>
>> simo@simo:~/hadoop-2.6.1$ jps
>> 10107 Jps
>> 4114 NodeManager
>> 3533 SecondaryNameNode
>> 3344 NameNode
>> 4014 DataNode
>> 3737 ResourceManager
>>
>>
>> I check which user is running the NodeManager
>>
>> simo@simo:~/hadoop-2.6.1$ ps u 4114
>> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
>> COMMAND
>> simo 4114 1.9 8.7 1764708 346188 pts/3 Sl 17:37 0:39
>> /usr/bin/java -Dproc_nodemanager -Xmx1000m -server
>> -Dhadoop.log.dir=/home/simo/hadoop
>>
>>
>> I am running the nodemanager.
>>
>> Let me check the permission of the rumen jar
>>
>> simo@simo:~/hadoop-2.6.1$ ls -l share/hadoop/tools/lib/ | grep rumen
>> -rw-r--r-- 1 simo simo 278018 set 16 23:18 hadoop-rumen-2.6.1.jar
>>
>>
>> I can read the rumen jar.
>>
>> I don't have a lot of ideas of what I could be doing wrong...
>>
>> You have any thoughts ?
>>
>> Thanks for your time,
>>
>> Simone
>>
>>
>> On 15/12/2015 16:27, Wellington Chevreuil wrote:
>>>
>>> Hi,
>>>
>>> Since it's on yarn class path, it should already be on Nodemanager
>>> or the jobs path. You can check the Nodemanager process itself, with
>>> a ps command.
>>>
>>> Can you also check the file permissions for this jar? Make sure it's
>>> readable by the user running Nodemanager process.
>>>
>>> Em 15/12/2015 12:53, "siscia" <si...@yahoo.com> escreveu:
>>>
>>> Hi Wellington,
>>>
>>> I am missing a passage, if I check the classpath of both hadoop
>>> and yarn they should include hadoop-rumen-2.6.1.jar
>>>
>>> simo@simo:~/hadoop-2.6.1$
>>> <mailto:simo@simo:%7E/hadoop-2.6.1$> bin/hadoop classpath
>>> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
>>>
>>> simo@simo:~/hadoop-2.6.1$
>>> <mailto:simo@simo:%7E/hadoop-2.6.1$> bin/yarn classpath
>>> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/*home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
>>>
>>>
>>> Given that
>>>
>>> simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
>>> hadoop-extras-2.6.1.jar servlet-api-2.5.jar
>>> *hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
>>> hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
>>> hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
>>> hadoop-sls-2.6.1.jar xz-1.0.jar
>>> hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
>>>
>>> Am I doing something wrong ? How do I check the classpath of the
>>> NodeManager ?
>>>
>>> Greets
>>>
>>> Simone
>>>
>>> On 15/12/2015 12:45, Wellington Chevreuil wrote:
>>>> Hi Simone,
>>>>
>>>> You should make sure to include hadoop-rumen-2.6.0.jar on the
>>>> classpath for the Nodemanagers, or include it on the classpath
>>>> of your job.
>>>>
>>>>> On 14 Dec 2015, at 09:56, siscia
>>>>> <si...@yahoo.com.INVALID> wrote:
>>>>>
>>>>> Hello folks,
>>>>>
>>>>> I am trying to run a simulation with GridMix but the
>>>>> simulation fails every time.
>>>>>
>>>>> If I inspect the log I find the error with looks like this:
>>>>>
>>>>> "Error:
>>>>> *java.lang.ClassNotFoundException:
>>>>> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>>>>>
>>>>> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
>>>>> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
>>>>> java.security.AccessController.doPrivileged(Native
>>>>> Method)\n\tat
>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat java.lang.Class.getDeclaredConstructors0(Native
>>>>> Method)\n\tat
>>>>> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
>>>>> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
>>>>> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat
>>>>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
>>>>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
>>>>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
>>>>> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
>>>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
>>>>> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
>>>>> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat
>>>>> java.security.AccessController.doPrivileged(Native
>>>>> Method)\n\tat
>>>>> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
>>>>> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>>>>>
>>>>> The error is pretty clear, but I have no idea how to fix it.
>>>>>
>>>>> I run GridMix by command line with:
>>>>>
>>>>> bin/hadoop jar
>>>>> share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath
>>>>> trace.json
>>>>>
>>>>> Any hits ?
>>>>>
>>>>> Cheers :)
>>>>>
>>>>> Simone
>>>>
>>>
>>
>
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by siscia <si...@yahoo.com.INVALID>.
Hi :)
It seems perfectly fine:
simo@simo:~/hadoop-2.6.1$ jar -tvf
share/hadoop/tools/lib/hadoop-rumen-2.6.1.jar | grep
ResourceUsageMetrics
3696 Wed Sep 16 21:17:24 CEST 2015
org/apache/hadoop/tools/rumen/ResourceUsageMetrics.class
I honestly believe that the problem is something way more trivial...
But I have no idea what to check out...
Cheers
Simone
On 16/12/2015 13:20, Wellington Chevreuil wrote:
> Hi,
>
> It seems the classpath definition is ok. Can you inspect the rumen jar
> to make sure the file is not corrupt somehow?
>
> jar -tvf share/hadoop/tools/lib/hadoop-rumen-2.6.1.jar | grep
> ResourceUsageMetrics
>
>> On 15 Dec 2015, at 17:14, siscia <sisciamirror@yahoo.com
>> <ma...@yahoo.com>> wrote:
>>
>> Hi,
>>
>> It looks like everything is in place but still doesn't work...
>>
>> Let me recap, I am running in pseudo distributed mode.
>>
>> I start the system with this sequence of command.
>>
>> cd hadoop-2.6.1
>> sbin/start-dfs.sh
>> sbin/start-yarn.sh
>> sbin/hadoop-daemon.sh start datanode
>> sbin/yarn-daemon.sh start nodemanager
>>
>>
>> Now the single node of the system is healthy and everything looks cool.
>>
>> jps give me this:
>>
>> simo@simo:~/hadoop-2.6.1$ jps
>> 10107 Jps
>> 4114 NodeManager
>> 3533 SecondaryNameNode
>> 3344 NameNode
>> 4014 DataNode
>> 3737 ResourceManager
>>
>>
>> I check which user is running the NodeManager
>>
>> simo@simo:~/hadoop-2.6.1$ ps u 4114
>> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
>> COMMAND
>> simo 4114 1.9 8.7 1764708 346188 pts/3 Sl 17:37 0:39
>> /usr/bin/java -Dproc_nodemanager -Xmx1000m -server
>> -Dhadoop.log.dir=/home/simo/hadoop
>>
>>
>> I am running the nodemanager.
>>
>> Let me check the permission of the rumen jar
>>
>> simo@simo:~/hadoop-2.6.1$ ls -l share/hadoop/tools/lib/ | grep rumen
>> -rw-r--r-- 1 simo simo 278018 set 16 23:18 hadoop-rumen-2.6.1.jar
>>
>>
>> I can read the rumen jar.
>>
>> I don't have a lot of ideas of what I could be doing wrong...
>>
>> You have any thoughts ?
>>
>> Thanks for your time,
>>
>> Simone
>>
>>
>> On 15/12/2015 16:27, Wellington Chevreuil wrote:
>>>
>>> Hi,
>>>
>>> Since it's on yarn class path, it should already be on Nodemanager
>>> or the jobs path. You can check the Nodemanager process itself, with
>>> a ps command.
>>>
>>> Can you also check the file permissions for this jar? Make sure it's
>>> readable by the user running Nodemanager process.
>>>
>>> Em 15/12/2015 12:53, "siscia" <si...@yahoo.com> escreveu:
>>>
>>> Hi Wellington,
>>>
>>> I am missing a passage, if I check the classpath of both hadoop
>>> and yarn they should include hadoop-rumen-2.6.1.jar
>>>
>>> simo@simo:~/hadoop-2.6.1$
>>> <mailto:simo@simo:%7E/hadoop-2.6.1$> bin/hadoop classpath
>>> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
>>>
>>> simo@simo:~/hadoop-2.6.1$
>>> <mailto:simo@simo:%7E/hadoop-2.6.1$> bin/yarn classpath
>>> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/*home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
>>>
>>>
>>> Given that
>>>
>>> simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
>>> hadoop-extras-2.6.1.jar servlet-api-2.5.jar
>>> *hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
>>> hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
>>> hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
>>> hadoop-sls-2.6.1.jar xz-1.0.jar
>>> hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
>>>
>>> Am I doing something wrong ? How do I check the classpath of the
>>> NodeManager ?
>>>
>>> Greets
>>>
>>> Simone
>>>
>>> On 15/12/2015 12:45, Wellington Chevreuil wrote:
>>>> Hi Simone,
>>>>
>>>> You should make sure to include hadoop-rumen-2.6.0.jar on the
>>>> classpath for the Nodemanagers, or include it on the classpath
>>>> of your job.
>>>>
>>>>> On 14 Dec 2015, at 09:56, siscia
>>>>> <si...@yahoo.com.INVALID> wrote:
>>>>>
>>>>> Hello folks,
>>>>>
>>>>> I am trying to run a simulation with GridMix but the
>>>>> simulation fails every time.
>>>>>
>>>>> If I inspect the log I find the error with looks like this:
>>>>>
>>>>> "Error:
>>>>> *java.lang.ClassNotFoundException:
>>>>> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>>>>>
>>>>> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
>>>>> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
>>>>> java.security.AccessController.doPrivileged(Native
>>>>> Method)\n\tat
>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat java.lang.Class.getDeclaredConstructors0(Native
>>>>> Method)\n\tat
>>>>> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
>>>>> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
>>>>> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat
>>>>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
>>>>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
>>>>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
>>>>> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
>>>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
>>>>> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
>>>>> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat
>>>>> java.security.AccessController.doPrivileged(Native
>>>>> Method)\n\tat
>>>>> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
>>>>> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>>>>>
>>>>> The error is pretty clear, but I have no idea how to fix it.
>>>>>
>>>>> I run GridMix by command line with:
>>>>>
>>>>> bin/hadoop jar
>>>>> share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath
>>>>> trace.json
>>>>>
>>>>> Any hits ?
>>>>>
>>>>> Cheers :)
>>>>>
>>>>> Simone
>>>>
>>>
>>
>
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by siscia <si...@yahoo.com.INVALID>.
Hi :)
It seems perfectly fine:
simo@simo:~/hadoop-2.6.1$ jar -tvf
share/hadoop/tools/lib/hadoop-rumen-2.6.1.jar | grep
ResourceUsageMetrics
3696 Wed Sep 16 21:17:24 CEST 2015
org/apache/hadoop/tools/rumen/ResourceUsageMetrics.class
I honestly believe that the problem is something way more trivial...
But I have no idea what to check out...
Cheers
Simone
On 16/12/2015 13:20, Wellington Chevreuil wrote:
> Hi,
>
> It seems the classpath definition is ok. Can you inspect the rumen jar
> to make sure the file is not corrupt somehow?
>
> jar -tvf share/hadoop/tools/lib/hadoop-rumen-2.6.1.jar | grep
> ResourceUsageMetrics
>
>> On 15 Dec 2015, at 17:14, siscia <sisciamirror@yahoo.com
>> <ma...@yahoo.com>> wrote:
>>
>> Hi,
>>
>> It looks like everything is in place but still doesn't work...
>>
>> Let me recap, I am running in pseudo distributed mode.
>>
>> I start the system with this sequence of command.
>>
>> cd hadoop-2.6.1
>> sbin/start-dfs.sh
>> sbin/start-yarn.sh
>> sbin/hadoop-daemon.sh start datanode
>> sbin/yarn-daemon.sh start nodemanager
>>
>>
>> Now the single node of the system is healthy and everything looks cool.
>>
>> jps give me this:
>>
>> simo@simo:~/hadoop-2.6.1$ jps
>> 10107 Jps
>> 4114 NodeManager
>> 3533 SecondaryNameNode
>> 3344 NameNode
>> 4014 DataNode
>> 3737 ResourceManager
>>
>>
>> I check which user is running the NodeManager
>>
>> simo@simo:~/hadoop-2.6.1$ ps u 4114
>> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME
>> COMMAND
>> simo 4114 1.9 8.7 1764708 346188 pts/3 Sl 17:37 0:39
>> /usr/bin/java -Dproc_nodemanager -Xmx1000m -server
>> -Dhadoop.log.dir=/home/simo/hadoop
>>
>>
>> I am running the nodemanager.
>>
>> Let me check the permission of the rumen jar
>>
>> simo@simo:~/hadoop-2.6.1$ ls -l share/hadoop/tools/lib/ | grep rumen
>> -rw-r--r-- 1 simo simo 278018 set 16 23:18 hadoop-rumen-2.6.1.jar
>>
>>
>> I can read the rumen jar.
>>
>> I don't have a lot of ideas of what I could be doing wrong...
>>
>> You have any thoughts ?
>>
>> Thanks for your time,
>>
>> Simone
>>
>>
>> On 15/12/2015 16:27, Wellington Chevreuil wrote:
>>>
>>> Hi,
>>>
>>> Since it's on yarn class path, it should already be on Nodemanager
>>> or the jobs path. You can check the Nodemanager process itself, with
>>> a ps command.
>>>
>>> Can you also check the file permissions for this jar? Make sure it's
>>> readable by the user running Nodemanager process.
>>>
>>> Em 15/12/2015 12:53, "siscia" <si...@yahoo.com> escreveu:
>>>
>>> Hi Wellington,
>>>
>>> I am missing a passage, if I check the classpath of both hadoop
>>> and yarn they should include hadoop-rumen-2.6.1.jar
>>>
>>> simo@simo:~/hadoop-2.6.1$
>>> <mailto:simo@simo:%7E/hadoop-2.6.1$> bin/hadoop classpath
>>> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
>>>
>>> simo@simo:~/hadoop-2.6.1$
>>> <mailto:simo@simo:%7E/hadoop-2.6.1$> bin/yarn classpath
>>> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/*home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
>>>
>>>
>>> Given that
>>>
>>> simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
>>> hadoop-extras-2.6.1.jar servlet-api-2.5.jar
>>> *hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
>>> hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
>>> hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
>>> hadoop-sls-2.6.1.jar xz-1.0.jar
>>> hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
>>>
>>> Am I doing something wrong ? How do I check the classpath of the
>>> NodeManager ?
>>>
>>> Greets
>>>
>>> Simone
>>>
>>> On 15/12/2015 12:45, Wellington Chevreuil wrote:
>>>> Hi Simone,
>>>>
>>>> You should make sure to include hadoop-rumen-2.6.0.jar on the
>>>> classpath for the Nodemanagers, or include it on the classpath
>>>> of your job.
>>>>
>>>>> On 14 Dec 2015, at 09:56, siscia
>>>>> <si...@yahoo.com.INVALID> wrote:
>>>>>
>>>>> Hello folks,
>>>>>
>>>>> I am trying to run a simulation with GridMix but the
>>>>> simulation fails every time.
>>>>>
>>>>> If I inspect the log I find the error with looks like this:
>>>>>
>>>>> "Error:
>>>>> *java.lang.ClassNotFoundException:
>>>>> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>>>>>
>>>>> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
>>>>> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
>>>>> java.security.AccessController.doPrivileged(Native
>>>>> Method)\n\tat
>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat java.lang.Class.getDeclaredConstructors0(Native
>>>>> Method)\n\tat
>>>>> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
>>>>> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
>>>>> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat
>>>>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
>>>>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
>>>>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
>>>>> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
>>>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
>>>>> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
>>>>> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat
>>>>> java.security.AccessController.doPrivileged(Native
>>>>> Method)\n\tat
>>>>> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
>>>>> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>>>>>
>>>>> The error is pretty clear, but I have no idea how to fix it.
>>>>>
>>>>> I run GridMix by command line with:
>>>>>
>>>>> bin/hadoop jar
>>>>> share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath
>>>>> trace.json
>>>>>
>>>>> Any hits ?
>>>>>
>>>>> Cheers :)
>>>>>
>>>>> Simone
>>>>
>>>
>>
>
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by siscia <si...@yahoo.com.INVALID>.
Hi,
It looks like everything is in place but still doesn't work...
Let me recap, I am running in pseudo distributed mode.
I start the system with this sequence of command.
cd hadoop-2.6.1
sbin/start-dfs.sh
sbin/start-yarn.sh
sbin/hadoop-daemon.sh start datanode
sbin/yarn-daemon.sh start nodemanager
Now the single node of the system is healthy and everything looks cool.
jps give me this:
simo@simo:~/hadoop-2.6.1$ jps
10107 Jps
4114 NodeManager
3533 SecondaryNameNode
3344 NameNode
4014 DataNode
3737 ResourceManager
I check which user is running the NodeManager
simo@simo:~/hadoop-2.6.1$ ps u 4114
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
simo 4114 1.9 8.7 1764708 346188 pts/3 Sl 17:37 0:39
/usr/bin/java -Dproc_nodemanager -Xmx1000m -server
-Dhadoop.log.dir=/home/simo/hadoop
I am running the nodemanager.
Let me check the permission of the rumen jar
simo@simo:~/hadoop-2.6.1$ ls -l share/hadoop/tools/lib/ | grep rumen
-rw-r--r-- 1 simo simo 278018 set 16 23:18 hadoop-rumen-2.6.1.jar
I can read the rumen jar.
I don't have a lot of ideas of what I could be doing wrong...
You have any thoughts ?
Thanks for your time,
Simone
On 15/12/2015 16:27, Wellington Chevreuil wrote:
>
> Hi,
>
> Since it's on yarn class path, it should already be on Nodemanager or
> the jobs path. You can check the Nodemanager process itself, with a ps
> command.
>
> Can you also check the file permissions for this jar? Make sure it's
> readable by the user running Nodemanager process.
>
> Em 15/12/2015 12:53, "siscia" <sisciamirror@yahoo.com
> <ma...@yahoo.com>> escreveu:
>
> Hi Wellington,
>
> I am missing a passage, if I check the classpath of both hadoop
> and yarn they should include hadoop-rumen-2.6.1.jar
>
> simo@simo:~/hadoop-2.6.1$ <mailto:simo@simo:%7E/hadoop-2.6.1$>
> bin/hadoop classpath
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
>
> simo@simo:~/hadoop-2.6.1$ <mailto:simo@simo:%7E/hadoop-2.6.1$>
> bin/yarn classpath
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/*home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
>
>
> Given that
>
> simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
> hadoop-extras-2.6.1.jar servlet-api-2.5.jar
> *hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
> hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
> hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
> hadoop-sls-2.6.1.jar xz-1.0.jar
> hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
>
> Am I doing something wrong ? How do I check the classpath of the
> NodeManager ?
>
> Greets
>
> Simone
>
> On 15/12/2015 12:45, Wellington Chevreuil wrote:
>> Hi Simone,
>>
>> You should make sure to include hadoop-rumen-2.6.0.jar on the
>> classpath for the Nodemanagers, or include it on the classpath of
>> your job.
>>
>>> On 14 Dec 2015, at 09:56, siscia <sisciamirror@yahoo.com.INVALID
>>> <ma...@yahoo.com.INVALID>> wrote:
>>>
>>> Hello folks,
>>>
>>> I am trying to run a simulation with GridMix but the simulation
>>> fails every time.
>>>
>>> If I inspect the log I find the error with looks like this:
>>>
>>> "Error:
>>> *java.lang.ClassNotFoundException:
>>> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>>>
>>> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
>>> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
>>> java.security.AccessController.doPrivileged(Native
>>> Method)\n\tat
>>> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat
>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat
>>> java.lang.Class.getDeclaredConstructors0(Native
>>> Method)\n\tat
>>> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
>>> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
>>> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
>>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
>>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
>>> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
>>> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
>>> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat
>>> java.security.AccessController.doPrivileged(Native
>>> Method)\n\tat
>>> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
>>> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>>>
>>> The error is pretty clear, but I have no idea how to fix it.
>>>
>>> I run GridMix by command line with:
>>>
>>> bin/hadoop jar
>>> share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath
>>> trace.json
>>>
>>> Any hits ?
>>>
>>> Cheers :)
>>>
>>> Simone
>>
>
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by siscia <si...@yahoo.com.INVALID>.
Hi,
It looks like everything is in place but still doesn't work...
Let me recap, I am running in pseudo distributed mode.
I start the system with this sequence of command.
cd hadoop-2.6.1
sbin/start-dfs.sh
sbin/start-yarn.sh
sbin/hadoop-daemon.sh start datanode
sbin/yarn-daemon.sh start nodemanager
Now the single node of the system is healthy and everything looks cool.
jps give me this:
simo@simo:~/hadoop-2.6.1$ jps
10107 Jps
4114 NodeManager
3533 SecondaryNameNode
3344 NameNode
4014 DataNode
3737 ResourceManager
I check which user is running the NodeManager
simo@simo:~/hadoop-2.6.1$ ps u 4114
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
simo 4114 1.9 8.7 1764708 346188 pts/3 Sl 17:37 0:39
/usr/bin/java -Dproc_nodemanager -Xmx1000m -server
-Dhadoop.log.dir=/home/simo/hadoop
I am running the nodemanager.
Let me check the permission of the rumen jar
simo@simo:~/hadoop-2.6.1$ ls -l share/hadoop/tools/lib/ | grep rumen
-rw-r--r-- 1 simo simo 278018 set 16 23:18 hadoop-rumen-2.6.1.jar
I can read the rumen jar.
I don't have a lot of ideas of what I could be doing wrong...
You have any thoughts ?
Thanks for your time,
Simone
On 15/12/2015 16:27, Wellington Chevreuil wrote:
>
> Hi,
>
> Since it's on yarn class path, it should already be on Nodemanager or
> the jobs path. You can check the Nodemanager process itself, with a ps
> command.
>
> Can you also check the file permissions for this jar? Make sure it's
> readable by the user running Nodemanager process.
>
> Em 15/12/2015 12:53, "siscia" <sisciamirror@yahoo.com
> <ma...@yahoo.com>> escreveu:
>
> Hi Wellington,
>
> I am missing a passage, if I check the classpath of both hadoop
> and yarn they should include hadoop-rumen-2.6.1.jar
>
> simo@simo:~/hadoop-2.6.1$ <mailto:simo@simo:%7E/hadoop-2.6.1$>
> bin/hadoop classpath
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
>
> simo@simo:~/hadoop-2.6.1$ <mailto:simo@simo:%7E/hadoop-2.6.1$>
> bin/yarn classpath
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/*home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
>
>
> Given that
>
> simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
> hadoop-extras-2.6.1.jar servlet-api-2.5.jar
> *hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
> hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
> hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
> hadoop-sls-2.6.1.jar xz-1.0.jar
> hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
>
> Am I doing something wrong ? How do I check the classpath of the
> NodeManager ?
>
> Greets
>
> Simone
>
> On 15/12/2015 12:45, Wellington Chevreuil wrote:
>> Hi Simone,
>>
>> You should make sure to include hadoop-rumen-2.6.0.jar on the
>> classpath for the Nodemanagers, or include it on the classpath of
>> your job.
>>
>>> On 14 Dec 2015, at 09:56, siscia <sisciamirror@yahoo.com.INVALID
>>> <ma...@yahoo.com.INVALID>> wrote:
>>>
>>> Hello folks,
>>>
>>> I am trying to run a simulation with GridMix but the simulation
>>> fails every time.
>>>
>>> If I inspect the log I find the error with looks like this:
>>>
>>> "Error:
>>> *java.lang.ClassNotFoundException:
>>> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>>>
>>> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
>>> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
>>> java.security.AccessController.doPrivileged(Native
>>> Method)\n\tat
>>> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat
>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat
>>> java.lang.Class.getDeclaredConstructors0(Native
>>> Method)\n\tat
>>> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
>>> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
>>> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
>>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
>>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
>>> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
>>> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
>>> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat
>>> java.security.AccessController.doPrivileged(Native
>>> Method)\n\tat
>>> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
>>> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>>>
>>> The error is pretty clear, but I have no idea how to fix it.
>>>
>>> I run GridMix by command line with:
>>>
>>> bin/hadoop jar
>>> share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath
>>> trace.json
>>>
>>> Any hits ?
>>>
>>> Cheers :)
>>>
>>> Simone
>>
>
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by siscia <si...@yahoo.com.INVALID>.
Hi,
It looks like everything is in place but still doesn't work...
Let me recap, I am running in pseudo distributed mode.
I start the system with this sequence of command.
cd hadoop-2.6.1
sbin/start-dfs.sh
sbin/start-yarn.sh
sbin/hadoop-daemon.sh start datanode
sbin/yarn-daemon.sh start nodemanager
Now the single node of the system is healthy and everything looks cool.
jps give me this:
simo@simo:~/hadoop-2.6.1$ jps
10107 Jps
4114 NodeManager
3533 SecondaryNameNode
3344 NameNode
4014 DataNode
3737 ResourceManager
I check which user is running the NodeManager
simo@simo:~/hadoop-2.6.1$ ps u 4114
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
simo 4114 1.9 8.7 1764708 346188 pts/3 Sl 17:37 0:39
/usr/bin/java -Dproc_nodemanager -Xmx1000m -server
-Dhadoop.log.dir=/home/simo/hadoop
I am running the nodemanager.
Let me check the permission of the rumen jar
simo@simo:~/hadoop-2.6.1$ ls -l share/hadoop/tools/lib/ | grep rumen
-rw-r--r-- 1 simo simo 278018 set 16 23:18 hadoop-rumen-2.6.1.jar
I can read the rumen jar.
I don't have a lot of ideas of what I could be doing wrong...
You have any thoughts ?
Thanks for your time,
Simone
On 15/12/2015 16:27, Wellington Chevreuil wrote:
>
> Hi,
>
> Since it's on yarn class path, it should already be on Nodemanager or
> the jobs path. You can check the Nodemanager process itself, with a ps
> command.
>
> Can you also check the file permissions for this jar? Make sure it's
> readable by the user running Nodemanager process.
>
> Em 15/12/2015 12:53, "siscia" <sisciamirror@yahoo.com
> <ma...@yahoo.com>> escreveu:
>
> Hi Wellington,
>
> I am missing a passage, if I check the classpath of both hadoop
> and yarn they should include hadoop-rumen-2.6.1.jar
>
> simo@simo:~/hadoop-2.6.1$ <mailto:simo@simo:%7E/hadoop-2.6.1$>
> bin/hadoop classpath
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
>
> simo@simo:~/hadoop-2.6.1$ <mailto:simo@simo:%7E/hadoop-2.6.1$>
> bin/yarn classpath
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/*home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
>
>
> Given that
>
> simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
> hadoop-extras-2.6.1.jar servlet-api-2.5.jar
> *hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
> hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
> hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
> hadoop-sls-2.6.1.jar xz-1.0.jar
> hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
>
> Am I doing something wrong ? How do I check the classpath of the
> NodeManager ?
>
> Greets
>
> Simone
>
> On 15/12/2015 12:45, Wellington Chevreuil wrote:
>> Hi Simone,
>>
>> You should make sure to include hadoop-rumen-2.6.0.jar on the
>> classpath for the Nodemanagers, or include it on the classpath of
>> your job.
>>
>>> On 14 Dec 2015, at 09:56, siscia <sisciamirror@yahoo.com.INVALID
>>> <ma...@yahoo.com.INVALID>> wrote:
>>>
>>> Hello folks,
>>>
>>> I am trying to run a simulation with GridMix but the simulation
>>> fails every time.
>>>
>>> If I inspect the log I find the error with looks like this:
>>>
>>> "Error:
>>> *java.lang.ClassNotFoundException:
>>> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>>>
>>> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
>>> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
>>> java.security.AccessController.doPrivileged(Native
>>> Method)\n\tat
>>> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat
>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat
>>> java.lang.Class.getDeclaredConstructors0(Native
>>> Method)\n\tat
>>> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
>>> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
>>> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
>>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
>>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
>>> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
>>> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
>>> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat
>>> java.security.AccessController.doPrivileged(Native
>>> Method)\n\tat
>>> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
>>> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>>>
>>> The error is pretty clear, but I have no idea how to fix it.
>>>
>>> I run GridMix by command line with:
>>>
>>> bin/hadoop jar
>>> share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath
>>> trace.json
>>>
>>> Any hits ?
>>>
>>> Cheers :)
>>>
>>> Simone
>>
>
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by siscia <si...@yahoo.com.INVALID>.
Hi,
It looks like everything is in place but still doesn't work...
Let me recap, I am running in pseudo distributed mode.
I start the system with this sequence of command.
cd hadoop-2.6.1
sbin/start-dfs.sh
sbin/start-yarn.sh
sbin/hadoop-daemon.sh start datanode
sbin/yarn-daemon.sh start nodemanager
Now the single node of the system is healthy and everything looks cool.
jps give me this:
simo@simo:~/hadoop-2.6.1$ jps
10107 Jps
4114 NodeManager
3533 SecondaryNameNode
3344 NameNode
4014 DataNode
3737 ResourceManager
I check which user is running the NodeManager
simo@simo:~/hadoop-2.6.1$ ps u 4114
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
simo 4114 1.9 8.7 1764708 346188 pts/3 Sl 17:37 0:39
/usr/bin/java -Dproc_nodemanager -Xmx1000m -server
-Dhadoop.log.dir=/home/simo/hadoop
I am running the nodemanager.
Let me check the permission of the rumen jar
simo@simo:~/hadoop-2.6.1$ ls -l share/hadoop/tools/lib/ | grep rumen
-rw-r--r-- 1 simo simo 278018 set 16 23:18 hadoop-rumen-2.6.1.jar
I can read the rumen jar.
I don't have a lot of ideas of what I could be doing wrong...
You have any thoughts ?
Thanks for your time,
Simone
On 15/12/2015 16:27, Wellington Chevreuil wrote:
>
> Hi,
>
> Since it's on yarn class path, it should already be on Nodemanager or
> the jobs path. You can check the Nodemanager process itself, with a ps
> command.
>
> Can you also check the file permissions for this jar? Make sure it's
> readable by the user running Nodemanager process.
>
> Em 15/12/2015 12:53, "siscia" <sisciamirror@yahoo.com
> <ma...@yahoo.com>> escreveu:
>
> Hi Wellington,
>
> I am missing a passage, if I check the classpath of both hadoop
> and yarn they should include hadoop-rumen-2.6.1.jar
>
> simo@simo:~/hadoop-2.6.1$ <mailto:simo@simo:%7E/hadoop-2.6.1$>
> bin/hadoop classpath
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
>
> simo@simo:~/hadoop-2.6.1$ <mailto:simo@simo:%7E/hadoop-2.6.1$>
> bin/yarn classpath
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/*home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
>
>
> Given that
>
> simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
> hadoop-extras-2.6.1.jar servlet-api-2.5.jar
> *hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
> hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
> hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
> hadoop-sls-2.6.1.jar xz-1.0.jar
> hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
>
> Am I doing something wrong ? How do I check the classpath of the
> NodeManager ?
>
> Greets
>
> Simone
>
> On 15/12/2015 12:45, Wellington Chevreuil wrote:
>> Hi Simone,
>>
>> You should make sure to include hadoop-rumen-2.6.0.jar on the
>> classpath for the Nodemanagers, or include it on the classpath of
>> your job.
>>
>>> On 14 Dec 2015, at 09:56, siscia <sisciamirror@yahoo.com.INVALID
>>> <ma...@yahoo.com.INVALID>> wrote:
>>>
>>> Hello folks,
>>>
>>> I am trying to run a simulation with GridMix but the simulation
>>> fails every time.
>>>
>>> If I inspect the log I find the error with looks like this:
>>>
>>> "Error:
>>> *java.lang.ClassNotFoundException:
>>> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>>>
>>> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
>>> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
>>> java.security.AccessController.doPrivileged(Native
>>> Method)\n\tat
>>> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat
>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat
>>> java.lang.Class.getDeclaredConstructors0(Native
>>> Method)\n\tat
>>> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
>>> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
>>> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
>>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
>>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
>>> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
>>> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
>>> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat
>>> java.security.AccessController.doPrivileged(Native
>>> Method)\n\tat
>>> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
>>> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>>>
>>> The error is pretty clear, but I have no idea how to fix it.
>>>
>>> I run GridMix by command line with:
>>>
>>> bin/hadoop jar
>>> share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath
>>> trace.json
>>>
>>> Any hits ?
>>>
>>> Cheers :)
>>>
>>> Simone
>>
>
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by Wellington Chevreuil <we...@gmail.com>.
Hi,
Since it's on yarn class path, it should already be on Nodemanager or the
jobs path. You can check the Nodemanager process itself, with a ps command.
Can you also check the file permissions for this jar? Make sure it's
readable by the user running Nodemanager process.
Em 15/12/2015 12:53, "siscia" <si...@yahoo.com> escreveu:
> Hi Wellington,
>
> I am missing a passage, if I check the classpath of both hadoop and yarn
> they should include hadoop-rumen-2.6.1.jar
>
> simo@simo:~/hadoop-2.6.1$ bin/hadoop classpath
>
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:
> */home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:
> */home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**
> :/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
>
> simo@simo:~/hadoop-2.6.1$ bin/yarn classpath
>
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/
> *home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:
> */home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*
> *:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
>
>
> Given that
>
> simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
> hadoop-extras-2.6.1.jar servlet-api-2.5.jar
> *hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
> hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
> hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
> hadoop-sls-2.6.1.jar xz-1.0.jar
> hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
>
> Am I doing something wrong ? How do I check the classpath of the
> NodeManager ?
>
> Greets
>
> Simone
>
> On 15/12/2015 12:45, Wellington Chevreuil wrote:
>
> Hi Simone,
>
> You should make sure to include hadoop-rumen-2.6.0.jar on the classpath
> for the Nodemanagers, or include it on the classpath of your job.
>
> On 14 Dec 2015, at 09:56, siscia <sisciamirror@yahoo.com.INVALID
> <si...@yahoo.com.invalid>> wrote:
>
> Hello folks,
>
> I am trying to run a simulation with GridMix but the simulation fails
> every time.
>
> If I inspect the log I find the error with looks like this:
>
> "Error:
> *java.lang.ClassNotFoundException:
> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>
> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
> java.security.AccessController.doPrivileged(Native Method)\n\tat
> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat
> java.lang.Class.getDeclaredConstructors0(Native Method)\n\tat
> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat
> java.security.AccessController.doPrivileged(Native Method)\n\tat
> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>
> The error is pretty clear, but I have no idea how to fix it.
>
> I run GridMix by command line with:
>
> bin/hadoop jar share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath
> trace.json
>
> Any hits ?
>
> Cheers :)
>
> Simone
>
>
>
>
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by Wellington Chevreuil <we...@gmail.com>.
Hi,
Since it's on yarn class path, it should already be on Nodemanager or the
jobs path. You can check the Nodemanager process itself, with a ps command.
Can you also check the file permissions for this jar? Make sure it's
readable by the user running Nodemanager process.
Em 15/12/2015 12:53, "siscia" <si...@yahoo.com> escreveu:
> Hi Wellington,
>
> I am missing a passage, if I check the classpath of both hadoop and yarn
> they should include hadoop-rumen-2.6.1.jar
>
> simo@simo:~/hadoop-2.6.1$ bin/hadoop classpath
>
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:
> */home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:
> */home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**
> :/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
>
> simo@simo:~/hadoop-2.6.1$ bin/yarn classpath
>
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/
> *home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:
> */home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*
> *:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
>
>
> Given that
>
> simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
> hadoop-extras-2.6.1.jar servlet-api-2.5.jar
> *hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
> hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
> hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
> hadoop-sls-2.6.1.jar xz-1.0.jar
> hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
>
> Am I doing something wrong ? How do I check the classpath of the
> NodeManager ?
>
> Greets
>
> Simone
>
> On 15/12/2015 12:45, Wellington Chevreuil wrote:
>
> Hi Simone,
>
> You should make sure to include hadoop-rumen-2.6.0.jar on the classpath
> for the Nodemanagers, or include it on the classpath of your job.
>
> On 14 Dec 2015, at 09:56, siscia <sisciamirror@yahoo.com.INVALID
> <si...@yahoo.com.invalid>> wrote:
>
> Hello folks,
>
> I am trying to run a simulation with GridMix but the simulation fails
> every time.
>
> If I inspect the log I find the error with looks like this:
>
> "Error:
> *java.lang.ClassNotFoundException:
> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>
> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
> java.security.AccessController.doPrivileged(Native Method)\n\tat
> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat
> java.lang.Class.getDeclaredConstructors0(Native Method)\n\tat
> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat
> java.security.AccessController.doPrivileged(Native Method)\n\tat
> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>
> The error is pretty clear, but I have no idea how to fix it.
>
> I run GridMix by command line with:
>
> bin/hadoop jar share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath
> trace.json
>
> Any hits ?
>
> Cheers :)
>
> Simone
>
>
>
>
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by Wellington Chevreuil <we...@gmail.com>.
Hi,
Since it's on yarn class path, it should already be on Nodemanager or the
jobs path. You can check the Nodemanager process itself, with a ps command.
Can you also check the file permissions for this jar? Make sure it's
readable by the user running Nodemanager process.
Em 15/12/2015 12:53, "siscia" <si...@yahoo.com> escreveu:
> Hi Wellington,
>
> I am missing a passage, if I check the classpath of both hadoop and yarn
> they should include hadoop-rumen-2.6.1.jar
>
> simo@simo:~/hadoop-2.6.1$ bin/hadoop classpath
>
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:
> */home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:
> */home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**
> :/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
>
> simo@simo:~/hadoop-2.6.1$ bin/yarn classpath
>
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/
> *home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:
> */home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*
> *:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
>
>
> Given that
>
> simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
> hadoop-extras-2.6.1.jar servlet-api-2.5.jar
> *hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
> hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
> hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
> hadoop-sls-2.6.1.jar xz-1.0.jar
> hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
>
> Am I doing something wrong ? How do I check the classpath of the
> NodeManager ?
>
> Greets
>
> Simone
>
> On 15/12/2015 12:45, Wellington Chevreuil wrote:
>
> Hi Simone,
>
> You should make sure to include hadoop-rumen-2.6.0.jar on the classpath
> for the Nodemanagers, or include it on the classpath of your job.
>
> On 14 Dec 2015, at 09:56, siscia <sisciamirror@yahoo.com.INVALID
> <si...@yahoo.com.invalid>> wrote:
>
> Hello folks,
>
> I am trying to run a simulation with GridMix but the simulation fails
> every time.
>
> If I inspect the log I find the error with looks like this:
>
> "Error:
> *java.lang.ClassNotFoundException:
> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>
> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
> java.security.AccessController.doPrivileged(Native Method)\n\tat
> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat
> java.lang.Class.getDeclaredConstructors0(Native Method)\n\tat
> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat
> java.security.AccessController.doPrivileged(Native Method)\n\tat
> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>
> The error is pretty clear, but I have no idea how to fix it.
>
> I run GridMix by command line with:
>
> bin/hadoop jar share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath
> trace.json
>
> Any hits ?
>
> Cheers :)
>
> Simone
>
>
>
>
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by Wellington Chevreuil <we...@gmail.com>.
Hi,
Since it's on yarn class path, it should already be on Nodemanager or the
jobs path. You can check the Nodemanager process itself, with a ps command.
Can you also check the file permissions for this jar? Make sure it's
readable by the user running Nodemanager process.
Em 15/12/2015 12:53, "siscia" <si...@yahoo.com> escreveu:
> Hi Wellington,
>
> I am missing a passage, if I check the classpath of both hadoop and yarn
> they should include hadoop-rumen-2.6.1.jar
>
> simo@simo:~/hadoop-2.6.1$ bin/hadoop classpath
>
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:
> */home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:
> */home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**
> :/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
>
> simo@simo:~/hadoop-2.6.1$ bin/yarn classpath
>
> /home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/
> *home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:
> */home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*
> *:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
>
>
> Given that
>
> simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
> hadoop-extras-2.6.1.jar servlet-api-2.5.jar
> *hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
> hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
> hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
> hadoop-sls-2.6.1.jar xz-1.0.jar
> hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
>
> Am I doing something wrong ? How do I check the classpath of the
> NodeManager ?
>
> Greets
>
> Simone
>
> On 15/12/2015 12:45, Wellington Chevreuil wrote:
>
> Hi Simone,
>
> You should make sure to include hadoop-rumen-2.6.0.jar on the classpath
> for the Nodemanagers, or include it on the classpath of your job.
>
> On 14 Dec 2015, at 09:56, siscia <sisciamirror@yahoo.com.INVALID
> <si...@yahoo.com.invalid>> wrote:
>
> Hello folks,
>
> I am trying to run a simulation with GridMix but the simulation fails
> every time.
>
> If I inspect the log I find the error with looks like this:
>
> "Error:
> *java.lang.ClassNotFoundException:
> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>
> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
> java.security.AccessController.doPrivileged(Native Method)\n\tat
> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat
> java.lang.Class.getDeclaredConstructors0(Native Method)\n\tat
> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat
> java.security.AccessController.doPrivileged(Native Method)\n\tat
> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>
> The error is pretty clear, but I have no idea how to fix it.
>
> I run GridMix by command line with:
>
> bin/hadoop jar share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath
> trace.json
>
> Any hits ?
>
> Cheers :)
>
> Simone
>
>
>
>
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by siscia <si...@yahoo.com.INVALID>.
Hi Wellington,
I am missing a passage, if I check the classpath of both hadoop and yarn
they should include hadoop-rumen-2.6.1.jar
simo@simo:~/hadoop-2.6.1$ bin/hadoop classpath
/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
simo@simo:~/hadoop-2.6.1$ bin/yarn classpath
/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/*home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
Given that
simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
hadoop-extras-2.6.1.jar servlet-api-2.5.jar
*hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
hadoop-sls-2.6.1.jar xz-1.0.jar
hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
Am I doing something wrong ? How do I check the classpath of the
NodeManager ?
Greets
Simone
On 15/12/2015 12:45, Wellington Chevreuil wrote:
> Hi Simone,
>
> You should make sure to include hadoop-rumen-2.6.0.jar on the
> classpath for the Nodemanagers, or include it on the classpath of your
> job.
>
>> On 14 Dec 2015, at 09:56, siscia <sisciamirror@yahoo.com.INVALID
>> <ma...@yahoo.com.invalid>> wrote:
>>
>> Hello folks,
>>
>> I am trying to run a simulation with GridMix but the simulation fails
>> every time.
>>
>> If I inspect the log I find the error with looks like this:
>>
>> "Error:
>> *java.lang.ClassNotFoundException:
>> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>>
>> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
>> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
>> java.security.AccessController.doPrivileged(Native Method)\n\tat
>> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat
>> java.lang.Class.getDeclaredConstructors0(Native Method)\n\tat
>> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
>> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
>> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat
>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
>> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
>> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
>> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat java.security.AccessController.doPrivileged(Native
>> Method)\n\tat
>> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
>> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>>
>> The error is pretty clear, but I have no idea how to fix it.
>>
>> I run GridMix by command line with:
>>
>> bin/hadoop jar share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar
>> iopath trace.json
>>
>> Any hits ?
>>
>> Cheers :)
>>
>> Simone
>
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by siscia <si...@yahoo.com.INVALID>.
Hi Wellington,
I am missing a passage, if I check the classpath of both hadoop and yarn
they should include hadoop-rumen-2.6.1.jar
simo@simo:~/hadoop-2.6.1$ bin/hadoop classpath
/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
simo@simo:~/hadoop-2.6.1$ bin/yarn classpath
/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/*home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
Given that
simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
hadoop-extras-2.6.1.jar servlet-api-2.5.jar
*hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
hadoop-sls-2.6.1.jar xz-1.0.jar
hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
Am I doing something wrong ? How do I check the classpath of the
NodeManager ?
Greets
Simone
On 15/12/2015 12:45, Wellington Chevreuil wrote:
> Hi Simone,
>
> You should make sure to include hadoop-rumen-2.6.0.jar on the
> classpath for the Nodemanagers, or include it on the classpath of your
> job.
>
>> On 14 Dec 2015, at 09:56, siscia <sisciamirror@yahoo.com.INVALID
>> <ma...@yahoo.com.invalid>> wrote:
>>
>> Hello folks,
>>
>> I am trying to run a simulation with GridMix but the simulation fails
>> every time.
>>
>> If I inspect the log I find the error with looks like this:
>>
>> "Error:
>> *java.lang.ClassNotFoundException:
>> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>>
>> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
>> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
>> java.security.AccessController.doPrivileged(Native Method)\n\tat
>> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat
>> java.lang.Class.getDeclaredConstructors0(Native Method)\n\tat
>> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
>> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
>> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat
>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
>> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
>> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
>> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat java.security.AccessController.doPrivileged(Native
>> Method)\n\tat
>> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
>> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>>
>> The error is pretty clear, but I have no idea how to fix it.
>>
>> I run GridMix by command line with:
>>
>> bin/hadoop jar share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar
>> iopath trace.json
>>
>> Any hits ?
>>
>> Cheers :)
>>
>> Simone
>
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by siscia <si...@yahoo.com.INVALID>.
Hi Wellington,
I am missing a passage, if I check the classpath of both hadoop and yarn
they should include hadoop-rumen-2.6.1.jar
simo@simo:~/hadoop-2.6.1$ bin/hadoop classpath
/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
simo@simo:~/hadoop-2.6.1$ bin/yarn classpath
/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/*home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
Given that
simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
hadoop-extras-2.6.1.jar servlet-api-2.5.jar
*hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
hadoop-sls-2.6.1.jar xz-1.0.jar
hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
Am I doing something wrong ? How do I check the classpath of the
NodeManager ?
Greets
Simone
On 15/12/2015 12:45, Wellington Chevreuil wrote:
> Hi Simone,
>
> You should make sure to include hadoop-rumen-2.6.0.jar on the
> classpath for the Nodemanagers, or include it on the classpath of your
> job.
>
>> On 14 Dec 2015, at 09:56, siscia <sisciamirror@yahoo.com.INVALID
>> <ma...@yahoo.com.invalid>> wrote:
>>
>> Hello folks,
>>
>> I am trying to run a simulation with GridMix but the simulation fails
>> every time.
>>
>> If I inspect the log I find the error with looks like this:
>>
>> "Error:
>> *java.lang.ClassNotFoundException:
>> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>>
>> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
>> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
>> java.security.AccessController.doPrivileged(Native Method)\n\tat
>> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat
>> java.lang.Class.getDeclaredConstructors0(Native Method)\n\tat
>> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
>> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
>> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat
>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
>> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
>> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
>> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat java.security.AccessController.doPrivileged(Native
>> Method)\n\tat
>> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
>> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>>
>> The error is pretty clear, but I have no idea how to fix it.
>>
>> I run GridMix by command line with:
>>
>> bin/hadoop jar share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar
>> iopath trace.json
>>
>> Any hits ?
>>
>> Cheers :)
>>
>> Simone
>
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by siscia <si...@yahoo.com.INVALID>.
Hi Wellington,
I am missing a passage, if I check the classpath of both hadoop and yarn
they should include hadoop-rumen-2.6.1.jar
simo@simo:~/hadoop-2.6.1$ bin/hadoop classpath
/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH
simo@simo:~/hadoop-2.6.1$ bin/yarn classpath
/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/conf:/home/simo/hadoop-2.6.1/share/hadoop/common/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/common/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/hdfs/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/lib/*:/home/simo/hadoop-2.6.1/share/hadoop/mapreduce/*:/*home/simo/hadoop-2.6.1/share/hadoop/tools/lib/*:*/home/simo/hadoop-2.6.1/share/hadoop/tools/lib/**:/usr/lib/jvm/java-7-openjdk-amd64/lib/:/usr/lib/jvm/java-1.7.0-openjdk-amd64/lib/tools.jar:HADOOP_CLASSPATH:/home/simo/hadoop-2.6.1/share/hadoop/yarn/*:/home/simo/hadoop-2.6.1/share/hadoop/yarn/lib/*
Given that
simo@simo:~*/hadoop-2.6.1/share/hadoop/tools/lib*$ ls
hadoop-extras-2.6.1.jar servlet-api-2.5.jar
*hadoop-gridmix-2.6.1.jar* snappy-java-1.0.4.1.jar
hadoop-openstack-2.6.1.jar stax-api-1.0-2.jar
hadoop-rumen-2.6.1.jar xmlenc-0.52.jar
hadoop-sls-2.6.1.jar xz-1.0.jar
hadoop-streaming-2.6.1.jar zookeeper-3.4.6.jar
Am I doing something wrong ? How do I check the classpath of the
NodeManager ?
Greets
Simone
On 15/12/2015 12:45, Wellington Chevreuil wrote:
> Hi Simone,
>
> You should make sure to include hadoop-rumen-2.6.0.jar on the
> classpath for the Nodemanagers, or include it on the classpath of your
> job.
>
>> On 14 Dec 2015, at 09:56, siscia <sisciamirror@yahoo.com.INVALID
>> <ma...@yahoo.com.invalid>> wrote:
>>
>> Hello folks,
>>
>> I am trying to run a simulation with GridMix but the simulation fails
>> every time.
>>
>> If I inspect the log I find the error with looks like this:
>>
>> "Error:
>> *java.lang.ClassNotFoundException:
>> org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat*
>>
>> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat
>> java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat
>> java.security.AccessController.doPrivileged(Native Method)\n\tat
>> java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat
>> java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat
>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat
>> java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat
>> java.lang.Class.getDeclaredConstructors0(Native Method)\n\tat
>> java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat
>> java.lang.Class.getConstructor0(Class.java:2895)\n\tat
>> java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat
>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat
>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat
>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat
>> org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat
>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat
>> org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat
>> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat java.security.AccessController.doPrivileged(Native
>> Method)\n\tat
>> javax.security.auth.Subject.doAs(Subject.java:415)\n\tat
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat
>> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>>
>> The error is pretty clear, but I have no idea how to fix it.
>>
>> I run GridMix by command line with:
>>
>> bin/hadoop jar share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar
>> iopath trace.json
>>
>> Any hits ?
>>
>> Cheers :)
>>
>> Simone
>
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by Wellington Chevreuil <we...@gmail.com>.
Hi Simone,
You should make sure to include hadoop-rumen-2.6.0.jar on the classpath for the Nodemanagers, or include it on the classpath of your job.
> On 14 Dec 2015, at 09:56, siscia <si...@yahoo.com.INVALID> wrote:
>
> Hello folks,
>
> I am trying to run a simulation with GridMix but the simulation fails every time.
>
> If I inspect the log I find the error with looks like this:
>
> "Error:
> java.lang.ClassNotFoundException: org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat
>
> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat java.security.AccessController.doPrivileged(Native Method)\n\tat java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat java.lang.Class.getDeclaredConstructors0(Native Method)\n\tat java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat java.lang.Class.getConstructor0(Class.java:2895)\n\tat java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat java.security.AccessController.doPrivileged(Native Method)\n\tat javax.security.auth.Subject.doAs(Subject.java:415)\n\tat org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>
> The error is pretty clear, but I have no idea how to fix it.
>
> I run GridMix by command line with:
>
> bin/hadoop jar share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath trace.json
>
> Any hits ?
>
> Cheers :)
>
> Simone
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by Wellington Chevreuil <we...@gmail.com>.
Hi Simone,
You should make sure to include hadoop-rumen-2.6.0.jar on the classpath for the Nodemanagers, or include it on the classpath of your job.
> On 14 Dec 2015, at 09:56, siscia <si...@yahoo.com.INVALID> wrote:
>
> Hello folks,
>
> I am trying to run a simulation with GridMix but the simulation fails every time.
>
> If I inspect the log I find the error with looks like this:
>
> "Error:
> java.lang.ClassNotFoundException: org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat
>
> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat java.security.AccessController.doPrivileged(Native Method)\n\tat java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat java.lang.Class.getDeclaredConstructors0(Native Method)\n\tat java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat java.lang.Class.getConstructor0(Class.java:2895)\n\tat java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat java.security.AccessController.doPrivileged(Native Method)\n\tat javax.security.auth.Subject.doAs(Subject.java:415)\n\tat org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>
> The error is pretty clear, but I have no idea how to fix it.
>
> I run GridMix by command line with:
>
> bin/hadoop jar share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath trace.json
>
> Any hits ?
>
> Cheers :)
>
> Simone
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by Wellington Chevreuil <we...@gmail.com>.
Hi Simone,
You should make sure to include hadoop-rumen-2.6.0.jar on the classpath for the Nodemanagers, or include it on the classpath of your job.
> On 14 Dec 2015, at 09:56, siscia <si...@yahoo.com.INVALID> wrote:
>
> Hello folks,
>
> I am trying to run a simulation with GridMix but the simulation fails every time.
>
> If I inspect the log I find the error with looks like this:
>
> "Error:
> java.lang.ClassNotFoundException: org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat
>
> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat java.security.AccessController.doPrivileged(Native Method)\n\tat java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat java.lang.Class.getDeclaredConstructors0(Native Method)\n\tat java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat java.lang.Class.getConstructor0(Class.java:2895)\n\tat java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat java.security.AccessController.doPrivileged(Native Method)\n\tat javax.security.auth.Subject.doAs(Subject.java:415)\n\tat org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>
> The error is pretty clear, but I have no idea how to fix it.
>
> I run GridMix by command line with:
>
> bin/hadoop jar share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath trace.json
>
> Any hits ?
>
> Cheers :)
>
> Simone
Re: GridMix doesn't run ClassNotFoundException Rumen
Posted by Wellington Chevreuil <we...@gmail.com>.
Hi Simone,
You should make sure to include hadoop-rumen-2.6.0.jar on the classpath for the Nodemanagers, or include it on the classpath of your job.
> On 14 Dec 2015, at 09:56, siscia <si...@yahoo.com.INVALID> wrote:
>
> Hello folks,
>
> I am trying to run a simulation with GridMix but the simulation fails every time.
>
> If I inspect the log I find the error with looks like this:
>
> "Error:
> java.lang.ClassNotFoundException: org.apache.hadoop.tools.rumen.ResourceUsageMetrics\n\tat
>
> java.net.URLClassLoader$1.run(URLClassLoader.java:366)\n\tat java.net.URLClassLoader$1.run(URLClassLoader.java:355)\n\tat java.security.AccessController.doPrivileged(Native Method)\n\tat java.net.URLClassLoader.findClass(URLClassLoader.java:354)\n\tat java.lang.ClassLoader.loadClass(ClassLoader.java:425)\n\tat sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)\n\tat java.lang.ClassLoader.loadClass(ClassLoader.java:358)\n\tat java.lang.Class.getDeclaredConstructors0(Native Method)\n\tat java.lang.Class.privateGetDeclaredConstructors(Class.java:2595)\n\tat java.lang.Class.getConstructor0(Class.java:2895)\n\tat java.lang.Class.getDeclaredConstructor(Class.java:2066)\n\tat org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)\n\tat org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:66)\n\tat org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)\n\tat org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:372)\n\tat org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)\n\tat org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)\n\tat org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)\n\tat java.security.AccessController.doPrivileged(Native Method)\n\tat javax.security.auth.Subject.doAs(Subject.java:415)\n\tat org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)\n\tat org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)\n",
>
> The error is pretty clear, but I have no idea how to fix it.
>
> I run GridMix by command line with:
>
> bin/hadoop jar share/hadoop/tools/lib/hadoop-gridmix-2.6.1.jar iopath trace.json
>
> Any hits ?
>
> Cheers :)
>
> Simone