You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by "Sengupta, Sohini IN BLR SISL" <so...@siemens.com> on 2011/03/15 10:13:53 UTC
Error: java.lang.ClassNotFoundException:
org.apache.mahout.math.Vector
Hi all,
I am encountering following error when I try to run mahout on a cluster. I have set the classpath to all dependencies in hadoop-env.sh.
Any input/ pointers will be helpful.
Thanks a lot in advance
Sohini
/usr/local/mahout-distribution-0.4$ /usr/local/hadoop/bin/hadoop jar /usr/local/mahout-distribution-0.4/examples/target/mahout-examples-0.4-job.jar org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
11/03/15 14:35:50 INFO kmeans.Job: Running with default arguments
11/03/15 14:35:51 INFO common.HadoopUtil: Deleting output
11/03/15 14:35:51 INFO kmeans.Job: Preparing Input
11/03/15 14:35:51 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
11/03/15 14:35:53 INFO input.FileInputFormat: Total input paths to process : 1
11/03/15 14:35:54 INFO mapred.JobClient: Running job: job_201103111106_0086
11/03/15 14:35:55 INFO mapred.JobClient: map 0% reduce 0%
11/03/15 14:36:05 INFO mapred.JobClient: map 100% reduce 0%
11/03/15 14:36:08 INFO mapred.JobClient: Job complete: job_201103111106_0086
11/03/15 14:36:08 INFO mapred.JobClient: Counters: 7
11/03/15 14:36:08 INFO mapred.JobClient: Job Counters
11/03/15 14:36:08 INFO mapred.JobClient: Rack-local map tasks=1
11/03/15 14:36:08 INFO mapred.JobClient: Launched map tasks=1
11/03/15 14:36:08 INFO mapred.JobClient: FileSystemCounters
11/03/15 14:36:08 INFO mapred.JobClient: HDFS_BYTES_READ=288374
11/03/15 14:36:08 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=335470
11/03/15 14:36:08 INFO mapred.JobClient: Map-Reduce Framework
11/03/15 14:36:08 INFO mapred.JobClient: Map input records=600
11/03/15 14:36:08 INFO mapred.JobClient: Spilled Records=0
11/03/15 14:36:08 INFO mapred.JobClient: Map output records=600
11/03/15 14:36:08 INFO kmeans.Job: Running Canopy to get initial clusters
11/03/15 14:36:08 INFO canopy.CanopyDriver: Build Clusters Input: output/data Out: output Measure: org.apache.mahout.common.distance.EuclideanDistanceMeasure@5fcf29 t1: 80.0 t2: 55.0
11/03/15 14:36:08 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
11/03/15 14:36:08 INFO input.FileInputFormat: Total input paths to process : 1
11/03/15 14:36:09 INFO mapred.JobClient: Running job: job_201103111106_0087
11/03/15 14:36:10 INFO mapred.JobClient: map 0% reduce 0%
11/03/15 14:36:19 INFO mapred.JobClient: Task Id : attempt_201103111106_0087_m_000000_0, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:833)
at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:551)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:793)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:524)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
11/03/15 14:36:44 INFO mapred.JobClient: Job complete: job_201103111106_0087
11/03/15 14:36:44 INFO mapred.JobClient: Counters: 4
11/03/15 14:36:44 INFO mapred.JobClient: Job Counters
11/03/15 14:36:44 INFO mapred.JobClient: Rack-local map tasks=2
11/03/15 14:36:44 INFO mapred.JobClient: Launched map tasks=4
11/03/15 14:36:44 INFO mapred.JobClient: Data-local map tasks=2
11/03/15 14:36:44 INFO mapred.JobClient: Failed map tasks=1
Exception in thread "main" java.lang.InterruptedException: Canopy Job failed processing output/data
at org.apache.mahout.clustering.canopy.CanopyDriver.buildClustersMR(CanopyDriver.java:252)
at org.apache.mahout.clustering.canopy.CanopyDriver.buildClusters(CanopyDriver.java:167)
at org.apache.mahout.clustering.canopy.CanopyDriver.run(CanopyDriver.java:114)
at org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.run(Job.java:135)
at org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.main(Job.java:58)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
________________________________
Important notice: This e-mail and any attachment there to contains corporate proprietary information. If you have received it by mistake, please notify us immediately by reply e-mail and delete this e-mail and its attachments from your system.
Thank You.
RE: Error: java.lang.ClassNotFoundException:
org.apache.mahout.math.Vector
Posted by "Sengupta, Sohini IN BLR SISL" <so...@siemens.com>.
Hi,
I could execute the code now after executing "stop-all" and "start-all".
Thanks and regards,
Sohini
-----Original Message-----
From: Sean Owen [mailto:srowen@gmail.com]
Sent: Tuesday, March 15, 2011 9:56 PM
To: Sengupta, Sohini IN BLR SISL
Cc: user@mahout.apache.org
Subject: Re: Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector
I don't believe that's the right classpath. It's the classpath of the
workers that matters. This needs to be passed in the jar file that is
given to Hadoop to run on the workers.
On Tue, Mar 15, 2011 at 10:41 AM, Sengupta, Sohini IN BLR SISL
<so...@siemens.com> wrote:
> Hi,
>
> I had included following in hadoop-env.sh present in master and slave the
> following:
>
>
>
> export MAHOUT_JARS=
>
> MAHOUT_HOME=/usr/local/mahout-distribution-0.4
>
> #echo $MAHOUT_HOME
>
> # add release dependencies to CLASSPATH
>
> for f in $MAHOUT_HOME/mahout-*.jar; do
>
> MAHOUT_JARS=${MAHOUT_JARS}:$f;
>
> done
>
> # add dev targets if they exist
>
> for f in $MAHOUT_HOME/*/target/mahout-*-job.jar; do
>
> MAHOUT_JARS=${MAHOUT_JARS}:$f;
>
> done
>
> for f in $MAHOUT_HOME/lib/*.jar; do
>
> MAHOUT_JARS=${MAHOUT_JARS}:$f;
>
> #echo $f
>
> done
>
> # add development dependencies to CLASSPATH
>
> for f in $MAHOUT_HOME/examples/target/dependency/*.jar; do
>
> MAHOUT_JARS=${MAHOUT_JARS}:$f;
>
> done
>
> export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$MAHOUT_JARS
>
>
>
> I still get the same error.
>
> Thanks,
>
> Sohini
>
> ________________________________
>
> From: Sean Owen [mailto:srowen@gmail.com]
> Sent: Tuesday, March 15, 2011 2:56 PM
> To: user@mahout.apache.org
> Cc: Sengupta, Sohini IN BLR SISL
> Subject: Re: Error: java.lang.ClassNotFoundException:
> org.apache.mahout.math.Vector
>
>
>
> This indicates you still don't have mahout-math classes in the Hadoop worker
> classpath. So something is not set up correctly.
>
>
>
> This usually means you haven't packaged all Mahout classes into the jar you
> execute on Hadoop. Use the Mahout ".job" jar files and add your code to it.
>
> On Tue, Mar 15, 2011 at 9:13 AM, Sengupta, Sohini IN BLR SISL
> <so...@siemens.com> wrote:
>
> Hi all,
> I am encountering following error when I try to run mahout on a cluster. I
> have set the classpath to all dependencies in hadoop-env.sh.
>
> Any input/ pointers will be helpful.
> Thanks a lot in advance
> Sohini
>
> /usr/local/mahout-distribution-0.4$ /usr/local/hadoop/bin/hadoop jar
> /usr/local/mahout-distribution-0.4/examples/target/mahout-examples-0.4-job.jar
> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
> 11/03/15 14:35:50 INFO kmeans.Job: Running with default arguments
> 11/03/15 14:35:51 INFO common.HadoopUtil: Deleting output
> 11/03/15 14:35:51 INFO kmeans.Job: Preparing Input
> 11/03/15 14:35:51 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> 11/03/15 14:35:53 INFO input.FileInputFormat: Total input paths to process :
> 1
> 11/03/15 14:35:54 INFO mapred.JobClient: Running job: job_201103111106_0086
> 11/03/15 14:35:55 INFO mapred.JobClient: map 0% reduce 0%
> 11/03/15 14:36:05 INFO mapred.JobClient: map 100% reduce 0%
> 11/03/15 14:36:08 INFO mapred.JobClient: Job complete: job_201103111106_0086
> 11/03/15 14:36:08 INFO mapred.JobClient: Counters: 7
> 11/03/15 14:36:08 INFO mapred.JobClient: Job Counters
> 11/03/15 14:36:08 INFO mapred.JobClient: Rack-local map tasks=1
> 11/03/15 14:36:08 INFO mapred.JobClient: Launched map tasks=1
> 11/03/15 14:36:08 INFO mapred.JobClient: FileSystemCounters
> 11/03/15 14:36:08 INFO mapred.JobClient: HDFS_BYTES_READ=288374
> 11/03/15 14:36:08 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=335470
> 11/03/15 14:36:08 INFO mapred.JobClient: Map-Reduce Framework
> 11/03/15 14:36:08 INFO mapred.JobClient: Map input records=600
> 11/03/15 14:36:08 INFO mapred.JobClient: Spilled Records=0
> 11/03/15 14:36:08 INFO mapred.JobClient: Map output records=600
> 11/03/15 14:36:08 INFO kmeans.Job: Running Canopy to get initial clusters
> 11/03/15 14:36:08 INFO canopy.CanopyDriver: Build Clusters Input:
> output/data Out: output Measure:
> org.apache.mahout.common.distance.EuclideanDistanceMeasure@5fcf29 t1: 80.0
> t2: 55.0
> 11/03/15 14:36:08 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> 11/03/15 14:36:08 INFO input.FileInputFormat: Total input paths to process :
> 1
> 11/03/15 14:36:09 INFO mapred.JobClient: Running job: job_201103111106_0087
> 11/03/15 14:36:10 INFO mapred.JobClient: map 0% reduce 0%
> 11/03/15 14:36:19 INFO mapred.JobClient: Task Id :
> attempt_201103111106_0087_m_000000_0, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:833)
> at
> org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:551)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:793)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:524)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 11/03/15 14:36:44 INFO mapred.JobClient: Job complete: job_201103111106_0087
> 11/03/15 14:36:44 INFO mapred.JobClient: Counters: 4
> 11/03/15 14:36:44 INFO mapred.JobClient: Job Counters
> 11/03/15 14:36:44 INFO mapred.JobClient: Rack-local map tasks=2
> 11/03/15 14:36:44 INFO mapred.JobClient: Launched map tasks=4
> 11/03/15 14:36:44 INFO mapred.JobClient: Data-local map tasks=2
> 11/03/15 14:36:44 INFO mapred.JobClient: Failed map tasks=1
> Exception in thread "main" java.lang.InterruptedException: Canopy Job failed
> processing output/data
> at
> org.apache.mahout.clustering.canopy.CanopyDriver.buildClustersMR(CanopyDriver.java:252)
> at
> org.apache.mahout.clustering.canopy.CanopyDriver.buildClusters(CanopyDriver.java:167)
> at
> org.apache.mahout.clustering.canopy.CanopyDriver.run(CanopyDriver.java:114)
> at
> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.run(Job.java:135)
> at
> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.main(Job.java:58)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
> ________________________________
> Important notice: This e-mail and any attachment there to contains corporate
> proprietary information. If you have received it by mistake, please notify
> us immediately by reply e-mail and delete this e-mail and its attachments
> from your system.
> Thank You.
>
>
>
> ________________________________
> Important notice: This e-mail and any attachment there to contains corporate
> proprietary information. If you have received it by mistake, please notify
> us immediately by reply e-mail and delete this e-mail and its attachments
> from your system.
> Thank You.
>
Important notice: This e-mail and any attachment there to contains corporate proprietary information. If you have received it by mistake, please notify us immediately by reply e-mail and delete this e-mail and its attachments from your system.
Thank You.
Re: Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector
Posted by Sean Owen <sr...@gmail.com>.
I don't believe that's the right classpath. It's the classpath of the
workers that matters. This needs to be passed in the jar file that is
given to Hadoop to run on the workers.
On Tue, Mar 15, 2011 at 10:41 AM, Sengupta, Sohini IN BLR SISL
<so...@siemens.com> wrote:
> Hi,
>
> I had included following in hadoop-env.sh present in master and slave the
> following:
>
>
>
> export MAHOUT_JARS=
>
> MAHOUT_HOME=/usr/local/mahout-distribution-0.4
>
> #echo $MAHOUT_HOME
>
> # add release dependencies to CLASSPATH
>
> for f in $MAHOUT_HOME/mahout-*.jar; do
>
> MAHOUT_JARS=${MAHOUT_JARS}:$f;
>
> done
>
> # add dev targets if they exist
>
> for f in $MAHOUT_HOME/*/target/mahout-*-job.jar; do
>
> MAHOUT_JARS=${MAHOUT_JARS}:$f;
>
> done
>
> for f in $MAHOUT_HOME/lib/*.jar; do
>
> MAHOUT_JARS=${MAHOUT_JARS}:$f;
>
> #echo $f
>
> done
>
> # add development dependencies to CLASSPATH
>
> for f in $MAHOUT_HOME/examples/target/dependency/*.jar; do
>
> MAHOUT_JARS=${MAHOUT_JARS}:$f;
>
> done
>
> export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$MAHOUT_JARS
>
>
>
> I still get the same error.
>
> Thanks,
>
> Sohini
>
> ________________________________
>
> From: Sean Owen [mailto:srowen@gmail.com]
> Sent: Tuesday, March 15, 2011 2:56 PM
> To: user@mahout.apache.org
> Cc: Sengupta, Sohini IN BLR SISL
> Subject: Re: Error: java.lang.ClassNotFoundException:
> org.apache.mahout.math.Vector
>
>
>
> This indicates you still don't have mahout-math classes in the Hadoop worker
> classpath. So something is not set up correctly.
>
>
>
> This usually means you haven't packaged all Mahout classes into the jar you
> execute on Hadoop. Use the Mahout ".job" jar files and add your code to it.
>
> On Tue, Mar 15, 2011 at 9:13 AM, Sengupta, Sohini IN BLR SISL
> <so...@siemens.com> wrote:
>
> Hi all,
> I am encountering following error when I try to run mahout on a cluster. I
> have set the classpath to all dependencies in hadoop-env.sh.
>
> Any input/ pointers will be helpful.
> Thanks a lot in advance
> Sohini
>
> /usr/local/mahout-distribution-0.4$ /usr/local/hadoop/bin/hadoop jar
> /usr/local/mahout-distribution-0.4/examples/target/mahout-examples-0.4-job.jar
> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
> 11/03/15 14:35:50 INFO kmeans.Job: Running with default arguments
> 11/03/15 14:35:51 INFO common.HadoopUtil: Deleting output
> 11/03/15 14:35:51 INFO kmeans.Job: Preparing Input
> 11/03/15 14:35:51 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> 11/03/15 14:35:53 INFO input.FileInputFormat: Total input paths to process :
> 1
> 11/03/15 14:35:54 INFO mapred.JobClient: Running job: job_201103111106_0086
> 11/03/15 14:35:55 INFO mapred.JobClient: map 0% reduce 0%
> 11/03/15 14:36:05 INFO mapred.JobClient: map 100% reduce 0%
> 11/03/15 14:36:08 INFO mapred.JobClient: Job complete: job_201103111106_0086
> 11/03/15 14:36:08 INFO mapred.JobClient: Counters: 7
> 11/03/15 14:36:08 INFO mapred.JobClient: Job Counters
> 11/03/15 14:36:08 INFO mapred.JobClient: Rack-local map tasks=1
> 11/03/15 14:36:08 INFO mapred.JobClient: Launched map tasks=1
> 11/03/15 14:36:08 INFO mapred.JobClient: FileSystemCounters
> 11/03/15 14:36:08 INFO mapred.JobClient: HDFS_BYTES_READ=288374
> 11/03/15 14:36:08 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=335470
> 11/03/15 14:36:08 INFO mapred.JobClient: Map-Reduce Framework
> 11/03/15 14:36:08 INFO mapred.JobClient: Map input records=600
> 11/03/15 14:36:08 INFO mapred.JobClient: Spilled Records=0
> 11/03/15 14:36:08 INFO mapred.JobClient: Map output records=600
> 11/03/15 14:36:08 INFO kmeans.Job: Running Canopy to get initial clusters
> 11/03/15 14:36:08 INFO canopy.CanopyDriver: Build Clusters Input:
> output/data Out: output Measure:
> org.apache.mahout.common.distance.EuclideanDistanceMeasure@5fcf29 t1: 80.0
> t2: 55.0
> 11/03/15 14:36:08 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> 11/03/15 14:36:08 INFO input.FileInputFormat: Total input paths to process :
> 1
> 11/03/15 14:36:09 INFO mapred.JobClient: Running job: job_201103111106_0087
> 11/03/15 14:36:10 INFO mapred.JobClient: map 0% reduce 0%
> 11/03/15 14:36:19 INFO mapred.JobClient: Task Id :
> attempt_201103111106_0087_m_000000_0, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:833)
> at
> org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:551)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:793)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:524)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 11/03/15 14:36:44 INFO mapred.JobClient: Job complete: job_201103111106_0087
> 11/03/15 14:36:44 INFO mapred.JobClient: Counters: 4
> 11/03/15 14:36:44 INFO mapred.JobClient: Job Counters
> 11/03/15 14:36:44 INFO mapred.JobClient: Rack-local map tasks=2
> 11/03/15 14:36:44 INFO mapred.JobClient: Launched map tasks=4
> 11/03/15 14:36:44 INFO mapred.JobClient: Data-local map tasks=2
> 11/03/15 14:36:44 INFO mapred.JobClient: Failed map tasks=1
> Exception in thread "main" java.lang.InterruptedException: Canopy Job failed
> processing output/data
> at
> org.apache.mahout.clustering.canopy.CanopyDriver.buildClustersMR(CanopyDriver.java:252)
> at
> org.apache.mahout.clustering.canopy.CanopyDriver.buildClusters(CanopyDriver.java:167)
> at
> org.apache.mahout.clustering.canopy.CanopyDriver.run(CanopyDriver.java:114)
> at
> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.run(Job.java:135)
> at
> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.main(Job.java:58)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
> ________________________________
> Important notice: This e-mail and any attachment there to contains corporate
> proprietary information. If you have received it by mistake, please notify
> us immediately by reply e-mail and delete this e-mail and its attachments
> from your system.
> Thank You.
>
>
>
> ________________________________
> Important notice: This e-mail and any attachment there to contains corporate
> proprietary information. If you have received it by mistake, please notify
> us immediately by reply e-mail and delete this e-mail and its attachments
> from your system.
> Thank You.
>
RE: Error: java.lang.ClassNotFoundException:
org.apache.mahout.math.Vector
Posted by "Sengupta, Sohini IN BLR SISL" <so...@siemens.com>.
Hi,
I had included following in hadoop-env.sh present in master and slave the following:
export MAHOUT_JARS=
MAHOUT_HOME=/usr/local/mahout-distribution-0.4
#echo $MAHOUT_HOME
# add release dependencies to CLASSPATH
for f in $MAHOUT_HOME/mahout-*.jar; do
MAHOUT_JARS=${MAHOUT_JARS}:$f;
done
# add dev targets if they exist
for f in $MAHOUT_HOME/*/target/mahout-*-job.jar; do
MAHOUT_JARS=${MAHOUT_JARS}:$f;
done
for f in $MAHOUT_HOME/lib/*.jar; do
MAHOUT_JARS=${MAHOUT_JARS}:$f;
#echo $f
done
# add development dependencies to CLASSPATH
for f in $MAHOUT_HOME/examples/target/dependency/*.jar; do
MAHOUT_JARS=${MAHOUT_JARS}:$f;
done
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$MAHOUT_JARS
I still get the same error.
Thanks,
Sohini
________________________________
From: Sean Owen [mailto:srowen@gmail.com]
Sent: Tuesday, March 15, 2011 2:56 PM
To: user@mahout.apache.org
Cc: Sengupta, Sohini IN BLR SISL
Subject: Re: Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector
This indicates you still don't have mahout-math classes in the Hadoop worker classpath. So something is not set up correctly.
This usually means you haven't packaged all Mahout classes into the jar you execute on Hadoop. Use the Mahout ".job" jar files and add your code to it.
On Tue, Mar 15, 2011 at 9:13 AM, Sengupta, Sohini IN BLR SISL <so...@siemens.com>> wrote:
Hi all,
I am encountering following error when I try to run mahout on a cluster. I have set the classpath to all dependencies in hadoop-env.sh.
Any input/ pointers will be helpful.
Thanks a lot in advance
Sohini
/usr/local/mahout-distribution-0.4$ /usr/local/hadoop/bin/hadoop jar /usr/local/mahout-distribution-0.4/examples/target/mahout-examples-0.4-job.jar org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
11/03/15 14:35:50 INFO kmeans.Job: Running with default arguments
11/03/15 14:35:51 INFO common.HadoopUtil: Deleting output
11/03/15 14:35:51 INFO kmeans.Job: Preparing Input
11/03/15 14:35:51 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
11/03/15 14:35:53 INFO input.FileInputFormat: Total input paths to process : 1
11/03/15 14:35:54 INFO mapred.JobClient: Running job: job_201103111106_0086
11/03/15 14:35:55 INFO mapred.JobClient: map 0% reduce 0%
11/03/15 14:36:05 INFO mapred.JobClient: map 100% reduce 0%
11/03/15 14:36:08 INFO mapred.JobClient: Job complete: job_201103111106_0086
11/03/15 14:36:08 INFO mapred.JobClient: Counters: 7
11/03/15 14:36:08 INFO mapred.JobClient: Job Counters
11/03/15 14:36:08 INFO mapred.JobClient: Rack-local map tasks=1
11/03/15 14:36:08 INFO mapred.JobClient: Launched map tasks=1
11/03/15 14:36:08 INFO mapred.JobClient: FileSystemCounters
11/03/15 14:36:08 INFO mapred.JobClient: HDFS_BYTES_READ=288374
11/03/15 14:36:08 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=335470
11/03/15 14:36:08 INFO mapred.JobClient: Map-Reduce Framework
11/03/15 14:36:08 INFO mapred.JobClient: Map input records=600
11/03/15 14:36:08 INFO mapred.JobClient: Spilled Records=0
11/03/15 14:36:08 INFO mapred.JobClient: Map output records=600
11/03/15 14:36:08 INFO kmeans.Job: Running Canopy to get initial clusters
11/03/15 14:36:08 INFO canopy.CanopyDriver: Build Clusters Input: output/data Out: output Measure: org.apache.mahout.common.distance.EuclideanDistanceMeasure@5fcf29 t1: 80.0 t2: 55.0
11/03/15 14:36:08 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
11/03/15 14:36:08 INFO input.FileInputFormat: Total input paths to process : 1
11/03/15 14:36:09 INFO mapred.JobClient: Running job: job_201103111106_0087
11/03/15 14:36:10 INFO mapred.JobClient: map 0% reduce 0%
11/03/15 14:36:19 INFO mapred.JobClient: Task Id : attempt_201103111106_0087_m_000000_0, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:833)
at org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:551)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:793)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:524)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
11/03/15 14:36:44 INFO mapred.JobClient: Job complete: job_201103111106_0087
11/03/15 14:36:44 INFO mapred.JobClient: Counters: 4
11/03/15 14:36:44 INFO mapred.JobClient: Job Counters
11/03/15 14:36:44 INFO mapred.JobClient: Rack-local map tasks=2
11/03/15 14:36:44 INFO mapred.JobClient: Launched map tasks=4
11/03/15 14:36:44 INFO mapred.JobClient: Data-local map tasks=2
11/03/15 14:36:44 INFO mapred.JobClient: Failed map tasks=1
Exception in thread "main" java.lang.InterruptedException: Canopy Job failed processing output/data
at org.apache.mahout.clustering.canopy.CanopyDriver.buildClustersMR(CanopyDriver.java:252)
at org.apache.mahout.clustering.canopy.CanopyDriver.buildClusters(CanopyDriver.java:167)
at org.apache.mahout.clustering.canopy.CanopyDriver.run(CanopyDriver.java:114)
at org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.run(Job.java:135)
at org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.main(Job.java:58)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
________________________________
Important notice: This e-mail and any attachment there to contains corporate proprietary information. If you have received it by mistake, please notify us immediately by reply e-mail and delete this e-mail and its attachments from your system.
Thank You.
________________________________
Important notice: This e-mail and any attachment there to contains corporate proprietary information. If you have received it by mistake, please notify us immediately by reply e-mail and delete this e-mail and its attachments from your system.
Thank You.
Re: Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector
Posted by Sean Owen <sr...@gmail.com>.
This indicates you still don't have mahout-math classes in the Hadoop worker
classpath. So something is not set up correctly.
This usually means you haven't packaged all Mahout classes into the jar you
execute on Hadoop. Use the Mahout ".job" jar files and add your code to it.
On Tue, Mar 15, 2011 at 9:13 AM, Sengupta, Sohini IN BLR SISL <
sohini.sengupta@siemens.com> wrote:
> Hi all,
> I am encountering following error when I try to run mahout on a cluster. I
> have set the classpath to all dependencies in hadoop-env.sh.
>
> Any input/ pointers will be helpful.
> Thanks a lot in advance
> Sohini
>
> /usr/local/mahout-distribution-0.4$ /usr/local/hadoop/bin/hadoop jar
> /usr/local/mahout-distribution-0.4/examples/target/mahout-examples-0.4-job.jar
> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
> 11/03/15 14:35:50 INFO kmeans.Job: Running with default arguments
> 11/03/15 14:35:51 INFO common.HadoopUtil: Deleting output
> 11/03/15 14:35:51 INFO kmeans.Job: Preparing Input
> 11/03/15 14:35:51 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> 11/03/15 14:35:53 INFO input.FileInputFormat: Total input paths to process
> : 1
> 11/03/15 14:35:54 INFO mapred.JobClient: Running job: job_201103111106_0086
> 11/03/15 14:35:55 INFO mapred.JobClient: map 0% reduce 0%
> 11/03/15 14:36:05 INFO mapred.JobClient: map 100% reduce 0%
> 11/03/15 14:36:08 INFO mapred.JobClient: Job complete:
> job_201103111106_0086
> 11/03/15 14:36:08 INFO mapred.JobClient: Counters: 7
> 11/03/15 14:36:08 INFO mapred.JobClient: Job Counters
> 11/03/15 14:36:08 INFO mapred.JobClient: Rack-local map tasks=1
> 11/03/15 14:36:08 INFO mapred.JobClient: Launched map tasks=1
> 11/03/15 14:36:08 INFO mapred.JobClient: FileSystemCounters
> 11/03/15 14:36:08 INFO mapred.JobClient: HDFS_BYTES_READ=288374
> 11/03/15 14:36:08 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=335470
> 11/03/15 14:36:08 INFO mapred.JobClient: Map-Reduce Framework
> 11/03/15 14:36:08 INFO mapred.JobClient: Map input records=600
> 11/03/15 14:36:08 INFO mapred.JobClient: Spilled Records=0
> 11/03/15 14:36:08 INFO mapred.JobClient: Map output records=600
> 11/03/15 14:36:08 INFO kmeans.Job: Running Canopy to get initial clusters
> 11/03/15 14:36:08 INFO canopy.CanopyDriver: Build Clusters Input:
> output/data Out: output Measure:
> org.apache.mahout.common.distance.EuclideanDistanceMeasure@5fcf29 t1: 80.0
> t2: 55.0
> 11/03/15 14:36:08 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> 11/03/15 14:36:08 INFO input.FileInputFormat: Total input paths to process
> : 1
> 11/03/15 14:36:09 INFO mapred.JobClient: Running job: job_201103111106_0087
> 11/03/15 14:36:10 INFO mapred.JobClient: map 0% reduce 0%
> 11/03/15 14:36:19 INFO mapred.JobClient: Task Id :
> attempt_201103111106_0087_m_000000_0, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:833)
> at
> org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:551)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:793)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:524)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 11/03/15 14:36:44 INFO mapred.JobClient: Job complete:
> job_201103111106_0087
> 11/03/15 14:36:44 INFO mapred.JobClient: Counters: 4
> 11/03/15 14:36:44 INFO mapred.JobClient: Job Counters
> 11/03/15 14:36:44 INFO mapred.JobClient: Rack-local map tasks=2
> 11/03/15 14:36:44 INFO mapred.JobClient: Launched map tasks=4
> 11/03/15 14:36:44 INFO mapred.JobClient: Data-local map tasks=2
> 11/03/15 14:36:44 INFO mapred.JobClient: Failed map tasks=1
> Exception in thread "main" java.lang.InterruptedException: Canopy Job
> failed processing output/data
> at
> org.apache.mahout.clustering.canopy.CanopyDriver.buildClustersMR(CanopyDriver.java:252)
> at
> org.apache.mahout.clustering.canopy.CanopyDriver.buildClusters(CanopyDriver.java:167)
> at
> org.apache.mahout.clustering.canopy.CanopyDriver.run(CanopyDriver.java:114)
> at
> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.run(Job.java:135)
> at
> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.main(Job.java:58)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
> ________________________________
> Important notice: This e-mail and any attachment there to contains
> corporate proprietary information. If you have received it by mistake,
> please notify us immediately by reply e-mail and delete this e-mail and its
> attachments from your system.
> Thank You.
>