You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Anbarasan Murthy <an...@hotmail.com> on 2013/02/04 16:18:46 UTC
Mahout KMeans - java.lang.NoSuchMethodError:
com.google.common.collect.Iterators.forArray([Ljava/lang/Object;)Lcom/google/common/collect/UnmodifiableIterator
I am getting the following exception when i tried executing the Mahout KMeans algorithm.
$ $MAHOUT_HOME/bin/mahout org.apache.mahout.clustering.syntheticcontrol.kmeans.JobWarning: $HADOOP_HOME is deprecated.
Running on hadoop, using /data/hadoop-1.0.1/bin/hadoop and HADOOP_CONF_DIR=MAHOUT-JOB: /home/hadoop/anbu/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jarWarning: $HADOOP_HOME is deprecated.
13/02/04 15:09:24 WARN driver.MahoutDriver: No org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.props found on classpath, will use command-line arguments only13/02/04 15:09:24 INFO kmeans.Job: Running with default arguments13/02/04 15:09:27 INFO common.HadoopUtil: Deleting output13/02/04 15:09:27 INFO kmeans.Job: Preparing Input13/02/04 15:09:27 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.****hdfs://localhost:54310/user/hadoop/testdata13/02/04 15:09:30 INFO input.FileInputFormat: Total input paths to process : 113/02/04 15:09:31 INFO mapred.JobClient: Running job: job_201302041411_000313/02/04 15:09:32 INFO mapred.JobClient: map 0% reduce 0%13/02/04 15:10:10 INFO mapred.JobClient: map 100% reduce 0%13/02/04 15:10:21 INFO mapred.JobClient: Job complete: job_201302041411_000313/02/04 15:10:21 INFO mapred.JobClient: Counters: 1913/02/04 15:10:21 INFO mapred.JobClient: Job Counters13/02/04 15:10:21 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=3262113/02/04 15:10:21 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=013/02/04 15:10:21 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=013/02/04 15:10:21 INFO mapred.JobClient: Launched map tasks=113/02/04 15:10:21 INFO mapred.JobClient: Data-local map tasks=113/02/04 15:10:21 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=013/02/04 15:10:21 INFO mapred.JobClient: File Output Format Counters13/02/04 15:10:21 INFO mapred.JobClient: Bytes Written=33547013/02/04 15:10:21 INFO mapred.JobClient: FileSystemCounters13/02/04 15:10:21 INFO mapred.JobClient: HDFS_BYTES_READ=28850513/02/04 15:10:21 INFO mapred.JobClient: FILE_BYTES_WRITTEN=2143413/02/04 15:10:21 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=33547013/02/04 15:10:21 INFO mapred.JobClient: File Input Format Counters13/02/04 15:10:21 INFO mapred.JobClient: Bytes Read=28837413/02/04 15:10:21 INFO mapred.JobClient: Map-Reduce Framework13/02/04 15:10:21 INFO mapred.JobClient: Map input records=60013/02/04 15:10:21 INFO mapred.JobClient: Physical memory (bytes) snapshot=6834995213/02/04 15:10:21 INFO mapred.JobClient: Spilled Records=013/02/04 15:10:21 INFO mapred.JobClient: CPU time spent (ms)=338013/02/04 15:10:21 INFO mapred.JobClient: Total committed heap usage (bytes)=2634547213/02/04 15:10:21 INFO mapred.JobClient: Virtual memory (bytes) snapshot=192717619213/02/04 15:10:21 INFO mapred.JobClient: Map output records=60013/02/04 15:10:21 INFO mapred.JobClient: SPLIT_RAW_BYTES=13113/02/04 15:10:21 INFO kmeans.Job: Running random seed to get initial clusters13/02/04 15:10:21 INFO util.NativeCodeLoader: Loaded the native-hadoop library13/02/04 15:10:21 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library13/02/04 15:10:21 INFO compress.CodecPool: Got brand-new compressor13/02/04 15:10:24 INFO kmeans.RandomSeedGenerator: Wrote 6 Klusters to output/clusters-0/part-randomSeed13/02/04 15:10:24 INFO kmeans.Job: Running KMeans13/02/04 15:10:24 INFO kmeans.KMeansDriver: Input: output/data Clusters In: output/clusters-0/part-randomSeed Out: output Distance: org.apache.mahout.common.distance.EuclideanDistanceMeasure13/02/04 15:10:24 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations: 10 num Reduce Tasks: org.apache.mahout.math.VectorWritable Input Vectors: {}Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.collect.Iterators.forArray([Ljava/lang/Object;)Lcom/google/common/collect/UnmodifiableIterator; at org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator.init(SequenceFileDirValueIterator.java:114) at org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator.<init>(SequenceFileDirValueIterator.java:73) at org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterable.iterator(SequenceFileDirValueIterable.java:76) at org.apache.mahout.clustering.kmeans.KMeansUtil.configureWithClusterInfo(KMeansUtil.java:51) at org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:215) at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:149) at org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.run(Job.java:134) at org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.main(Job.java:59) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Thanks,Anbu.
RE: Mahout KMeans - java.lang.NoSuchMethodError:
com.google.common.collect.Iterators.forArray([Ljava/lang/Object;)Lcom/google/common/collect/UnmodifiableIterator
Posted by Anbarasan Murthy <an...@hotmail.com>.
Hi Mahesh,
Thanks for your reply.
I am executing the Mahout api example - KMeans algorithm
https://cwiki.apache.org/MAHOUT/clustering-of-synthetic-control-data.html
$MAHOUT_HOME/bin/mahout org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
I am using the following VersionHadoop 1.0.1Mahout 0.7
Thanks,Anbu
> Date: Mon, 4 Feb 2013 21:26:14 +0530
> Subject: Re: Mahout KMeans - java.lang.NoSuchMethodError: com.google.common.collect.Iterators.forArray([Ljava/lang/Object;)Lcom/google/common/collect/UnmodifiableIterator
> From: balijamahesh.mca@gmail.com
> To: user@mahout.apache.org
>
> Hi Murthy,
>
> It seems to be NOT a mahout's issue to me, rather I suspect
> that you might have compiled your source with some latest/legacy jars and
> running in a legacy environment where you don't have latest API.
>
> Best,
> Mahesh Balija,
> CalsoftLabs.
>
> On Mon, Feb 4, 2013 at 8:48 PM, Anbarasan Murthy <an...@hotmail.com>wrote:
>
> > I am getting the following exception when i tried executing the Mahout
> > KMeans algorithm.
> > $ $MAHOUT_HOME/bin/mahout
> > org.apache.mahout.clustering.syntheticcontrol.kmeans.JobWarning:
> > $HADOOP_HOME is deprecated.
> > Running on hadoop, using /data/hadoop-1.0.1/bin/hadoop and
> > HADOOP_CONF_DIR=MAHOUT-JOB:
> > /home/hadoop/anbu/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jarWarning:
> > $HADOOP_HOME is deprecated.
> > 13/02/04 15:09:24 WARN driver.MahoutDriver: No
> > org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.props found on
> > classpath, will use command-line arguments only13/02/04 15:09:24 INFO
> > kmeans.Job: Running with default arguments13/02/04 15:09:27 INFO
> > common.HadoopUtil: Deleting output13/02/04 15:09:27 INFO kmeans.Job:
> > Preparing Input13/02/04 15:09:27 WARN mapred.JobClient: Use
> > GenericOptionsParser for parsing the arguments. Applications should
> > implement Tool for the
> > same.****hdfs://localhost:54310/user/hadoop/testdata13/02/04 15:09:30 INFO
> > input.FileInputFormat: Total input paths to process : 113/02/04 15:09:31
> > INFO mapred.JobClient: Running job: job_201302041411_000313/02/04 15:09:32
> > INFO mapred.JobClient: map 0% reduce 0%13/02/04 15:10:10 INFO
> > mapred.JobClient: map 100% reduce 0%13/02/04 15:10:21 INFO
> > mapred.JobClient: Job complete: job_201302041411_000313/02/04 15:10:21 INFO
> > mapred.JobClient: Counters: 1913/02/04 15:10:21 INFO mapred.JobClient:
> > Job Counters13/02/04 15:10:21 INFO mapred.JobClient:
> > SLOTS_MILLIS_MAPS=3262113/02/04 15:10:21 INFO mapred.JobClient: Total
> > time spent by all reduces waiting after reserving slots (ms)=013/02/04
> > 15:10:21 INFO mapred.JobClient: Total time spent by all maps waiting
> > after reserving slots (ms)=013/02/04 15:10:21 INFO mapred.JobClient:
> > Launched map tasks=113/02/04 15:10:21 INFO mapred.JobClient: Data-local
> > map tasks=113/02/04 15:10:21 INFO mapred.JobClient:
> > SLOTS_MILLIS_REDUCES=013/02/04 15:10:21 INFO mapred.JobClient: File
> > Output Format Counters13/02/04 15:10:21 INFO mapred.JobClient: Bytes
> > Written=33547013/02/04 15:10:21 INFO mapred.JobClient:
> > FileSystemCounters13/02/04 15:10:21 INFO mapred.JobClient:
> > HDFS_BYTES_READ=28850513/02/04 15:10:21 INFO mapred.JobClient:
> > FILE_BYTES_WRITTEN=2143413/02/04 15:10:21 INFO mapred.JobClient:
> > HDFS_BYTES_WRITTEN=33547013/02/04 15:10:21 INFO mapred.JobClient: File
> > Input Format Counters13/02/04 15:10:21 INFO mapred.JobClient: Bytes
> > Read=28837413/02/04 15:10:21 INFO mapred.JobClient: Map-Reduce
> > Framework13/02/04 15:10:21 INFO mapred.JobClient: Map input
> > records=60013/02/04 15:10:21 INFO mapred.JobClient: Physical memory
> > (bytes) snapshot=6834995213/02/04 15:10:21 INFO mapred.JobClient:
> > Spilled Records=013/02/04 15:10:21 INFO mapred.JobClient: CPU time
> > spent (ms)=338013/02/04 15:10:21 INFO mapred.JobClient: Total committed
> > heap usage (bytes)=2634547213/02/04 15:10:21 INFO mapred.JobClient:
> > Virtual memory (bytes) snapshot=192717619213/02/04 15:10:21 INFO
> > mapred.JobClient: Map output records=60013/02/04 15:10:21 INFO
> > mapred.JobClient: SPLIT_RAW_BYTES=13113/02/04 15:10:21 INFO kmeans.Job:
> > Running random seed to get initial clusters13/02/04 15:10:21 INFO
> > util.NativeCodeLoader: Loaded the native-hadoop library13/02/04 15:10:21
> > INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib
> > library13/02/04 15:10:21 INFO compress.CodecPool: Got brand-new
> > compressor13/02/04 15:10:24 INFO kmeans.RandomSeedGenerator: Wrote 6
> > Klusters to output/clusters-0/part-randomSeed13/02/04 15:10:24 INFO
> > kmeans.Job: Running KMeans13/02/04 15:10:24 INFO kmeans.KMeansDriver:
> > Input: output/data Clusters In: output/clusters-0/part-randomSeed Out:
> > output Distance:
> > org.apache.mahout.common.distance.EuclideanDistanceMeasure13/02/04 15:10:24
> > INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations: 10 num Reduce
> > Tasks: org.apache.mahout.math.VectorWritable Input Vectors: {}Exception in
> > thread "main" java.lang.NoSuchMethodError:
> > com.google.common.collect.Iterators.forArray([Ljava/lang/Object;)Lcom/google/common/collect/UnmodifiableIterator;
> > at
> > org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator.init(SequenceFileDirValueIterator.java:114)
> > at
> > org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator.<init>(SequenceFileDirValueIterator.java:73)
> > at
> > org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterable.iterator(SequenceFileDirValueIterable.java:76)
> > at
> > org.apache.mahout.clustering.kmeans.KMeansUtil.configureWithClusterInfo(KMeansUtil.java:51)
> > at
> > org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:215)
> > at
> > org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:149)
> > at
> > org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.run(Job.java:134)
> > at
> > org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.main(Job.java:59)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:601) at
> > org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
> > at
> > org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
> > at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:601) at
> > org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> >
> >
> > Thanks,Anbu.
Re: Mahout KMeans - java.lang.NoSuchMethodError: com.google.common.collect.Iterators.forArray([Ljava/lang/Object;)Lcom/google/common/collect/UnmodifiableIterator
Posted by Mahesh Balija <ba...@gmail.com>.
Hi Murthy,
It seems to be NOT a mahout's issue to me, rather I suspect
that you might have compiled your source with some latest/legacy jars and
running in a legacy environment where you don't have latest API.
Best,
Mahesh Balija,
CalsoftLabs.
On Mon, Feb 4, 2013 at 8:48 PM, Anbarasan Murthy <an...@hotmail.com>wrote:
> I am getting the following exception when i tried executing the Mahout
> KMeans algorithm.
> $ $MAHOUT_HOME/bin/mahout
> org.apache.mahout.clustering.syntheticcontrol.kmeans.JobWarning:
> $HADOOP_HOME is deprecated.
> Running on hadoop, using /data/hadoop-1.0.1/bin/hadoop and
> HADOOP_CONF_DIR=MAHOUT-JOB:
> /home/hadoop/anbu/mahout/mahout-distribution-0.7/mahout-examples-0.7-job.jarWarning:
> $HADOOP_HOME is deprecated.
> 13/02/04 15:09:24 WARN driver.MahoutDriver: No
> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.props found on
> classpath, will use command-line arguments only13/02/04 15:09:24 INFO
> kmeans.Job: Running with default arguments13/02/04 15:09:27 INFO
> common.HadoopUtil: Deleting output13/02/04 15:09:27 INFO kmeans.Job:
> Preparing Input13/02/04 15:09:27 WARN mapred.JobClient: Use
> GenericOptionsParser for parsing the arguments. Applications should
> implement Tool for the
> same.****hdfs://localhost:54310/user/hadoop/testdata13/02/04 15:09:30 INFO
> input.FileInputFormat: Total input paths to process : 113/02/04 15:09:31
> INFO mapred.JobClient: Running job: job_201302041411_000313/02/04 15:09:32
> INFO mapred.JobClient: map 0% reduce 0%13/02/04 15:10:10 INFO
> mapred.JobClient: map 100% reduce 0%13/02/04 15:10:21 INFO
> mapred.JobClient: Job complete: job_201302041411_000313/02/04 15:10:21 INFO
> mapred.JobClient: Counters: 1913/02/04 15:10:21 INFO mapred.JobClient:
> Job Counters13/02/04 15:10:21 INFO mapred.JobClient:
> SLOTS_MILLIS_MAPS=3262113/02/04 15:10:21 INFO mapred.JobClient: Total
> time spent by all reduces waiting after reserving slots (ms)=013/02/04
> 15:10:21 INFO mapred.JobClient: Total time spent by all maps waiting
> after reserving slots (ms)=013/02/04 15:10:21 INFO mapred.JobClient:
> Launched map tasks=113/02/04 15:10:21 INFO mapred.JobClient: Data-local
> map tasks=113/02/04 15:10:21 INFO mapred.JobClient:
> SLOTS_MILLIS_REDUCES=013/02/04 15:10:21 INFO mapred.JobClient: File
> Output Format Counters13/02/04 15:10:21 INFO mapred.JobClient: Bytes
> Written=33547013/02/04 15:10:21 INFO mapred.JobClient:
> FileSystemCounters13/02/04 15:10:21 INFO mapred.JobClient:
> HDFS_BYTES_READ=28850513/02/04 15:10:21 INFO mapred.JobClient:
> FILE_BYTES_WRITTEN=2143413/02/04 15:10:21 INFO mapred.JobClient:
> HDFS_BYTES_WRITTEN=33547013/02/04 15:10:21 INFO mapred.JobClient: File
> Input Format Counters13/02/04 15:10:21 INFO mapred.JobClient: Bytes
> Read=28837413/02/04 15:10:21 INFO mapred.JobClient: Map-Reduce
> Framework13/02/04 15:10:21 INFO mapred.JobClient: Map input
> records=60013/02/04 15:10:21 INFO mapred.JobClient: Physical memory
> (bytes) snapshot=6834995213/02/04 15:10:21 INFO mapred.JobClient:
> Spilled Records=013/02/04 15:10:21 INFO mapred.JobClient: CPU time
> spent (ms)=338013/02/04 15:10:21 INFO mapred.JobClient: Total committed
> heap usage (bytes)=2634547213/02/04 15:10:21 INFO mapred.JobClient:
> Virtual memory (bytes) snapshot=192717619213/02/04 15:10:21 INFO
> mapred.JobClient: Map output records=60013/02/04 15:10:21 INFO
> mapred.JobClient: SPLIT_RAW_BYTES=13113/02/04 15:10:21 INFO kmeans.Job:
> Running random seed to get initial clusters13/02/04 15:10:21 INFO
> util.NativeCodeLoader: Loaded the native-hadoop library13/02/04 15:10:21
> INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib
> library13/02/04 15:10:21 INFO compress.CodecPool: Got brand-new
> compressor13/02/04 15:10:24 INFO kmeans.RandomSeedGenerator: Wrote 6
> Klusters to output/clusters-0/part-randomSeed13/02/04 15:10:24 INFO
> kmeans.Job: Running KMeans13/02/04 15:10:24 INFO kmeans.KMeansDriver:
> Input: output/data Clusters In: output/clusters-0/part-randomSeed Out:
> output Distance:
> org.apache.mahout.common.distance.EuclideanDistanceMeasure13/02/04 15:10:24
> INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations: 10 num Reduce
> Tasks: org.apache.mahout.math.VectorWritable Input Vectors: {}Exception in
> thread "main" java.lang.NoSuchMethodError:
> com.google.common.collect.Iterators.forArray([Ljava/lang/Object;)Lcom/google/common/collect/UnmodifiableIterator;
> at
> org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator.init(SequenceFileDirValueIterator.java:114)
> at
> org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterator.<init>(SequenceFileDirValueIterator.java:73)
> at
> org.apache.mahout.common.iterator.sequencefile.SequenceFileDirValueIterable.iterator(SequenceFileDirValueIterable.java:76)
> at
> org.apache.mahout.clustering.kmeans.KMeansUtil.configureWithClusterInfo(KMeansUtil.java:51)
> at
> org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:215)
> at
> org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:149)
> at
> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.run(Job.java:134)
> at
> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job.main(Job.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601) at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
> at
> org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
> at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601) at
> org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
>
> Thanks,Anbu.