You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Delroy Cameron <de...@gmail.com> on 2010/05/06 08:01:25 UTC
k-means InvocationTargetException
i ran k-means clustering a few days ago without any problems...did and svn
update to get the latest trunk and now the same exact command that i ran
before gives me this.
InvocationTargetException..i don't remember adding any conflicts..what is
even more puzzling is that Dirichlet Clustering runs just fine...
hadoop jar /mahout/core/target/mahout-core-0.4-SNAPSHOT.job
org.apache.mahout.clustering.kmeans.KMeansDriver -i
input-data-vectors/vectors/part-00000 -o input-data-kmeans-clusters -c
clusters -m org.apache.mahout.common.distance.CosineDistanceMeasure -v
org.apache.mahout.matrix.SparseVector -x 20 -d 0.5 -k 26 -w -r 8
10/05/06 01:59:55 WARN common.HadoopUtil: Deleting
input-data-kmeans-clusters
10/05/06 01:59:55 WARN common.HadoopUtil: Creating dir
input-data-kmeans-clusters
10/05/06 01:59:55 INFO util.NativeCodeLoader: Loaded the native-hadoop
library
10/05/06 01:59:55 INFO zlib.ZlibFactory: Successfully loaded & initialized
native-zlib library
10/05/06 01:59:55 INFO compress.CodecPool: Got brand-new compressor
10/05/06 01:59:58 INFO kmeans.RandomSeedGenerator: Wrote 26 vectors to
clusters/part-randomSeed
10/05/06 01:59:58 INFO kmeans.KMeansDriver: Input:
input-data-vectors/vectors/part-00000 Clusters In: clusters/part-randomSeed
Out: input-data-kmeans-clusters Distance:
org.apache.mahout.common.distance.CosineDistanceMeasure
10/05/06 01:59:58 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations:
20 num Reduce Tasks: 8 Input Vectors: org.apache.mahout.math.VectorWritable
10/05/06 01:59:58 INFO kmeans.KMeansDriver: Iteration 0
10/05/06 01:59:58 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
10/05/06 01:59:59 INFO mapred.FileInputFormat: Total input paths to process
: 1
10/05/06 01:59:59 INFO mapred.JobClient: Running job: job_201004230623_0119
10/05/06 02:00:00 INFO mapred.JobClient: map 0% reduce 0%
10/05/06 02:00:09 INFO mapred.JobClient: Task Id :
attempt_201004230623_0119_m_000000_0, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 5 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 13 more
Caused by: java.lang.IllegalStateException: Cluster is empty!
at
org.apache.mahout.clustering.kmeans.KMeansMapper.configure(KMeansMapper.java:73)
... 18 more
10/05/06 02:00:11 INFO mapred.JobClient: Task Id :
attempt_201004230623_0119_m_000001_0, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 5 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 13 more
Caused by: java.lang.IllegalStateException: Cluster is empty!
at
org.apache.mahout.clustering.kmeans.KMeansMapper.configure(KMeansMapper.java:73)
... 18 more
-----
--cheers
Delroy
--
View this message in context: http://lucene.472066.n3.nabble.com/k-means-InvocationTargetException-tp780778p780778.html
Sent from the Mahout User List mailing list archive at Nabble.com.
Re: k-means InvocationTargetException
Posted by Jeff Eastman <jd...@windwardsolutions.com>.
I just ran this on an EC2 machine built from trunk yesterday:
$HADOOP_HOME/bin/hadoop jar
$MAHOUT_HOME/examples/target/mahout-examples-$MAHOUT_VERSION.job
org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
and got this:
root@ip-10-243-90-4:~# $HADOOP_HOME/bin/hadoop fs -ls outputFound 13 items
drwxr-xr-x - root supergroup 0 2010-05-13 18:14
/user/root/output/clusteredPoints
drwxr-xr-x - root supergroup 0 2010-05-13 18:10
/user/root/output/clusters-0
drwxr-xr-x - root supergroup 0 2010-05-13 18:10
/user/root/output/clusters-1
drwxr-xr-x - root supergroup 0 2010-05-13 18:14
/user/root/output/clusters-10
drwxr-xr-x - root supergroup 0 2010-05-13 18:11
/user/root/output/clusters-2
drwxr-xr-x - root supergroup 0 2010-05-13 18:11
/user/root/output/clusters-3
drwxr-xr-x - root supergroup 0 2010-05-13 18:12
/user/root/output/clusters-4
drwxr-xr-x - root supergroup 0 2010-05-13 18:12
/user/root/output/clusters-5
drwxr-xr-x - root supergroup 0 2010-05-13 18:12
/user/root/output/clusters-6
drwxr-xr-x - root supergroup 0 2010-05-13 18:13
/user/root/output/clusters-7
drwxr-xr-x - root supergroup 0 2010-05-13 18:13
/user/root/output/clusters-8
drwxr-xr-x - root supergroup 0 2010-05-13 18:14
/user/root/output/clusters-9
drwxr-xr-x - root supergroup 0 2010-05-13 18:10
/user/root/output/data
I really don't know what's happening on your installation.
On 5/13/10 11:05 AM, Jeff Eastman wrote:
> It's not finding your initial clusters. What's in output/clusters-0?
> Your input values for T1 and T2 may be the culprits. Are you using the
> same values you used for Canopy?
>
> On 5/13/10 10:06 AM, Sai R wrote:
>> I am also getting the same error when running synthetic control dataset
>> example for kmeans
>> (https://cwiki.apache.org/MAHOUT/syntheticcontroldata.html) . Tried with
>> mahout 0.3 as well as 0.4 and get the same result. The dirichlet,
>> canopy,
>> meanshift clustering examples work fine. Any suggestions on what the
>> problem
>> could be for kmeans clustering?
>>
>> $HADOOP_HOME/bin/hadoop jar
>> mahout-0.4/trunk/examples/target/mahout-examples-0.4-SNAPSHOT.job
>> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
>> 10/05/13 20:22:06 INFO common.HadoopUtil: Deleting output
>> 10/05/13 20:22:06 INFO kmeans.Job: Preparing Input
>> 10/05/13 20:22:06 WARN mapred.JobClient: Use GenericOptionsParser for
>> parsing the arguments. Applications should implement Tool for the same.
>> 10/05/13 20:22:07 INFO mapred.FileInputFormat: Total input paths to
>> process
>> : 1
>> 10/05/13 20:22:07 INFO mapred.JobClient: Running job:
>> job_201005132016_0001
>> 10/05/13 20:22:08 INFO mapred.JobClient: map 0% reduce 0%
>> 10/05/13 20:22:18 INFO mapred.JobClient: map 100% reduce 0%
>> 10/05/13 20:22:20 INFO mapred.JobClient: Job complete:
>> job_201005132016_0001
>> 10/05/13 20:22:20 INFO mapred.JobClient: Counters: 8
>> 10/05/13 20:22:20 INFO mapred.JobClient: Job Counters
>> 10/05/13 20:22:20 INFO mapred.JobClient: Launched map tasks=2
>> 10/05/13 20:22:20 INFO mapred.JobClient: Data-local map tasks=2
>> 10/05/13 20:22:20 INFO mapred.JobClient: FileSystemCounters
>> 10/05/13 20:22:20 INFO mapred.JobClient: HDFS_BYTES_READ=291644
>> 10/05/13 20:22:20 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=335560
>> 10/05/13 20:22:20 INFO mapred.JobClient: Map-Reduce Framework
>> 10/05/13 20:22:20 INFO mapred.JobClient: Map input records=600
>> 10/05/13 20:22:20 INFO mapred.JobClient: Spilled Records=0
>> 10/05/13 20:22:20 INFO mapred.JobClient: Map input bytes=288374
>> 10/05/13 20:22:20 INFO mapred.JobClient: Map output records=600
>> 10/05/13 20:22:20 INFO kmeans.Job: Running Canopy to get initial
>> clusters
>> 10/05/13 20:22:20 INFO canopy.CanopyDriver: Input: output/data Out:
>> output
>> Measure: org.apache.mahout.common.distance.EuclideanDistanceMeasure
>> t1: 80.0
>> t2: 55.0
>> 10/05/13 20:22:20 WARN mapred.JobClient: Use GenericOptionsParser for
>> parsing the arguments. Applications should implement Tool for the same.
>> 10/05/13 20:22:21 INFO mapred.FileInputFormat: Total input paths to
>> process
>> : 2
>> 10/05/13 20:22:21 INFO mapred.JobClient: Running job:
>> job_201005132016_0002
>> 10/05/13 20:22:22 INFO mapred.JobClient: map 0% reduce 0%
>> 10/05/13 20:22:30 INFO mapred.JobClient: map 100% reduce 0%
>> 10/05/13 20:22:42 INFO mapred.JobClient: map 100% reduce 100%
>> 10/05/13 20:22:44 INFO mapred.JobClient: Job complete:
>> job_201005132016_0002
>> 10/05/13 20:22:44 INFO mapred.JobClient: Counters: 18
>> 10/05/13 20:22:44 INFO mapred.JobClient: Job Counters
>> 10/05/13 20:22:44 INFO mapred.JobClient: Launched reduce tasks=1
>> 10/05/13 20:22:44 INFO mapred.JobClient: Launched map tasks=2
>> 10/05/13 20:22:44 INFO mapred.JobClient: Data-local map tasks=2
>> 10/05/13 20:22:44 INFO mapred.JobClient: FileSystemCounters
>> 10/05/13 20:22:44 INFO mapred.JobClient: FILE_BYTES_READ=15574
>> 10/05/13 20:22:44 INFO mapred.JobClient: HDFS_BYTES_READ=335560
>> 10/05/13 20:22:44 INFO mapred.JobClient: FILE_BYTES_WRITTEN=31218
>> 10/05/13 20:22:44 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=4028
>> 10/05/13 20:22:44 INFO mapred.JobClient: Map-Reduce Framework
>> 10/05/13 20:22:44 INFO mapred.JobClient: Reduce input groups=1
>> 10/05/13 20:22:44 INFO mapred.JobClient: Combine output records=0
>> 10/05/13 20:22:44 INFO mapred.JobClient: Map input records=600
>> 10/05/13 20:22:44 INFO mapred.JobClient: Reduce shuffle bytes=8346
>> 10/05/13 20:22:44 INFO mapred.JobClient: Reduce output records=7
>> 10/05/13 20:22:44 INFO mapred.JobClient: Spilled Records=56
>> 10/05/13 20:22:44 INFO mapred.JobClient: Map output bytes=15456
>> 10/05/13 20:22:44 INFO mapred.JobClient: Map input bytes=335380
>> 10/05/13 20:22:44 INFO mapred.JobClient: Combine input records=0
>> 10/05/13 20:22:44 INFO mapred.JobClient: Map output records=28
>> 10/05/13 20:22:44 INFO mapred.JobClient: Reduce input records=28
>> 10/05/13 20:22:44 INFO kmeans.Job: Running KMeans
>> 10/05/13 20:22:44 INFO kmeans.KMeansDriver: Input: output/data
>> Clusters In:
>> output/clusters-0 Out: output Distance:
>> org.apache.mahout.common.distance.EuclideanDistanceMeasure
>> 10/05/13 20:22:44 INFO kmeans.KMeansDriver: convergence: 0.5 max
>> Iterations:
>> 10 num Reduce Tasks: 1 Input Vectors:
>> org.apache.mahout.math.VectorWritable
>> 10/05/13 20:22:44 INFO kmeans.KMeansDriver: Iteration 1
>> 10/05/13 20:22:44 WARN mapred.JobClient: Use GenericOptionsParser for
>> parsing the arguments. Applications should implement Tool for the same.
>> 10/05/13 20:22:45 INFO mapred.FileInputFormat: Total input paths to
>> process
>> : 2
>> 10/05/13 20:22:46 INFO mapred.JobClient: Running job:
>> job_201005132016_0003
>> 10/05/13 20:22:47 INFO mapred.JobClient: map 0% reduce 0%
>> 10/05/13 20:22:56 INFO mapred.JobClient: Task Id :
>> attempt_201005132016_0003_m_000000_0, Status : FAILED
>> java.lang.RuntimeException: Error in configuring object
>> at
>> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
>>
>> at
>> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
>> at
>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
>>
>> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
>> at org.apache.hadoop.mapred.Child.main(Child.java:170)
>> Caused by: java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> at java.lang.reflect.Method.invoke(Method.java:616)
>> at
>> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
>>
>> ... 5 more
>> Caused by: java.lang.RuntimeException: Error in configuring object
>> at
>> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
>>
>> at
>> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
>> at
>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
>>
>> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
>> ... 10 more
>> Caused by: java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> at java.lang.reflect.Method.invoke(Method.java:616)
>> at
>> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
>>
>> ... 13 more
>> Caused by: java.lang.IllegalStateException: Cluster is empty!
>> at
>> org.apache.mahout.clustering.kmeans.KMeansMapper.configure(KMeansMapper.java:74)
>>
>> ... 18 more
>>
>>
>>
>>
>>
>
>
Re: k-means InvocationTargetException
Posted by Jeff Eastman <jd...@windwardsolutions.com>.
I agree it is inexplicable, since I've run k-means on synthetic control
today on a fresh build/install and it worked fine. Could you provide
some more information on your setup, and exactly what you are running
(command line, platform OS, Hadoop version, cluster deployment?), etc?
It's really hard for me to help if I cannot duplicate the problem.
That said, here are some ideas:
* I did fix a problem with Canopy Driver (on 5/12) that caused
k-means to fail on synthetic control. It was deleting the output
directory but that caused a different (file not found) error. This
caused the synthetic control k-means job to fail. I don't think
that affected 0.3 though, as I believe I introduced it during
refactoring the prior week. Building again would be a good idea,
just to be safe.
* k-Means needs to be primed with k initial clusters. These can be
selected at random from the input data points or you can run
Canopy to produce them, as synthetic control does. If the program
is failing to load any initial clusters then you can run the
ClusterDumper on the clusters-0 directory contents to inspect that
input.
* If you are running this on a Hadoop cluster, then you will need to
copy clusters-0 to local storage, since ClusterDumper is not
Hadoopable.
* If that directory does not contain any clusters then we need to
look at why that might occur. The synthetic control job first runs
Canopy to obtain the initial clusters. Does Canopy work as
advertised on your system? Do you get the same clusters from it
that you see in clusters-0?
I need some more information in order to suggest more ideas.
Jeff
On 5/13/10 5:44 PM, Delroy Cameron wrote:
> guys, this is an inexplicable bug, especially since other algorithms i.e.
> Dirichlet, LDA run just fine..
> i'm sure it is not a problem with creating the vectors...if it is that the
> initial clusters are empty, then how do i solve that?...i might be will to
> checkout, compile and install mahout from trunk again and try this one more
> time...
>
> -----
> --cheers
> Delroy
>
Re: k-means InvocationTargetException
Posted by Delroy Cameron <de...@gmail.com>.
guys, this is an inexplicable bug, especially since other algorithms i.e.
Dirichlet, LDA run just fine..
i'm sure it is not a problem with creating the vectors...if it is that the
initial clusters are empty, then how do i solve that?...i might be will to
checkout, compile and install mahout from trunk again and try this one more
time...
-----
--cheers
Delroy
--
View this message in context: http://lucene.472066.n3.nabble.com/k-means-InvocationTargetException-tp780778p816168.html
Sent from the Mahout User List mailing list archive at Nabble.com.
Re: k-means InvocationTargetException
Posted by Jeff Eastman <jd...@windwardsolutions.com>.
It's not finding your initial clusters. What's in output/clusters-0?
Your input values for T1 and T2 may be the culprits. Are you using the
same values you used for Canopy?
On 5/13/10 10:06 AM, Sai R wrote:
> I am also getting the same error when running synthetic control dataset
> example for kmeans
> (https://cwiki.apache.org/MAHOUT/syntheticcontroldata.html) . Tried with
> mahout 0.3 as well as 0.4 and get the same result. The dirichlet, canopy,
> meanshift clustering examples work fine. Any suggestions on what the problem
> could be for kmeans clustering?
>
> $HADOOP_HOME/bin/hadoop jar
> mahout-0.4/trunk/examples/target/mahout-examples-0.4-SNAPSHOT.job
> org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
> 10/05/13 20:22:06 INFO common.HadoopUtil: Deleting output
> 10/05/13 20:22:06 INFO kmeans.Job: Preparing Input
> 10/05/13 20:22:06 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> 10/05/13 20:22:07 INFO mapred.FileInputFormat: Total input paths to process
> : 1
> 10/05/13 20:22:07 INFO mapred.JobClient: Running job: job_201005132016_0001
> 10/05/13 20:22:08 INFO mapred.JobClient: map 0% reduce 0%
> 10/05/13 20:22:18 INFO mapred.JobClient: map 100% reduce 0%
> 10/05/13 20:22:20 INFO mapred.JobClient: Job complete: job_201005132016_0001
> 10/05/13 20:22:20 INFO mapred.JobClient: Counters: 8
> 10/05/13 20:22:20 INFO mapred.JobClient: Job Counters
> 10/05/13 20:22:20 INFO mapred.JobClient: Launched map tasks=2
> 10/05/13 20:22:20 INFO mapred.JobClient: Data-local map tasks=2
> 10/05/13 20:22:20 INFO mapred.JobClient: FileSystemCounters
> 10/05/13 20:22:20 INFO mapred.JobClient: HDFS_BYTES_READ=291644
> 10/05/13 20:22:20 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=335560
> 10/05/13 20:22:20 INFO mapred.JobClient: Map-Reduce Framework
> 10/05/13 20:22:20 INFO mapred.JobClient: Map input records=600
> 10/05/13 20:22:20 INFO mapred.JobClient: Spilled Records=0
> 10/05/13 20:22:20 INFO mapred.JobClient: Map input bytes=288374
> 10/05/13 20:22:20 INFO mapred.JobClient: Map output records=600
> 10/05/13 20:22:20 INFO kmeans.Job: Running Canopy to get initial clusters
> 10/05/13 20:22:20 INFO canopy.CanopyDriver: Input: output/data Out: output
> Measure: org.apache.mahout.common.distance.EuclideanDistanceMeasure t1: 80.0
> t2: 55.0
> 10/05/13 20:22:20 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> 10/05/13 20:22:21 INFO mapred.FileInputFormat: Total input paths to process
> : 2
> 10/05/13 20:22:21 INFO mapred.JobClient: Running job: job_201005132016_0002
> 10/05/13 20:22:22 INFO mapred.JobClient: map 0% reduce 0%
> 10/05/13 20:22:30 INFO mapred.JobClient: map 100% reduce 0%
> 10/05/13 20:22:42 INFO mapred.JobClient: map 100% reduce 100%
> 10/05/13 20:22:44 INFO mapred.JobClient: Job complete: job_201005132016_0002
> 10/05/13 20:22:44 INFO mapred.JobClient: Counters: 18
> 10/05/13 20:22:44 INFO mapred.JobClient: Job Counters
> 10/05/13 20:22:44 INFO mapred.JobClient: Launched reduce tasks=1
> 10/05/13 20:22:44 INFO mapred.JobClient: Launched map tasks=2
> 10/05/13 20:22:44 INFO mapred.JobClient: Data-local map tasks=2
> 10/05/13 20:22:44 INFO mapred.JobClient: FileSystemCounters
> 10/05/13 20:22:44 INFO mapred.JobClient: FILE_BYTES_READ=15574
> 10/05/13 20:22:44 INFO mapred.JobClient: HDFS_BYTES_READ=335560
> 10/05/13 20:22:44 INFO mapred.JobClient: FILE_BYTES_WRITTEN=31218
> 10/05/13 20:22:44 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=4028
> 10/05/13 20:22:44 INFO mapred.JobClient: Map-Reduce Framework
> 10/05/13 20:22:44 INFO mapred.JobClient: Reduce input groups=1
> 10/05/13 20:22:44 INFO mapred.JobClient: Combine output records=0
> 10/05/13 20:22:44 INFO mapred.JobClient: Map input records=600
> 10/05/13 20:22:44 INFO mapred.JobClient: Reduce shuffle bytes=8346
> 10/05/13 20:22:44 INFO mapred.JobClient: Reduce output records=7
> 10/05/13 20:22:44 INFO mapred.JobClient: Spilled Records=56
> 10/05/13 20:22:44 INFO mapred.JobClient: Map output bytes=15456
> 10/05/13 20:22:44 INFO mapred.JobClient: Map input bytes=335380
> 10/05/13 20:22:44 INFO mapred.JobClient: Combine input records=0
> 10/05/13 20:22:44 INFO mapred.JobClient: Map output records=28
> 10/05/13 20:22:44 INFO mapred.JobClient: Reduce input records=28
> 10/05/13 20:22:44 INFO kmeans.Job: Running KMeans
> 10/05/13 20:22:44 INFO kmeans.KMeansDriver: Input: output/data Clusters In:
> output/clusters-0 Out: output Distance:
> org.apache.mahout.common.distance.EuclideanDistanceMeasure
> 10/05/13 20:22:44 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations:
> 10 num Reduce Tasks: 1 Input Vectors: org.apache.mahout.math.VectorWritable
> 10/05/13 20:22:44 INFO kmeans.KMeansDriver: Iteration 1
> 10/05/13 20:22:44 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> 10/05/13 20:22:45 INFO mapred.FileInputFormat: Total input paths to process
> : 2
> 10/05/13 20:22:46 INFO mapred.JobClient: Running job: job_201005132016_0003
> 10/05/13 20:22:47 INFO mapred.JobClient: map 0% reduce 0%
> 10/05/13 20:22:56 INFO mapred.JobClient: Task Id :
> attempt_201005132016_0003_m_000000_0, Status : FAILED
> java.lang.RuntimeException: Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ... 5 more
> Caused by: java.lang.RuntimeException: Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> ... 10 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ... 13 more
> Caused by: java.lang.IllegalStateException: Cluster is empty!
> at
> org.apache.mahout.clustering.kmeans.KMeansMapper.configure(KMeansMapper.java:74)
> ... 18 more
>
>
>
>
>
>
Re: k-means InvocationTargetException
Posted by Sai R <sa...@gmail.com>.
I am also getting the same error when running synthetic control dataset
example for kmeans
(https://cwiki.apache.org/MAHOUT/syntheticcontroldata.html) . Tried with
mahout 0.3 as well as 0.4 and get the same result. The dirichlet, canopy,
meanshift clustering examples work fine. Any suggestions on what the problem
could be for kmeans clustering?
$HADOOP_HOME/bin/hadoop jar
mahout-0.4/trunk/examples/target/mahout-examples-0.4-SNAPSHOT.job
org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
10/05/13 20:22:06 INFO common.HadoopUtil: Deleting output
10/05/13 20:22:06 INFO kmeans.Job: Preparing Input
10/05/13 20:22:06 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
10/05/13 20:22:07 INFO mapred.FileInputFormat: Total input paths to process
: 1
10/05/13 20:22:07 INFO mapred.JobClient: Running job: job_201005132016_0001
10/05/13 20:22:08 INFO mapred.JobClient: map 0% reduce 0%
10/05/13 20:22:18 INFO mapred.JobClient: map 100% reduce 0%
10/05/13 20:22:20 INFO mapred.JobClient: Job complete: job_201005132016_0001
10/05/13 20:22:20 INFO mapred.JobClient: Counters: 8
10/05/13 20:22:20 INFO mapred.JobClient: Job Counters
10/05/13 20:22:20 INFO mapred.JobClient: Launched map tasks=2
10/05/13 20:22:20 INFO mapred.JobClient: Data-local map tasks=2
10/05/13 20:22:20 INFO mapred.JobClient: FileSystemCounters
10/05/13 20:22:20 INFO mapred.JobClient: HDFS_BYTES_READ=291644
10/05/13 20:22:20 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=335560
10/05/13 20:22:20 INFO mapred.JobClient: Map-Reduce Framework
10/05/13 20:22:20 INFO mapred.JobClient: Map input records=600
10/05/13 20:22:20 INFO mapred.JobClient: Spilled Records=0
10/05/13 20:22:20 INFO mapred.JobClient: Map input bytes=288374
10/05/13 20:22:20 INFO mapred.JobClient: Map output records=600
10/05/13 20:22:20 INFO kmeans.Job: Running Canopy to get initial clusters
10/05/13 20:22:20 INFO canopy.CanopyDriver: Input: output/data Out: output
Measure: org.apache.mahout.common.distance.EuclideanDistanceMeasure t1: 80.0
t2: 55.0
10/05/13 20:22:20 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
10/05/13 20:22:21 INFO mapred.FileInputFormat: Total input paths to process
: 2
10/05/13 20:22:21 INFO mapred.JobClient: Running job: job_201005132016_0002
10/05/13 20:22:22 INFO mapred.JobClient: map 0% reduce 0%
10/05/13 20:22:30 INFO mapred.JobClient: map 100% reduce 0%
10/05/13 20:22:42 INFO mapred.JobClient: map 100% reduce 100%
10/05/13 20:22:44 INFO mapred.JobClient: Job complete: job_201005132016_0002
10/05/13 20:22:44 INFO mapred.JobClient: Counters: 18
10/05/13 20:22:44 INFO mapred.JobClient: Job Counters
10/05/13 20:22:44 INFO mapred.JobClient: Launched reduce tasks=1
10/05/13 20:22:44 INFO mapred.JobClient: Launched map tasks=2
10/05/13 20:22:44 INFO mapred.JobClient: Data-local map tasks=2
10/05/13 20:22:44 INFO mapred.JobClient: FileSystemCounters
10/05/13 20:22:44 INFO mapred.JobClient: FILE_BYTES_READ=15574
10/05/13 20:22:44 INFO mapred.JobClient: HDFS_BYTES_READ=335560
10/05/13 20:22:44 INFO mapred.JobClient: FILE_BYTES_WRITTEN=31218
10/05/13 20:22:44 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=4028
10/05/13 20:22:44 INFO mapred.JobClient: Map-Reduce Framework
10/05/13 20:22:44 INFO mapred.JobClient: Reduce input groups=1
10/05/13 20:22:44 INFO mapred.JobClient: Combine output records=0
10/05/13 20:22:44 INFO mapred.JobClient: Map input records=600
10/05/13 20:22:44 INFO mapred.JobClient: Reduce shuffle bytes=8346
10/05/13 20:22:44 INFO mapred.JobClient: Reduce output records=7
10/05/13 20:22:44 INFO mapred.JobClient: Spilled Records=56
10/05/13 20:22:44 INFO mapred.JobClient: Map output bytes=15456
10/05/13 20:22:44 INFO mapred.JobClient: Map input bytes=335380
10/05/13 20:22:44 INFO mapred.JobClient: Combine input records=0
10/05/13 20:22:44 INFO mapred.JobClient: Map output records=28
10/05/13 20:22:44 INFO mapred.JobClient: Reduce input records=28
10/05/13 20:22:44 INFO kmeans.Job: Running KMeans
10/05/13 20:22:44 INFO kmeans.KMeansDriver: Input: output/data Clusters In:
output/clusters-0 Out: output Distance:
org.apache.mahout.common.distance.EuclideanDistanceMeasure
10/05/13 20:22:44 INFO kmeans.KMeansDriver: convergence: 0.5 max Iterations:
10 num Reduce Tasks: 1 Input Vectors: org.apache.mahout.math.VectorWritable
10/05/13 20:22:44 INFO kmeans.KMeansDriver: Iteration 1
10/05/13 20:22:44 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
10/05/13 20:22:45 INFO mapred.FileInputFormat: Total input paths to process
: 2
10/05/13 20:22:46 INFO mapred.JobClient: Running job: job_201005132016_0003
10/05/13 20:22:47 INFO mapred.JobClient: map 0% reduce 0%
10/05/13 20:22:56 INFO mapred.JobClient: Task Id :
attempt_201005132016_0003_m_000000_0, Status : FAILED
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 5 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 13 more
Caused by: java.lang.IllegalStateException: Cluster is empty!
at
org.apache.mahout.clustering.kmeans.KMeansMapper.configure(KMeansMapper.java:74)
... 18 more
--
View this message in context: http://lucene.472066.n3.nabble.com/k-means-InvocationTargetException-tp780778p815385.html
Sent from the Mahout User List mailing list archive at Nabble.com.