You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by deneche abdelhakim <a_...@yahoo.fr> on 2009/08/17 13:36:41 UTC
class not found bug ?
I moved recently some of the "Decision Forest" examples from the core project to the examples project. While in core they worked perfectly in hadoop 0..19.1 (pseudo-distributed), but now they don't !!!
For example, running my org.apache.mahout.df.BuildForest gives the following exception:
....
09/08/17 12:02:36 INFO mapred.JobClient: Running job: job_200908171136_0020
09/08/17 12:02:37 INFO mapred.JobClient: map 0% reduce 0%
09/08/17 12:02:43 INFO mapred.JobClient: Task Id : attempt_200908171136_0020_m_000000_0, Status : FAILED
java.lang.NoClassDefFoundError: com/thoughtworks/xstream/XStream
at org.apache.mahout.utils.StringUtils.<clinit>(StringUtils.java:28)
at org.apache.mahout.df.mapred.Builder.getTreeBuilder(Builder.java:117)
at org.apache.mahout.df.mapred.MapredMapper.configure(MapredMapper.java:74)
at org.apache.mahout.df.mapred.partial.Step1Mapper.configure(Step1Mapper.java:75)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
...
So I tried instead to run one of mahout's : following the wiki, kmeans gives me the following error:
...
09/08/17 11:59:27 INFO kmeans.KMeansDriver: Iteration 4
...
09/08/17 11:59:43 INFO kmeans.KMeansDriver: Clustering
09/08/17 11:59:43 INFO kmeans.KMeansDriver: Running Clustering
09/08/17 11:59:43 INFO kmeans.KMeansDriver: Input: output/data Clusters In: output/clusters-4 Out: output/points Distance: org.apache.mahout.utils.EuclideanDistanceMeasure
09/08/17 11:59:43 INFO kmeans.KMeansDriver: convergence: 0.5 Input Vectors: org.apache.mahout.matrix.SparseVector
09/08/17 11:59:43 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
09/08/17 11:59:43 INFO mapred..FileInputFormat: Total input paths to process : 2
09/08/17 11:59:43 INFO mapred.JobClient: Running job: job_200908171136_0019
09/08/17 11:59:44 INFO mapred.JobClient: map 0% reduce 0%
09/08/17 11:59:54 INFO mapred.JobClient: Task Id : attempt_200908171136_0019_m_000000_0, Status : FAILED
java.lang.NoClassDefFoundError: com/google/gson/reflect/TypeToken
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:637)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:277)
at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
at java.security.AccessController.doPrivileged(Native Method)
...
The problem seems related to the fact that mahout-core.jar is being packed inside examples.jar. So I modified maven/build.xml to pack the core classes instead (because they are available):
Index: maven/build.xml
===================================================================
--- maven/build.xml (revision 804891)
+++ maven/build.xml (working copy)
@@ -45,9 +45,9 @@
includes="**/*.jar"/>
<zipfileset dir="${core-lib}" prefix="lib"
includes="**/*.jar" excludes="hadoop-*.jar"/>
- <zipfileset dir="../core/target/" prefix="lib" includes="apache-mahout-core-${version}.jar"/>
+ <zipfileset dir="../core/target/classes"/>
<zipfileset dir="${dest}/dependency" prefix="lib"
- includes="**/*.jar"/>
+ includes="**/*.jar" excludes="apache-mahout-core-${version}.jar"/>
<zipfileset dir="../core/target/dependency" prefix="lib"
includes="**/*.jar"/>
</jar>
This seems to solve the problem, but I didn't try it on all examples
__________________________________________________
Do You Yahoo!?
En finir avec le spam? Yahoo! Mail vous offre la meilleure protection possible contre les messages non sollicités
http://mail.yahoo.fr Yahoo! Mail
Re: class not found bug ?
Posted by deneche abdelhakim <a_...@yahoo.fr>.
> Try 0.20.0. Also, you may need to flatten the JOB jar
> into a single jar.
My example and Kmeans example Don't work on 0.20.0 without actually flattening the JOB jar. But as I wrote in my last post, there may be a simpler solution: modifying "maven/build.xml" seems to solve the problem, at least for my examples and kmeans:
Index: maven/build.xml
===================================================================
--- maven/build.xml (revision 804891)
+++ maven/build.xml (working copy)
@@ -45,9 +45,9 @@
includes="**/*.jar"/>
<zipfileset dir="${core-lib}" prefix="lib"
includes="**/*.jar" excludes="hadoop-*.jar"/>
- <zipfileset dir="../core/target/" prefix="lib" includes="apache-mahout-core-${version}.jar"/>
+ <zipfileset dir="../core/target/classes"/>
<zipfileset dir="${dest}/dependency" prefix="lib"
- includes="**/*.jar"/>
+ includes="**/*.jar" excludes="apache-mahout-core-${version}..jar"/>
<zipfileset dir="../core/target/dependency" prefix="lib"
includes="**/*.jar"/>
</jar>
--- En date de : Lun 17.8.09, Grant Ingersoll <gs...@apache.org> a écrit :
> De: Grant Ingersoll <gs...@apache.org>
> Objet: Re: class not found bug ?
> À: mahout-dev@lucene.apache.org
> Date: Lundi 17 Août 2009, 14h46
>
> On Aug 17, 2009, at 7:36 AM, deneche abdelhakim wrote:
>
> > I moved recently some of the "Decision Forest"
> examples from the core project to the examples project.
> While in core they worked perfectly in hadoop 0..19.1
> (pseudo-distributed), but now they don't !!!
>
> Try 0.20.0. Also, you may need to flatten the JOB jar
> into a single jar.
>
>
> >
> > For example, running my
> org.apache.mahout.df.BuildForest gives the following
> exception:
> >
> > ....
> > 09/08/17 12:02:36 INFO mapred.JobClient: Running job:
> job_200908171136_0020
> > 09/08/17 12:02:37 INFO mapred.JobClient: map 0%
> reduce 0%
> > 09/08/17 12:02:43 INFO mapred.JobClient: Task Id :
> attempt_200908171136_0020_m_000000_0, Status : FAILED
> > java.lang.NoClassDefFoundError:
> com/thoughtworks/xstream/XStream
> > at
> org.apache.mahout.utils.StringUtils.<clinit>(StringUtils.java:28)
> > at
> org.apache.mahout.df.mapred.Builder.getTreeBuilder(Builder.java:117)
> > at
> org.apache.mahout.df.mapred.MapredMapper.configure(MapredMapper.java:74)
> > at
> org.apache.mahout.df.mapred.partial.Step1Mapper.configure(Step1Mapper.java:75)
> > at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
> > at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
> > ...
> >
> >
> > So I tried instead to run one of mahout's : following
> the wiki, kmeans gives me the following error:
> >
> > ...
> > 09/08/17 11:59:27 INFO kmeans.KMeansDriver: Iteration
> 4
> > ...
> > 09/08/17 11:59:43 INFO kmeans.KMeansDriver:
> Clustering
> > 09/08/17 11:59:43 INFO kmeans.KMeansDriver: Running
> Clustering
> > 09/08/17 11:59:43 INFO kmeans.KMeansDriver: Input:
> output/data Clusters In: output/clusters-4 Out:
> output/points Distance:
> org.apache.mahout.utils.EuclideanDistanceMeasure
> > 09/08/17 11:59:43 INFO kmeans.KMeansDriver:
> convergence: 0.5 Input Vectors:
> org.apache.mahout.matrix.SparseVector
> > 09/08/17 11:59:43 WARN mapred.JobClient: Use
> GenericOptionsParser for parsing the arguments. Applications
> should implement Tool for the same.
> > 09/08/17 11:59:43 INFO mapred..FileInputFormat: Total
> input paths to process : 2
> > 09/08/17 11:59:43 INFO mapred.JobClient: Running job:
> job_200908171136_0019
> > 09/08/17 11:59:44 INFO mapred.JobClient: map 0%
> reduce 0%
> > 09/08/17 11:59:54 INFO mapred.JobClient: Task Id :
> attempt_200908171136_0019_m_000000_0, Status : FAILED
> > java.lang.NoClassDefFoundError:
> com/google/gson/reflect/TypeToken
> > at
> java.lang.ClassLoader.defineClass1(Native Method)
> > at
> java.lang.ClassLoader.defineClass(ClassLoader.java:637)
> > at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> > at
> java.net.URLClassLoader.defineClass(URLClassLoader.java:277)
> > at
> java.net.URLClassLoader.access$000(URLClassLoader.java:73)
> > at
> java.net.URLClassLoader$1.run(URLClassLoader.java:212)
> > at
> java.security.AccessController.doPrivileged(Native Method)
> > ...
> >
> > The problem seems related to the fact that
> mahout-core.jar is being packed inside examples.jar. So I
> modified maven/build.xml to pack the core classes instead
> (because they are available):
> >
> > Index: maven/build.xml
> >
> ===================================================================
> > --- maven/build.xml (revision
> 804891)
> > +++ maven/build.xml (working copy)
> > @@ -45,9 +45,9 @@
> >
> includes="**/*.jar"/>
> > <zipfileset
> dir="${core-lib}" prefix="lib"
> >
> includes="**/*.jar"
> excludes="hadoop-*.jar"/>
> > - <zipfileset
> dir="../core/target/" prefix="lib"
> includes="apache-mahout-core-${version}.jar"/>
> > + <zipfileset
> dir="../core/target/classes"/>
> > <zipfileset
> dir="${dest}/dependency" prefix="lib"
> > -
> includes="**/*.jar"/>
> > +
> includes="**/*.jar"
> excludes="apache-mahout-core-${version}.jar"/>
> > <zipfileset
> dir="../core/target/dependency" prefix="lib"
> >
> includes="**/*.jar"/>
> > </jar>
> >
> > This seems to solve the problem, but I didn't try it
> on all examples
> >
> > __________________________________________________
> > Do You Yahoo!?
> > En finir avec le spam? Yahoo! Mail vous offre la
> meilleure protection possible contre les messages non
> sollicités
> > http://mail.yahoo.fr Yahoo! Mail
> >
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
> Search the Lucene ecosystem
> (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene:
> http://www.lucidimagination.com/search
>
>
__________________________________________________
Do You Yahoo!?
En finir avec le spam? Yahoo! Mail vous offre la meilleure protection possible contre les messages non sollicités
http://mail.yahoo.fr Yahoo! Mail
Re: class not found bug ?
Posted by Grant Ingersoll <gs...@apache.org>.
On Aug 17, 2009, at 7:36 AM, deneche abdelhakim wrote:
> I moved recently some of the "Decision Forest" examples from the
> core project to the examples project. While in core they worked
> perfectly in hadoop 0..19.1 (pseudo-distributed), but now they
> don't !!!
Try 0.20.0. Also, you may need to flatten the JOB jar into a single
jar.
>
> For example, running my org.apache.mahout.df.BuildForest gives the
> following exception:
>
> ....
> 09/08/17 12:02:36 INFO mapred.JobClient: Running job:
> job_200908171136_0020
> 09/08/17 12:02:37 INFO mapred.JobClient: map 0% reduce 0%
> 09/08/17 12:02:43 INFO mapred.JobClient: Task Id :
> attempt_200908171136_0020_m_000000_0, Status : FAILED
> java.lang.NoClassDefFoundError: com/thoughtworks/xstream/XStream
> at org.apache.mahout.utils.StringUtils.<clinit>(StringUtils.java:28)
> at org.apache.mahout.df.mapred.Builder.getTreeBuilder(Builder.java:
> 117)
> at
> org.apache.mahout.df.mapred.MapredMapper.configure(MapredMapper.java:
> 74)
> at
> org
> .apache
> .mahout.df.mapred.partial.Step1Mapper.configure(Step1Mapper.java:75)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:
> 58)
> at
> org
> .apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:
> 83)
> ...
>
>
> So I tried instead to run one of mahout's : following the wiki,
> kmeans gives me the following error:
>
> ...
> 09/08/17 11:59:27 INFO kmeans.KMeansDriver: Iteration 4
> ...
> 09/08/17 11:59:43 INFO kmeans.KMeansDriver: Clustering
> 09/08/17 11:59:43 INFO kmeans.KMeansDriver: Running Clustering
> 09/08/17 11:59:43 INFO kmeans.KMeansDriver: Input: output/data
> Clusters In: output/clusters-4 Out: output/points Distance:
> org.apache.mahout.utils.EuclideanDistanceMeasure
> 09/08/17 11:59:43 INFO kmeans.KMeansDriver: convergence: 0.5 Input
> Vectors: org.apache.mahout.matrix.SparseVector
> 09/08/17 11:59:43 WARN mapred.JobClient: Use GenericOptionsParser
> for parsing the arguments. Applications should implement Tool for
> the same.
> 09/08/17 11:59:43 INFO mapred..FileInputFormat: Total input paths to
> process : 2
> 09/08/17 11:59:43 INFO mapred.JobClient: Running job:
> job_200908171136_0019
> 09/08/17 11:59:44 INFO mapred.JobClient: map 0% reduce 0%
> 09/08/17 11:59:54 INFO mapred.JobClient: Task Id :
> attempt_200908171136_0019_m_000000_0, Status : FAILED
> java.lang.NoClassDefFoundError: com/google/gson/reflect/TypeToken
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:637)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:
> 142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:277)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
> at java.security.AccessController.doPrivileged(Native Method)
> ...
>
> The problem seems related to the fact that mahout-core.jar is being
> packed inside examples.jar. So I modified maven/build.xml to pack
> the core classes instead (because they are available):
>
> Index: maven/build.xml
> ===================================================================
> --- maven/build.xml (revision 804891)
> +++ maven/build.xml (working copy)
> @@ -45,9 +45,9 @@
> includes="**/*.jar"/>
> <zipfileset dir="${core-lib}" prefix="lib"
> includes="**/*.jar" excludes="hadoop-*.jar"/>
> - <zipfileset dir="../core/target/" prefix="lib"
> includes="apache-mahout-core-${version}.jar"/>
> + <zipfileset dir="../core/target/classes"/>
> <zipfileset dir="${dest}/dependency" prefix="lib"
> - includes="**/*.jar"/>
> + includes="**/*.jar" excludes="apache-mahout-core-$
> {version}.jar"/>
> <zipfileset dir="../core/target/dependency" prefix="lib"
> includes="**/*.jar"/>
> </jar>
>
> This seems to solve the problem, but I didn't try it on all examples
>
> __________________________________________________
> Do You Yahoo!?
> En finir avec le spam? Yahoo! Mail vous offre la meilleure
> protection possible contre les messages non sollicités
> http://mail.yahoo.fr Yahoo! Mail
>
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
using Solr/Lucene:
http://www.lucidimagination.com/search