You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Gary Snider <ga...@gmail.com> on 2011/12/08 18:37:02 UTC
basic classpath issue running KMeansDriver
I have a pojo that calls KMeansDriver. I'm on 0.6-SNAPSHOT Ubuntu Linux
Once the mapper gets run in Hadoop, hadoop complains with:
Caused by: java.lang.NoClassDefFoundError: com/google/common/collect/Lists
at
org.apache.mahout.clustering.kmeans.KMeansMapper.<init>(KMeansMapper.java:37)
... 12 more
I've tried different combinations of setting classpath but nothing helps.
com.google.common.collect.Lists is in guava-r09.jar
1) setting $HADOOP_CLASSPATH to point to ALL possible jars before starting
hadoop
2) setting $CLASSPATH before starting hadoop
3) passing -classpath to my pojo
4) setting $CLASSPATH before running my pojo
The thing is, if it runs 'locally' from the local filesystem it works.
Once I point it to hadoop, I get the error.
Is this a hadoop classpath/classloader issue?
Re: basic classpath issue running KMeansDriver
Posted by Gary Snider <ga...@gmail.com>.
Genius! Thanks Sean. I had the regular mahout core jar in the classpath.
Once I removed all of those and just used
$MAHOUT_HOME/core/target/mahout-core-0.6-SNAPSHOT-job.jar it worked!
If this is on the wiki I apologize. I'm going to go look for it and figure
out how I missed that step.
On Thu, Dec 8, 2011 at 12:45 PM, Sean Owen <sr...@gmail.com> wrote:
> Are you using the "job" file rather than the "jar" file? The former is the
> one with all transitive dependencies bundled. You do not need to set any
> classpaths.
>
> On Thu, Dec 8, 2011 at 5:37 PM, Gary Snider <ga...@gmail.com>
> wrote:
>
> > I have a pojo that calls KMeansDriver. I'm on 0.6-SNAPSHOT Ubuntu Linux
> >
> > Once the mapper gets run in Hadoop, hadoop complains with:
> > Caused by: java.lang.NoClassDefFoundError:
> com/google/common/collect/Lists
> > at
> >
> >
> org.apache.mahout.clustering.kmeans.KMeansMapper.<init>(KMeansMapper.java:37)
> > ... 12 more
> >
> > I've tried different combinations of setting classpath but nothing helps.
> > com.google.common.collect.Lists is in guava-r09.jar
> > 1) setting $HADOOP_CLASSPATH to point to ALL possible jars before
> starting
> > hadoop
> > 2) setting $CLASSPATH before starting hadoop
> > 3) passing -classpath to my pojo
> > 4) setting $CLASSPATH before running my pojo
> >
> > The thing is, if it runs 'locally' from the local filesystem it works.
> > Once I point it to hadoop, I get the error.
> > Is this a hadoop classpath/classloader issue?
> >
>
Re: basic classpath issue running KMeansDriver
Posted by Sean Owen <sr...@gmail.com>.
Are you using the "job" file rather than the "jar" file? The former is the
one with all transitive dependencies bundled. You do not need to set any
classpaths.
On Thu, Dec 8, 2011 at 5:37 PM, Gary Snider <ga...@gmail.com> wrote:
> I have a pojo that calls KMeansDriver. I'm on 0.6-SNAPSHOT Ubuntu Linux
>
> Once the mapper gets run in Hadoop, hadoop complains with:
> Caused by: java.lang.NoClassDefFoundError: com/google/common/collect/Lists
> at
>
> org.apache.mahout.clustering.kmeans.KMeansMapper.<init>(KMeansMapper.java:37)
> ... 12 more
>
> I've tried different combinations of setting classpath but nothing helps.
> com.google.common.collect.Lists is in guava-r09.jar
> 1) setting $HADOOP_CLASSPATH to point to ALL possible jars before starting
> hadoop
> 2) setting $CLASSPATH before starting hadoop
> 3) passing -classpath to my pojo
> 4) setting $CLASSPATH before running my pojo
>
> The thing is, if it runs 'locally' from the local filesystem it works.
> Once I point it to hadoop, I get the error.
> Is this a hadoop classpath/classloader issue?
>