Classpath issues for basic mapreduce

[Dan Filimon <da...@gmail.com>]

Hi!

I'm having trouble with the following code (an updated version of the
basic clustering code in Sean Owen's book):
https://gist.github.com/4038746

It compiles just fine, but I'm using a project based on Mahout for the
dependencies and class path and it won't run.

It initially threw a NoClassDefFoundError for org/apache/hadoop/io/Writable.

It turns out that the Hadoop Mahout 0.8 depends on is 0.20.204.0 but I
only have 1.0.4 installed on my machine so I updated the dependencies
in IntelliJ to include the Hadoop JARs that I have.

Now, it's failing because it can't find another class:
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/commons/logging/LogFactory

I can see this dependency available when I imported the Maven project
but it doesn't seem to be on the classpath.
Any ideas?
Thanks!

Re: Classpath issues for basic mapreduce

[Dan Filimon <da...@gmail.com>]

On Thu, Nov 8, 2012 at 3:31 PM, Sean Owen <sr...@gmail.com> wrote:
> (It isn't my book alone -- I did not write the clustering part.)

Oops, you're right! Anyway, great job to you and everybody else who helped!

> 0.8-SNAPSHOT depends on 1.0.4 actually. Try the latest code in SVN.
> This is not the source of your problem though. You are not including
> Hadoop's dependencies (like commons logging) in your project. Why, is
> specific to how you tried to configure your project.

Adding Hadoop dependencies and specifying the correct version of slf4j
in my pom.xml fixed it! :)
Thanks!

Re: Classpath issues for basic mapreduce

[Sean Owen <sr...@gmail.com>]

(It isn't my book alone -- I did not write the clustering part.)

0.8-SNAPSHOT depends on 1.0.4 actually. Try the latest code in SVN.
This is not the source of your problem though. You are not including
Hadoop's dependencies (like commons logging) in your project. Why, is
specific to how you tried to configure your project.


On Thu, Nov 8, 2012 at 1:23 PM, Dan Filimon <da...@gmail.com>wrote:

> Hi!
>
> I'm having trouble with the following code (an updated version of the
> basic clustering code in Sean Owen's book):
> https://gist.github.com/4038746
>
> It compiles just fine, but I'm using a project based on Mahout for the
> dependencies and class path and it won't run.
>
> It initially threw a NoClassDefFoundError for
> org/apache/hadoop/io/Writable.
>
> It turns out that the Hadoop Mahout 0.8 depends on is 0.20.204.0 but I
> only have 1.0.4 installed on my machine so I updated the dependencies
> in IntelliJ to include the Hadoop JARs that I have.
>
> Now, it's failing because it can't find another class:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/commons/logging/LogFactory
>
> I can see this dependency available when I imported the Maven project
> but it doesn't seem to be on the classpath.
> Any ideas?
> Thanks!
>