Best way to handle library dependencies

[David Engel <da...@istwok.net>]

Hi,

Please pardon me for asking what is probably a Java newbie question
here (I am one), but since I'm trying to use Mahout, I thought I'd
start here.

I'm picking up some proof of concept work that was started by a
coworker.  He started this work with Nutch 1.1, Hadoop 0.20.1 and
Mahout 0.4.  When I tried to switch to Mahout 0.5, things began
breaking due to various exceptions.  I finally traced the problems
down to version mismatches in the various library dependencies.  For
example, Mahout 0.5 was built with Lucene 3.1 while Nutch/Hadoop was
built with Lucene 3.0.  I can fix some problems by tweaking the
classpath ordering and/or copying and removing jar files, but doing so
tends to break other things.

My question then is what is the best way to handle dependencies in
Java in a situation like this?  My best guess is to rebuild everything
from source and try to make sure Nutch, Hadoop and Mahout all use the
same versions of libraries.  Does that sound reasonable?

Please note that my work will eventually be merged with another
project that is in early production using Nutch 1.1 and Hadoop 0.20.2.
Is it worth considering moving everything to the more current Nutch
1.4 and Hadoop 0.20.204.0, especially if it makes the dependency
issues easier to handle?

Thanks,
David
-- 
David Engel
david@istwok.net

Re: Best way to handle library dependencies

[Ted Dunning <te...@gmail.com>]

Yes.  If you have the option, rebuild.

With Mahout, it is relatively easy to tweak the versions because it uses
maven.  Just look in the top-level pom.xml file.

On Mon, Dec 19, 2011 at 1:39 PM, David Engel <da...@istwok.net> wrote:

> My question then is what is the best way to handle dependencies in
> Java in a situation like this?  My best guess is to rebuild everything
> from source and try to make sure Nutch, Hadoop and Mahout all use the
> same versions of libraries.  Does that sound reasonable?
>