You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by Edoardo Causarano <ed...@gmail.com> on 2014/09/04 15:17:58 UTC

Running on CDH5 (Hadoop 2)

Hello All,

I'm trying to build 2.2 against hadoop 2(.4.0) so that I can run it on a CDH5 cluster with HBase. A cursory glance online led me to believe it's a trivial task consisting of a simple recompilation but I have encountered a couple boulders that make me suspect it's a bit more involved than that... ;)

This is what I have now:

1. Gora 0.5-SNAPSHOT (current master) with the latest patch in GORA-346 applied (apparently it's still not included.)
2. Nutch 2.2.1 src tarball tweaked to use Gora 0.5 and Hadoop 2.4.0 (hadoop-common and hadoop-mapreduce-client-core)

... and a Christmas tree of compiler errors.

Has anyone already done some work on bumping to Gora to 0.5 and supporting Hadoop 2? If so, I'd love to share notes... :)


Eddy