nutch with Hadoop V2

[Mike Frampton <mi...@hotmail.com>]

Hi 
 
When last I checked Nutch was still using the Hadoop V1 Map Reduce API. Is there any plan to have Nutch work wit Yarn / the new API and if so
when will this be or which version of Nutch is it ? 
 
cheers
 
Mike Frampton
 		 	   		  

Re: nutch with Hadoop V2

[Ali Nazemian <al...@gmail.com>]

Dear Talat,
Hi,
What about nutch 1.x?! Is there any plan for migrating to Yarn in nutch1
road map?
Regards.


On Fri, Sep 5, 2014 at 8:53 AM, Talat Uyarer <ta...@uyarer.com> wrote:

> Hi Mike,
>
> As a Nutch 2.x user,  I will work on this issue. My plan I will finish
> until next release. Hence Gora should support Hadoop 2. They work on
> Gora-348.
>
> Talat
> Hi
>
> When last I checked Nutch was still using the Hadoop V1 Map Reduce API. Is
> there any plan to have Nutch work wit Yarn / the new API and if so
> when will this be or which version of Nutch is it ?
>
> cheers
>
> Mike Frampton
>



-- 
A.Nazemian

Re: nutch with Hadoop V2

[Talat Uyarer <ta...@uyarer.com>]

Hi Mike,

As a Nutch 2.x user,  I will work on this issue. My plan I will finish
until next release. Hence Gora should support Hadoop 2. They work on
Gora-348.

Talat
Hi

When last I checked Nutch was still using the Hadoop V1 Map Reduce API. Is
there any plan to have Nutch work wit Yarn / the new API and if so
when will this be or which version of Nutch is it ?

cheers

Mike Frampton

Re: nutch with Hadoop V2

["Jake K. Dodd" <ja...@ontopic.io>]

Nutch will function on a Hadoop 2 cluster. I have Nutch 1.9 running on a HDP 2.1 (Hadoop 2.4) provisioned cluster right now, and have previously used Nutch 1.7-8 in the Hadoop 2 environment. So, if your question is whether Nutch is compatible with Hadoop 2 and YARN, the answer is yes.

However, I think you're asking not just whether it will run (is it compatible), but whether Nutch's internals have been migrated from MapReduce (now sometimes called MRv1) to YARN. The answer is no, as far as I can tell. Nutch will run in Hadoop 2 environments because Hadoop 2 is compatible with MRv1, not because Nutch has migrated to using the YARN API.

Somebody should be able to give you a more clear (and maybe more accurate) answer than I can, but I hope this helps.

Cheers

Jake K Dodd
(703) 867 5170
jake.kensington.dodd@gmail.com


> On Sep 4, 2014, at 20:44, Mike Frampton <mi...@hotmail.com> wrote:
> 
> Hi 
> 
> When last I checked Nutch was still using the Hadoop V1 Map Reduce API. Is there any plan to have Nutch work wit Yarn / the new API and if so
> when will this be or which version of Nutch is it ? 
> 
> cheers
> 
> Mike Frampton
>