You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@crunch.apache.org by Peter Knap <pk...@yahoo.com> on 2012/12/19 04:56:39 UTC

Compatibility question

Hi,

I was wondering why Hadoop 0.20 is not supported by crunch. I like crunch and would like to use it but we have a hadoop cluster on the 0.20 release level and no immediate plans to upgrade it. Are you guys planning to support older version of hadoop? 


Thanks,
Piotr

Re: Compatibility question

Posted by Josh Wills <jw...@cloudera.com>.
Hey Piotr,

The feature that Crunch needs from later versions of Hadoop is the ability
to write multiple output files from a single mapper/reducer; it's baked
pretty deeply into the design and would be tough to pull out w/o a bunch of
work. If the version of Hadoop on the cluster supports multiple outputs and
the mapreduce.* APIs, Crunch should run-- if it doesn't, I'd like to know
more.

Josh


On Tue, Dec 18, 2012 at 7:56 PM, Peter Knap <pk...@yahoo.com> wrote:

> Hi,
>
> I was wondering why Hadoop 0.20 is not supported by crunch. I like crunch
> and would like to use it but we have a hadoop cluster on the 0.20 release
> level and no immediate plans to upgrade it. Are you guys planning to
> support older version of hadoop?
>
> Thanks,
> Piotr
>



-- 
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>