You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Aleksandar Stupar <st...@yahoo.com> on 2010/03/16 16:11:27 UTC

CombineFileInputFormat in 0.20.2 version

Hi all,

I want to use CombineFileInputFormat in 0.20.2 version but it can't be used with Job class.

Description:
org.apache.hadoop.mapred.lib.CombineFileInputFormat can not be used with org.apache.hadoop.mapreduce.Job
because Job.setInputFormat requires subclass of  org.apache.hadoop.mapreduce.InputFormat and CombineFileInputFormat
extends org.apache.hadoop.mapred.FileInputFormat. 

Also CombineFileInputFormat uses deprecated classes. 


Are there any workarounds?

Thanks,
Aleksandar Stupar.

Re: CombineFileInputFormat in 0.20.2 version

Posted by Aaron Kimball <aa...@cloudera.com>.

The most obvious workaround is to use the old API (continue to use Mapper,
Reducer, etc. from org.apache.hadoop.mapred, not .mapreduce).

If you really want to use the new API, though, I unfortunately don't see a
super-easy path. You could try to apply the patch from MAPREDUCE-364 to your
version of Hadoop and recompile, but that might be tricky since the
filenames will most likely not line up (due to the project split).

- Aaron

On Tue, Mar 16, 2010 at 8:11 AM, Aleksandar Stupar <
stupar.aleksandar@yahoo.com> wrote:

> Hi all,
>
> I want to use CombineFileInputFormat in 0.20.2 version but it can't be used
> with Job class.
>
> Description:
> org.apache.hadoop.mapred.lib.CombineFileInputFormat can not be used with
> org.apache.hadoop.mapreduce.Job
> because Job.setInputFormat requires subclass of
>  org.apache.hadoop.mapreduce.InputFormat and CombineFileInputFormat
> extends org.apache.hadoop.mapred.FileInputFormat.
>
> Also CombineFileInputFormat uses deprecated classes.
>
>
> Are there any workarounds?
>
> Thanks,
> Aleksandar Stupar.
>
>
>
>