mapreduce with non-text data

[Upendra Dadi <ud...@gmu.edu>]

Hi,
  Are there any use cases, examples of use of Hadoop MapReduce for non-text 
data? Only examples that I see on the web are for text data. Any pointers in 
that direction is greatly appreciated. Thanks.

Regards,
Upendra 

Pipes with the latest version of mapReduce

[Upendra Dadi <ud...@gmu.edu>]

Hi,
  I am trying to use MapReduce Pipes with version 0.20.1. of hadoop. If I 
used  org.apache.hadoop.mapreduce.lib.input.TextInputFormat as the input 
format with the             -InputFormat option, I am getting the following 
error:
    Exception in thread "main" java.lang.ClassCastException: class 
org.apache.hadoop.mapreduce.lib.input.TextInputFormat
     at java.lang.Class.asSubclass(Class.java:3018)
     at 
org.apache.hadoop.mapred.pipes.Submitter.getClass(Submitter.java:372)
     at org.apache.hadoop.mapred.pipes.Submitter.run(Submitter.java:421)
     at org.apache.hadoop.mapred.pipes.Submitter.main(Submitter.java:494)

I was able to run Pipes without the -InputFormat option, but my requirement 
is to extend TextInputFormat.  I guess Pipes is assuming the inputFormat  to 
be of type org.apache.hadoop.mapred.InputFormat, which is deprecated. So I 
guess 0.20.1 is not fully ported to work with Pipes. Am I right here? Which 
version should I use? Thanks.

Upendra



----- Original Message ----- 
From: "Erez Katz" <er...@yahoo.com>
To: <co...@hadoop.apache.org>
Sent: Friday, November 27, 2009 6:59 PM
Subject: Re: mapreduce with non-text data


> you could always base64 encode your binary data...
>
>
>
> --- On Thu, 11/26/09, Upendra Dadi <ud...@gmu.edu> wrote:
>
>> From: Upendra Dadi <ud...@gmu.edu>
>> Subject: mapreduce with non-text data
>> To: common-dev@hadoop.apache.org
>> Date: Thursday, November 26, 2009, 5:07 PM
>> Hi,
>>  Are there any use cases, examples of use of Hadoop
>> MapReduce for non-text data? Only examples that I see on the
>> web are for text data. Any pointers in that direction is
>> greatly appreciated. Thanks.
>>
>> Regards,
>> Upendra
>>
>
>
> 

Re: mapreduce with non-text data

[Erez Katz <er...@yahoo.com>]

you could always base64 encode your binary data...



--- On Thu, 11/26/09, Upendra Dadi <ud...@gmu.edu> wrote:

> From: Upendra Dadi <ud...@gmu.edu>
> Subject: mapreduce with non-text data
> To: common-dev@hadoop.apache.org
> Date: Thursday, November 26, 2009, 5:07 PM
> Hi,
>  Are there any use cases, examples of use of Hadoop
> MapReduce for non-text data? Only examples that I see on the
> web are for text data. Any pointers in that direction is
> greatly appreciated. Thanks.
> 
> Regards,
> Upendra 
>