You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-user@hadoop.apache.org by Hassen Riahi <Ha...@cern.ch> on 2012/04/18 22:13:38 UTC

Open the input file

Hi all,

I am using the streaming library of hadoop-0.20 to be able to write map and reduce functions in python.
Is it possible to make hadoop MapReduce not opening the input files? the map function does not need the stream of input files contents. Has someone tried this already?

Thanks!
Hassen

Re: Open the input file

Posted by Robert Evans <ev...@yahoo-inc.com>.

Hassen,

You can supply your own custom input format to hadoop.  If you look at the code for SleepJob it has a custom input format that does not require any input files at all.  I am not really sure how streaming will deal with a mapper that has no input, but I assume it should be OK.

--Bobby Evans

On 4/18/12 3:13 PM, "Hassen Riahi" <Ha...@cern.ch> wrote:

Hi all,

I am using the streaming library of hadoop-0.20 to be able to write map and reduce functions in python.
Is it possible to make hadoop MapReduce not opening the input files? the map function does not need the stream of input files contents. Has someone tried this already?

Thanks!
Hassen