Regarding chaining multiple map-reduce jobs in Hadoop streaming

[Varadharajan Mukundan <sr...@gmail.com>]

Hi,

I need to chain a couple of mapreduce jobs in Hadoop streaming. i am
planning to use python to write the mapper and reducer scripts. is
there any other way to chain these jobs other than using a shell
script and a temporary directory in HDFS?

-- 
Thanks,
M. Varadharajan

------------------------------------------------

"Experience is what you get when you didn't get what you wanted"
               -By Prof. Randy Pausch in "The Last Lecture"

My Journal :- www.thinkasgeek.wordpress.com

Re: Regarding chaining multiple map-reduce jobs in Hadoop streaming

[Harsh J <qw...@gmail.com>]

You can use Oozie from Yahoo! for building an elegant workflow out of
your streaming jobs; but you would still require output spots on your
HDFS as output records aren't really pipelined into the next job.

-- 
Harsh J
www.harshj.com