Re: Does the map task push map output to reduce task or reduce task pull it from map task

[Prabhu Hari Dhanapal <dr...@gmail.com>]

Well ,I m not sure But I think it might be the  pull.. because physically
the mappers and the reducers are the same nodes ,So if the Mappers  had to
push , it might be the case that all nodes are mapping and there are no
reducers  to  accept it. May be for this  reason ,unless all of the Mapper
tasks are finished, the reducers might not want to start  reducing  anything
@all..

There is also this sort shuffle layer between  maping and  reducing , it
 clearly demarcates the phases.. whihc seem to suggest that its the pull
rather than the push ..

You might think of this as a performance bottle neck, but in reality it
seems it isnt .

btw, Wait for some expert to answer, I m  a  beginner too !

On Mon, Oct 26, 2009 at 9:05 PM, Jeff Zhang <zj...@gmail.com> wrote:

> Hi all,
>
> I'd like to know does the map task push map output to reduce task or reduce
> task pull it from map task ? Which way is real in hadoop ?
>
> Thank you very much.
>
>
> Jeff zhang
>



-- 
Hari