You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Nathan Marz <na...@rapleaf.com> on 2009/02/26 23:03:13 UTC
Shuffle phase
Do the reducers batch copy map outputs from a machine? That is, if a
machine M has 15 intermediate map outputs destined for machine R, will
machine R copy the intermediate outputs one at a time or all at once?
Re: Shuffle phase
Posted by Owen O'Malley <om...@apache.org>.
On Feb 26, 2009, at 2:35 PM, Nathan Marz wrote:
> Interesting. Is there a JIRA for this?
Two. HADOOP-5223 subsumes HADOOP-1338.
-- Owen
Re: Shuffle phase
Posted by Nathan Marz <na...@rapleaf.com>.
Interesting. Is there a JIRA for this?
On Feb 26, 2009, at 2:07 PM, Owen O'Malley wrote:
>
> On Feb 26, 2009, at 2:03 PM, Nathan Marz wrote:
>
>> Do the reducers batch copy map outputs from a machine? That is, if
>> a machine M has 15 intermediate map outputs destined for machine R,
>> will machine R copy the intermediate outputs one at a time or all
>> at once?
>
> Currently, one at a time. In 0.21 it will be batched up.
>
> -- Owen
Re: Shuffle phase
Posted by Owen O'Malley <om...@apache.org>.
On Feb 26, 2009, at 2:03 PM, Nathan Marz wrote:
> Do the reducers batch copy map outputs from a machine? That is, if a
> machine M has 15 intermediate map outputs destined for machine R,
> will machine R copy the intermediate outputs one at a time or all at
> once?
Currently, one at a time. In 0.21 it will be batched up.
-- Owen