You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by patek tek <pa...@gmail.com> on 2008/12/10 16:36:24 UTC

Copy-rate of reducers decreases over time

Hello,
I have been running experiments with Hadoop and noticed that
the copy-rate of reducers descreases over time (even though there is the
same
number of mappers generating intermediate data).

Is there any obvious answer to this?

Looking forward to your reply!

patektek

Re: Copy-rate of reducers decreases over time

Posted by Jason Venner <ja...@attributor.com>.

The copy rate for the reduces is throttled by the availability of the 
data from the maps.

If the map data is not available yet, the effective copy rate goes toward 0.

patek tek wrote:
> Hello,
> I have been running experiments with Hadoop and noticed that
> the copy-rate of reducers descreases over time (even though there is the
> same
> number of mappers generating intermediate data).
>
> Is there any obvious answer to this?
>
> Looking forward to your reply!
>
> patektek
>
>