You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by patektek <pa...@gmail.com> on 2008/12/11 03:13:24 UTC

Question: why does the copy-rate of reducers decreases over time?

Hello,
I have noticed that for the simple wordcount example on 60GB of data
distributed across 3 data nodes and with 10 tasktrackers (including the 3
data nodes)
the copy-rate of reducers decreases as the job progresses. It starts with
~10MB/s
and by the end (95% mappers complete) is ~2MB/s

Is there any obvious answer to this?

Looking forward to your reply!

patektek