You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by Karol Rybak <ka...@gmail.com> on 2007/11/06 14:23:21 UTC

Reduce copy slow ?

I installed latest nightly build and it seems that my nutch is finally
stable, however i was wondering about one thing running reduce tasks are
showing something like:
reduce > copy (54 of 153 at 0.00 MB/s) why is it 0.00 MB/s ? sometimes i get
0.01 or 0.02 but it's still very slow. Is that normal or is this caused by
some bad configuration ? I'm using a large number of maps (150) and 9 reduce
jobs on a cluster of 3 machines. I tried lowering those, but i kept on
getting memory problems. Could it be caused by small amount of data in each
of map tasks ?

-- 
Karol Rybak
Programista / Programmer
Sekcja aplikacji / Applications section
Wyższa Szkoła Informatyki i Zarządzania / University of Internet Technology
and Management
+48(17)8661277