You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Vishal Shah <vi...@rediff.co.in> on 2006/09/05 15:20:11 UTC

Setting mapred.tasktracker.tasks.maximum doesn't change # of tasks executed in parallel

Hi,
 
  I am using Nutch 0.9 for crawling. I recollect that
mapred.tasktracker.tasks.maximum can be used to control the max # of
tasks executed in parallel by a tasktracker.
 
  I am running a fetch with the following config:
 
3 machines
 
My mapred-default.xml contains:
 
mapred.map.tasks=13
mapred.reduce.tasks=7
mapred.tasktracker.tasks.maximum=4
 
I ran generate using -numFetchers=12, however while fetching I see that
only 2 tasks are running at a time on each machine (instead of 4).
 
Has anyone had a similar problem before? How can I resolve this?
 
Thanks,
 
-vishal.