You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-user@hadoop.apache.org by Lin Yang <li...@gmail.com> on 2013/11/20 13:42:19 UTC

Should I set mapred.tasktracker.map.tasks.maximum = number of processor or number of cores?

Hi, all,

I'm running hadoop on a cluster consisting of 2 data nodes, each of which
has 24 CPUs (intel Xeon X5670@2.93G) and each CPU has 6 cores. So, totally
144 cores on a single node.

In this case, what value should I set for these parameters?


   - mapred.tasktracker.map.tasks.maximum
   - mapred.map.tasks
   - mapred.tasktracker.reduce.tasks.maximum
   - mapred.reduce.tasks

Actually, I've searched the answer on Internet, but I've been confused
since some articles said these should be related to #processors and the
others said it should be related to #cores.

Could anyone give me a confirmed formulation to calculate these parameters?

Thanks.

--
Lin Yang