Max Map tasks not taking effect

[krats <pr...@gmail.com>]

Hi 

I am trying to increase the max number of map tasks on a given machine by
increasing the mapred.tasktracker.map.tasks.maximum parameter to 4 (default
is 2). I tried restarting the tasktracker locally on that machine by usinhg
tasktracker start (using cloudera hadoop) but that doesnt seem to start the
tasktracker. When I restart jobtracker on some other machine, I read
somewhere that it starts the tasktracker but the new config doesnt seem to
take effect.

Help needed !!


-- 
View this message in context: http://old.nabble.com/Max-Map-tasks-not-taking-effect-tp32694843p32694843.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

Re: Max Map tasks not taking effect

[Harsh J <ha...@cloudera.com>]

Hi krats,

Moving your question to the cdh-user@cloudera.org lists (https://groups.google.com/a/cloudera.org/group/cdh-user/topics) since your startup issue may be specific to CDH. I've bcc'd common-user@ meanwhile.

To answer your question, here is how I'd change the tasktracker configuration and restart them using hadoop package installs and the service scripts:

$ ssh user@<tasktracker host>
remote $ sudo vim /etc/hadoop/conf/mapred-site.xml
remote $ # Tweak the property "mapred.tasktracker.map.tasks.maximum" and save the file.
remote $ sudo service hadoop-0.20-tasktracker restart

Verify in JT web UI that your new tracker has come in, and slot count reported by it is the newer one. Lemme know if this isn't the case and we can troubleshoot further.

That should do it. If you are using Cloudera SCM Express instead, please direct your question to the scm-users@cloudera.org group at https://groups.google.com/a/cloudera.org/group/scm-users/topics as the above method is not applicable for a cluster managed by SCM.

On 21-Oct-2011, at 2:13 PM, krats wrote:

> 
> Hi 
> 
> I am trying to increase the max number of map tasks on a given machine by
> increasing the mapred.tasktracker.map.tasks.maximum parameter to 4 (default
> is 2). I tried restarting the tasktracker locally on that machine by usinhg
> tasktracker start (using cloudera hadoop) but that doesnt seem to start the
> tasktracker. When I restart jobtracker on some other machine, I read
> somewhere that it starts the tasktracker but the new config doesnt seem to
> take effect.
> 
> Help needed !!
> 
> 
> -- 
> View this message in context: http://old.nabble.com/Max-Map-tasks-not-taking-effect-tp32694843p32694843.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>