You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Oleg Ruchovets <or...@gmail.com> on 2013/02/11 12:45:00 UTC
max number of map/reduce per node
Hi
I found that my job runs with such parameters:
mapred.tasktracker.map.tasks.maximum 4
mapred.tasktracker.reduce.tasks.maximum 2
I try to change these parameters from my java code
Properties properties = new Properties();
properties.put("mapred.tasktracker.map.tasks.maximum" , "8");
properties.put("mapred.tasktracker.reduce.tasks.maximum" , "4");
But executing the job I didn't get updated values of these parameters , it
remains:
mapred.tasktracker.map.tasks.maximum 4
mapred.tasktracker.reduce.tasks.maximum 2
Should I change the parameters on hadoop XML configuration files?
Please advice.
Re: max number of map/reduce per node
Posted by Harsh J <ha...@cloudera.com>.
Hi,
My reply inline.
On Mon, Feb 11, 2013 at 5:15 PM, Oleg Ruchovets <or...@gmail.com> wrote:
> Hi
> I found that my job runs with such parameters:
> mapred.tasktracker.map.tasks.maximum 4
> mapred.tasktracker.reduce.tasks.maximum 2
>
> I try to change these parameters from my java code
>
> Properties properties = new Properties();
> properties.put("mapred.tasktracker.map.tasks.maximum" , "8");
> properties.put("mapred.tasktracker.reduce.tasks.maximum" , "4");
These properties are a per-tasktracker configuration, not applicable
or read from clients.
Also, if you're tweaking client-end properties, using the Java
Properties class is not the right way to go about it. See
Configuration API:
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/conf/Configuration.html
> But executing the job I didn't get updated values of these parameters , it
> remains:
>
> mapred.tasktracker.map.tasks.maximum 4
> mapred.tasktracker.reduce.tasks.maximum 2
>
>
> Should I change the parameters on hadoop XML configuration files?
Yes, as these are per *tasktracker* properties, not client ones.
> Please advice.
>
>
>
>
>
>
--
Harsh J
Re: max number of map/reduce per node
Posted by Harsh J <ha...@cloudera.com>.
Hi,
My reply inline.
On Mon, Feb 11, 2013 at 5:15 PM, Oleg Ruchovets <or...@gmail.com> wrote:
> Hi
> I found that my job runs with such parameters:
> mapred.tasktracker.map.tasks.maximum 4
> mapred.tasktracker.reduce.tasks.maximum 2
>
> I try to change these parameters from my java code
>
> Properties properties = new Properties();
> properties.put("mapred.tasktracker.map.tasks.maximum" , "8");
> properties.put("mapred.tasktracker.reduce.tasks.maximum" , "4");
These properties are a per-tasktracker configuration, not applicable
or read from clients.
Also, if you're tweaking client-end properties, using the Java
Properties class is not the right way to go about it. See
Configuration API:
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/conf/Configuration.html
> But executing the job I didn't get updated values of these parameters , it
> remains:
>
> mapred.tasktracker.map.tasks.maximum 4
> mapred.tasktracker.reduce.tasks.maximum 2
>
>
> Should I change the parameters on hadoop XML configuration files?
Yes, as these are per *tasktracker* properties, not client ones.
> Please advice.
>
>
>
>
>
>
--
Harsh J
Re: max number of map/reduce per node
Posted by Harsh J <ha...@cloudera.com>.
Hi,
My reply inline.
On Mon, Feb 11, 2013 at 5:15 PM, Oleg Ruchovets <or...@gmail.com> wrote:
> Hi
> I found that my job runs with such parameters:
> mapred.tasktracker.map.tasks.maximum 4
> mapred.tasktracker.reduce.tasks.maximum 2
>
> I try to change these parameters from my java code
>
> Properties properties = new Properties();
> properties.put("mapred.tasktracker.map.tasks.maximum" , "8");
> properties.put("mapred.tasktracker.reduce.tasks.maximum" , "4");
These properties are a per-tasktracker configuration, not applicable
or read from clients.
Also, if you're tweaking client-end properties, using the Java
Properties class is not the right way to go about it. See
Configuration API:
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/conf/Configuration.html
> But executing the job I didn't get updated values of these parameters , it
> remains:
>
> mapred.tasktracker.map.tasks.maximum 4
> mapred.tasktracker.reduce.tasks.maximum 2
>
>
> Should I change the parameters on hadoop XML configuration files?
Yes, as these are per *tasktracker* properties, not client ones.
> Please advice.
>
>
>
>
>
>
--
Harsh J
Re: max number of map/reduce per node
Posted by Harsh J <ha...@cloudera.com>.
Hi,
My reply inline.
On Mon, Feb 11, 2013 at 5:15 PM, Oleg Ruchovets <or...@gmail.com> wrote:
> Hi
> I found that my job runs with such parameters:
> mapred.tasktracker.map.tasks.maximum 4
> mapred.tasktracker.reduce.tasks.maximum 2
>
> I try to change these parameters from my java code
>
> Properties properties = new Properties();
> properties.put("mapred.tasktracker.map.tasks.maximum" , "8");
> properties.put("mapred.tasktracker.reduce.tasks.maximum" , "4");
These properties are a per-tasktracker configuration, not applicable
or read from clients.
Also, if you're tweaking client-end properties, using the Java
Properties class is not the right way to go about it. See
Configuration API:
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/conf/Configuration.html
> But executing the job I didn't get updated values of these parameters , it
> remains:
>
> mapred.tasktracker.map.tasks.maximum 4
> mapred.tasktracker.reduce.tasks.maximum 2
>
>
> Should I change the parameters on hadoop XML configuration files?
Yes, as these are per *tasktracker* properties, not client ones.
> Please advice.
>
>
>
>
>
>
--
Harsh J