You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by Sebastian Rinke <ri...@cs.tu-darmstadt.de> on 2016/11/25 17:08:21 UTC

Hadoop 2.7.3: Restrict number of concurrent tasks

Dear all,

Please apologize if this is the wrong list for my question,
however, I thought it’s most likely that I get an answer here.

Question:

I would like to restrict the number of concurrently running tasks of a MapReduce job.
For this, I set the property mapreduce.jobtracker.maxtasks.perjob with the value 1
in mapred-site.xml.

Unfortunately, this setting is ignored and more than one mapper/reducer is run
concurrently.

Could you please help me with this problem.

Thank you very much in advance.

Best,
Sebastian



---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: Hadoop 2.7.3: Restrict number of concurrent tasks

Posted by Sebastian Rinke <ri...@cs.tu-darmstadt.de>.

Thanks a lot for the quick response!
I give it a try.

Thanks,
Sebastian

On 25 Nov 2016, at 18:37, Zhihai Xu <zh...@uber.com> wrote:

> Hi Sebastian,
> 
> The configuration to limit concurrently running tasks of a MapReduce job is the following:
> 
> <property>
>   <name>mapreduce.job.running.map.limit</name>
>   <value>0</value>
>   <description>The maximum number of simultaneous map tasks per job.
>   There is no limit if this value is 0 or negative.
>   </description>
> </property>
> 
> <property>
>   <name>mapreduce.job.running.reduce.limit</name>
>   <value>0</value>
>   <description>The maximum number of simultaneous reduce tasks per job.
>   There is no limit if this value is 0 or negative.
>   </description>
> </property>
> 
> The related JIRA is https://issues.apache.org/jira/browse/MAPREDUCE-5583
> 
> Looks like MAPREDUCE-5583 is fixed in 2.7.0.
> 
> Regards
> zhihai
> 
> 
> On Fri, Nov 25, 2016 at 9:08 AM, Sebastian Rinke <ri...@cs.tu-darmstadt.de> wrote:
> Dear all,
> 
> Please apologize if this is the wrong list for my question,
> however, I thought it’s most likely that I get an answer here.
> 
> Question:
> 
> I would like to restrict the number of concurrently running tasks of a MapReduce job.
> For this, I set the property mapreduce.jobtracker.maxtasks.perjob with the value 1
> in mapred-site.xml.
> 
> Unfortunately, this setting is ignored and more than one mapper/reducer is run
> concurrently.
> 
> Could you please help me with this problem.
> 
> Thank you very much in advance.
> 
> Best,
> Sebastian
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
> 
>

Re: Hadoop 2.7.3: Restrict number of concurrent tasks

Posted by Zhihai Xu <zh...@uber.com>.

Hi Sebastian,

The configuration to limit concurrently running tasks of a MapReduce job is
the following:

<property>
  <name>mapreduce.job.running.map.limit</name>
  <value>0</value>
  <description>The maximum number of simultaneous map tasks per job.
  There is no limit if this value is 0 or negative.
  </description>
</property>

<property>
  <name>mapreduce.job.running.reduce.limit</name>
  <value>0</value>
  <description>The maximum number of simultaneous reduce tasks per job.
  There is no limit if this value is 0 or negative.
  </description>
</property>

The related JIRA is https://issues.apache.org/jira/browse/MAPREDUCE-5583

Looks like MAPREDUCE-5583 is fixed in 2.7.0.

Regards
zhihai

On Fri, Nov 25, 2016 at 9:08 AM, Sebastian Rinke <ri...@cs.tu-darmstadt.de>
wrote:

> Dear all,
>
> Please apologize if this is the wrong list for my question,
> however, I thought it’s most likely that I get an answer here.
>
> Question:
>
> I would like to restrict the number of concurrently running tasks of a
> MapReduce job.
> For this, I set the property mapreduce.jobtracker.maxtasks.perjob with
> the value 1
> in mapred-site.xml.
>
> Unfortunately, this setting is ignored and more than one mapper/reducer is
> run
> concurrently.
>
> Could you please help me with this problem.
>
> Thank you very much in advance.
>
> Best,
> Sebastian
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>
>