Setting Number of Maps in 0.20.2

[Jason Reed <jm...@asu.edu>]

I am trying to figure out how to set the number of maps to use in 0.20.2.

If I was using JobConf in my program I could use:

conf.setNumMapTasks(numMaps);

However JobConf and the method are deprecated and when we started our
project we structured everything to use Configuration and Job because of
this. Is there a way to set the number of map tasks using Job and
Configuration?

Thank you for any help,
Jason

Re: Setting Number of Maps in 0.20.2

[Harsh J <ha...@cloudera.com>]

Jason,

Setting map tasks is no longer allowed (as it doesn't make perfect
sense -- map tasks are determined based on your input files and the
file splits (hint: this is where you should be tweaking, if that's
your goal)). You can instead set a minimum split size to control your
map task parallelism: "mapred.min.split.size" as documented at
http://hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hadoop/mapreduce/InputFormat.html

On Mon, Aug 15, 2011 at 2:20 AM, Jason Reed <jm...@asu.edu> wrote:
> I am trying to figure out how to set the number of maps to use in 0.20.2.
>
> If I was using JobConf in my program I could use:
>
> conf.setNumMapTasks(numMaps);
>
> However JobConf and the method are deprecated and when we started our
> project we structured everything to use Configuration and Job because of
> this. Is there a way to set the number of map tasks using Job and
> Configuration?
>
> Thank you for any help,
> Jason
>



-- 
Harsh J