You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Anurup Pavuluri <ap...@andrew.cmu.edu> on 2006/12/06 20:35:19 UTC

MapReduce parameter sensitivity

Hi,

I was wondering if anyone could throw some light on how the performance 
of MapReduce is effected by parameters such as number of Maps, number of 
Reduces, buffer sizes etc.

If someone could tell which parameters effect the performance I will try 
to experiment with them myself.

Thanks,
Anurup

Re: MapReduce parameter sensitivity

Posted by howard chen <ho...@gmail.com>.

On 12/7/06, Anurup Pavuluri <ap...@andrew.cmu.edu> wrote:
>
> Hi,
>
> I was wondering if anyone could throw some light on how the performance
> of MapReduce is effected by parameters such as number of Maps, number of
> Reduces, buffer sizes etc.
>
> If someone could tell which parameters effect the performance I will try
> to experiment with them myself.
>
> Thanks,
> Anurup
>
>

from the hadoop page, i forgot the link,said that...

if numbers of node, n

mapper = n * 10
reducer = n * 3

would give good performance

Re: MapReduce parameter sensitivity

Posted by Lee <le...@gmail.com>.

This wiki page should help:
http://wiki.apache.org/lucene-hadoop/HowManyMapsAndReduces

On 12/6/06, Anurup Pavuluri <ap...@andrew.cmu.edu> wrote:
>
>
> Hi,
>
> I was wondering if anyone could throw some light on how the performance
> of MapReduce is effected by parameters such as number of Maps, number of
> Reduces, buffer sizes etc.
>
> If someone could tell which parameters effect the performance I will try
> to experiment with them myself.
>
> Thanks,
> Anurup
>
>