You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Matthew John <tm...@gmail.com> on 2010/10/19 13:47:35 UTC

Reduce groups

Hi all,

The number of Reducer groups in my MapReduce is always the same as the
number of records output by the MapReduce. So what I understand is every
record from the Shuffle/Sort is going to different Reducer.reduce. How can I
change this? My key is BytesWritable and I tried writing my own Comparator
and set it in setOutputValueGroupingClass but still not more than one record
is entering the same reduce group. Someone please tell me the mechanism
behind this so that I can fix this problem . I am not caring about
Partitioner since I am using a single reducer.

Thanks,

Matthew