You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Andy Doddington <an...@doddington.net> on 2011/11/21 14:41:09 UTC

Problem understanding reduce and combine

Hi,

I have a Hadoop app in which each mapper generates a floating point result which I output, along with a single LongWritable key, set to zero.

My reducer receives a set of the results from which I need to select the minimum. I output the minimum as the result, again as a zero-valued key along with a floating point value (the minimum).

According to my understanding, I ought to be able to use my reducer as a combiner. However, when I do this, my final reducer doesn’t seem to get passed any values at all, although the reducers running as combiners are being called and seem to be producing the correct output.

Note that I have deliberately not done anything ‘clever’, such a defining custom partitioners or other customisation.

Any help most appreciated,

	Andy Doddington