Are the input values (associated with a specific key) of reduce() method sorted?

["James Y. Li" <yu...@gmail.com>]

Hi All,
I just begin to use Hadoop framework, and write my reduce() method. Wonder
if the Iterable input values (associated with the input key) are already
sorted. Thanks!

p.s. I am using the version 0.20.2-cdh3u1

-james

-- 
* <http://www.linkedin.com/in/jamesjamesli>*

Re: Are the input values (associated with a specific key) of reduce() method sorted?

[Joey Echeverria <jo...@cloudera.com>]

Hi James,

By default, there is no guarantees on value order. Using some of the
more advanced API features, you can perform a secondary sort of
values. You can read a good example of it here:

http://sonerbalkir.blogspot.com/2010/01/simulating-secondary-sort-on-values.html

-Joey

On Mon, Dec 5, 2011 at 7:55 PM, James Y. Li <yu...@gmail.com> wrote:
> Hi All,
> I just begin to use Hadoop framework, and write my reduce() method. Wonder
> if the Iterable input values (associated with the input key) are already
> sorted. Thanks!
>
> p.s. I am using the version 0.20.2-cdh3u1
>
> -james
>
> --
>
>



-- 
Joseph Echeverria
Cloudera, Inc.
443.305.9434

Re: Are the input values (associated with a specific key) of reduce() method sorted?

[Harsh J <ha...@cloudera.com>]

James,

No, values do not come in any order for a given key. But it is achievable.

If you need them to be ordered, you need to perform a secondary sort operation, by using multiple key components.

See this 3-part series on how to do that, for example: http://www.cloudera.com/blog/2011/04/simple-moving-average-secondary-sort-and-mapreduce-part-3/

On 06-Dec-2011, at 6:25 AM, James Y. Li wrote:

> Hi All,
> I just begin to use Hadoop framework, and write my reduce() method. Wonder if the Iterable input values (associated with the input key) are already sorted. Thanks!
> 
> p.s. I am using the version 0.20.2-cdh3u1
> 
> -james
> 
> -- 
> 
>