You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Tenaali Ram <te...@gmail.com> on 2008/09/12 22:58:30 UTC

Tips on sorting using Hadoop

Hi,
I want to sort my records ( consisting of string, int, float) using Hadoop.

One way I have found is to set number of reducers = 1, but this would mean
all the records go to 1 reducer and it won't be optimized. Can anyone point
me to some better way to do sorting using Hadoop ?

Thanks,
Tenaali

Re: Tips on sorting using Hadoop

Posted by "Edward J. Yoon" <ed...@apache.org>.

I would recommend that run map/reduce twice.

/Edward

On Sat, Sep 13, 2008 at 5:58 AM, Tenaali Ram <te...@gmail.com> wrote:
> Hi,
> I want to sort my records ( consisting of string, int, float) using Hadoop.
>
> One way I have found is to set number of reducers = 1, but this would mean
> all the records go to 1 reducer and it won't be optimized. Can anyone point
> me to some better way to do sorting using Hadoop ?
>
> Thanks,
> Tenaali
>



-- 
Best regards, Edward J. Yoon
edwardyoon@apache.org
http://blog.udanax.org