You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Devaraj Das <dd...@yahoo-inc.com> on 2007/09/05 11:29:24 UTC

Sort benchmark on 2000 nodes

This is FYI. We at Yahoo! could successfully run hadoop (upto date trunk
version) on a cluster of 2000 nodes. The programs we ran were RandomWriter
and Sort. Sort performance was pretty good - we could sort 20TB of data in
2.5 hours! Not many task failures - most of those that failed encountered
file checksum errors during merge and map output serving, some got killed
due to lack of progress reporting. Overall, a pretty successful run.


RE: Sort benchmark on 2000 nodes

Posted by Devaraj Das <dd...@yahoo-inc.com>.
Will do so as soon as I get hold of a 1000 node cluster.

> -----Original Message-----
> From: Enis Soztutar [mailto:enis.soz.nutch@gmail.com] 
> Sent: Wednesday, September 05, 2007 6:32 PM
> To: hadoop-user@lucene.apache.org
> Subject: Re: Sort benchmark on 2000 nodes
> 
> I am wondering how hadoop scores on sorting 1TB with say 1000 
> nodes. Is it possible for you to try the Terasort benchmark?
> 
> 
> Devaraj Das wrote:
> > This is FYI. We at Yahoo! could successfully run hadoop 
> (upto date trunk
> > version) on a cluster of 2000 nodes. The programs we ran 
> were RandomWriter
> > and Sort. Sort performance was pretty good - we could sort 
> 20TB of data in
> > 2.5 hours! Not many task failures - most of those that 
> failed encountered
> > file checksum errors during merge and map output serving, 
> some got killed
> > due to lack of progress reporting. Overall, a pretty successful run.
> >
> >
> >   
> 


Re: Sort benchmark on 2000 nodes

Posted by Enis Soztutar <en...@gmail.com>.
I am wondering how hadoop scores on sorting 1TB with say 1000 nodes. Is 
it possible for you to try the Terasort benchmark?


Devaraj Das wrote:
> This is FYI. We at Yahoo! could successfully run hadoop (upto date trunk
> version) on a cluster of 2000 nodes. The programs we ran were RandomWriter
> and Sort. Sort performance was pretty good - we could sort 20TB of data in
> 2.5 hours! Not many task failures - most of those that failed encountered
> file checksum errors during merge and map output serving, some got killed
> due to lack of progress reporting. Overall, a pretty successful run.
>
>
>   

RE: Sort benchmark on 2000 nodes

Posted by Devaraj Das <dd...@yahoo-inc.com>.
Thanks Eric for pointing out the hardware spec. I have updated the
hadoop-config on the hadoop FAQ - http://wiki.apache.org/lucene-hadoop/FAQ 

> -----Original Message-----
> From: Eric Baldeschwieler [mailto:eric14@yahoo-inc.com] 
> Sent: Thursday, September 06, 2007 1:37 PM
> To: hadoop-user@lucene.apache.org
> Subject: Re: Sort benchmark on 2000 nodes
> 
> hardware is similar top that discussed here:
> 
> http://wiki.apache.org/lucene-hadoop-data/attachments/
> HadoopPresentations/attachments/oscon-part-2.pdf
> 
> - 10:1 oversubscribed network (so 100mBit bandwidth all nodes to all
> nodes)
> - 40 nodes / leaf switch
> - Machines are beefy
>    - 4SATA drives, 500 or 750 GB each, 7200 RPM
>    - 4+ cores (modern Intels or AMDs)
>    - 4+ GB RAM
> 
> On Sep 5, 2007, at 10:19 AM, Joydeep Sen Sarma wrote:
> 
> > It will be very useful to see the hadoop/job config 
> settings and get 
> > some sense of the underlying hardware config.
> >
> > -----Original Message-----
> > From: Devaraj Das [mailto:ddas@yahoo-inc.com]
> > Sent: Wednesday, September 05, 2007 2:29 AM
> > To: hadoop-user@lucene.apache.org
> > Subject: Sort benchmark on 2000 nodes
> >
> > This is FYI. We at Yahoo! could successfully run hadoop (upto date 
> > trunk
> > version) on a cluster of 2000 nodes. The programs we ran were 
> > RandomWriter and Sort. Sort performance was pretty good - we could 
> > sort 20TB of data in
> > 2.5 hours! Not many task failures - most of those that failed 
> > encountered file checksum errors during merge and map 
> output serving, 
> > some got killed due to lack of progress reporting. Overall, 
> a pretty 
> > successful run.
> >
> >
> 
> 


Re: Sort benchmark on 2000 nodes

Posted by Eric Baldeschwieler <er...@yahoo-inc.com>.
hardware is similar top that discussed here:

http://wiki.apache.org/lucene-hadoop-data/attachments/ 
HadoopPresentations/attachments/oscon-part-2.pdf

- 10:1 oversubscribed network (so 100mBit bandwidth all nodes to all  
nodes)
- 40 nodes / leaf switch
- Machines are beefy
   - 4SATA drives, 500 or 750 GB each, 7200 RPM
   - 4+ cores (modern Intels or AMDs)
   - 4+ GB RAM

On Sep 5, 2007, at 10:19 AM, Joydeep Sen Sarma wrote:

> It will be very useful to see the hadoop/job config settings and get
> some sense of the underlying hardware config.
>
> -----Original Message-----
> From: Devaraj Das [mailto:ddas@yahoo-inc.com]
> Sent: Wednesday, September 05, 2007 2:29 AM
> To: hadoop-user@lucene.apache.org
> Subject: Sort benchmark on 2000 nodes
>
> This is FYI. We at Yahoo! could successfully run hadoop (upto date  
> trunk
> version) on a cluster of 2000 nodes. The programs we ran were
> RandomWriter
> and Sort. Sort performance was pretty good - we could sort 20TB of  
> data
> in
> 2.5 hours! Not many task failures - most of those that failed
> encountered
> file checksum errors during merge and map output serving, some got
> killed
> due to lack of progress reporting. Overall, a pretty successful run.
>
>


RE: Sort benchmark on 2000 nodes

Posted by Joydeep Sen Sarma <js...@facebook.com>.
It will be very useful to see the hadoop/job config settings and get
some sense of the underlying hardware config.

-----Original Message-----
From: Devaraj Das [mailto:ddas@yahoo-inc.com] 
Sent: Wednesday, September 05, 2007 2:29 AM
To: hadoop-user@lucene.apache.org
Subject: Sort benchmark on 2000 nodes

This is FYI. We at Yahoo! could successfully run hadoop (upto date trunk
version) on a cluster of 2000 nodes. The programs we ran were
RandomWriter
and Sort. Sort performance was pretty good - we could sort 20TB of data
in
2.5 hours! Not many task failures - most of those that failed
encountered
file checksum errors during merge and map output serving, some got
killed
due to lack of progress reporting. Overall, a pretty successful run.