You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-commits@hadoop.apache.org by Apache Wiki <wi...@apache.org> on 2007/08/16 00:04:23 UTC

[Lucene-hadoop Wiki] Update of "FAQ" by DougCutting

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by DougCutting:
http://wiki.apache.org/lucene-hadoop/FAQ

The comment on the change is:
updated chart

------------------------------------------------------------------------------
  
  == 3. How well does Hadoop scale? ==
  
- Hadoop has been demonstrated on clusters of up to 2000 nodes.  Sort performance on 900 nodes is good (sorting 9TB of data on 900 nodes takes around 2.5 hours) and [attachment:sort900-20070630.png improving] using these non-default configuration values:
+ Hadoop has been demonstrated on clusters of up to 2000 nodes.  Sort performance on 900 nodes is good (sorting 9TB of data on 900 nodes takes around 2.25 hours) and [attachment:sort900-20070815.png improving] using these non-default configuration values:
  
    * `dfs.block.size = 134217728`
    * `dfs.namenode.handler.count = 40`
    * `mapred.speculative.execution = true`
    * `mapred.reduce.parallel.copies = 20`
-   * `mapred.tasktracker.tasks.maximum = 2`
    * `mapred.child.java.opts = -Xmx512m`
    * `fs.inmemory.size.mb = 200`
    * `io.sort.factor = 100`