You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by "Arun C Murthy (JIRA)" <ji...@apache.org> on 2009/05/19 09:18:45 UTC

[jira] Created: (HADOOP-5870) Implement a memory-to-memory sort in the map task

Implement a memory-to-memory sort in the map task
-------------------------------------------------

                 Key: HADOOP-5870
                 URL: https://issues.apache.org/jira/browse/HADOOP-5870
             Project: Hadoop Core
          Issue Type: Improvement
          Components: mapred
            Reporter: Arun C Murthy


The motivation is similar to HADOOP-5831...

Currently we collect map-outputs in the sort buffer (io.sort.mb) which we eventually sort and spill to disk. For latency-sensitive applications with sufficient memory, e.g. terasort, we could do better by doing a memory-to-memory sort followed by a final memory-to-disk merge.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.