You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by "Todd Lipcon (Created) (JIRA)" <ji...@apache.org> on 2012/03/01 02:27:57 UTC

[jira] [Created] (MAPREDUCE-3951) Tasks are not evenly spread throughout cluster in MR2

Tasks are not evenly spread throughout cluster in MR2
-----------------------------------------------------

                 Key: MAPREDUCE-3951
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3951
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: scheduler
    Affects Versions: 0.23.0, 0.24.0
            Reporter: Todd Lipcon


In MR1 (at least with the fair and fifo schedulers), if you submit a job that needs fewer resources than the cluster can provide, the tasks are spread relatively evenly across the node. For example, submitting a 100-map job to a 50-node cluster, each with 10 slots, results in 2 tasks on each machine. In MR2, however, the tasks would pile up on the first 10 nodes of the cluster, leaving the other nodes unused. This is highly suboptimal for many use cases.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira