You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Harsh J (Resolved) (JIRA)" <ji...@apache.org> on 2011/12/31 10:54:31 UTC
[jira] [Resolved] (MAPREDUCE-68) Hadoop reduce scheduler sometimes
leaves machines idle
[ https://issues.apache.org/jira/browse/MAPREDUCE-68?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J resolved MAPREDUCE-68.
------------------------------
Resolution: Not A Problem
Not a problem with current release schedulers.
> Hadoop reduce scheduler sometimes leaves machines idle
> ------------------------------------------------------
>
> Key: MAPREDUCE-68
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-68
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Nathan Marz
>
> I have a MapReduce application with number of reducers equal to the number of machines in the cluster (and with speculative execution turned off). However, Hadoop schedules multiple reduces to run on single machines and leaves other machines idle. This causes contention and seriously slows down the job. Hadoop should employ the simple heuristic of utilizing as many machines as possible when scheduling reduces.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira