You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Scott Chen (JIRA)" <ji...@apache.org> on 2009/10/22 00:20:59 UTC
[jira] Created: (MAPREDUCE-1129) Assign multiple Map and Reduce
tasks in Fairscheduler
Assign multiple Map and Reduce tasks in Fairscheduler
-----------------------------------------------------
Key: MAPREDUCE-1129
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1129
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: contrib/fair-share
Affects Versions: 0.20.1
Reporter: Scott Chen
In Hadoop-0.20, the period of heartbeat becomes much longer.
Fairscheduler assigns at most one Map and one Reduce task per heartbeat.
This makes the cluster become very inefficient. Often time only half of the slots are used.
One idea is that we make Fairscheduler detect this situation (cluster under used) and scheduler more tasks in a heartbeat.
Thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (MAPREDUCE-1129) Assign multiple Map and Reduce
tasks in Fairscheduler
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon resolved MAPREDUCE-1129.
------------------------------------
Resolution: Duplicate
Resolving as duplicate since this is in MAPREDUCE-706. Regarding the backport for Cloudera's distro, we will definitely include this in 0.20 in CDH3, and it's looking like we may backport for CDH2 in the coming weeks as well.
> Assign multiple Map and Reduce tasks in Fairscheduler
> -----------------------------------------------------
>
> Key: MAPREDUCE-1129
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1129
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/fair-share
> Affects Versions: 0.20.1
> Reporter: Scott Chen
>
> In Hadoop-0.20, the period of heartbeat becomes much longer.
> Fairscheduler assigns at most one Map and one Reduce task per heartbeat.
> This makes the cluster become very inefficient. Often time only half of the slots are used.
> One idea is that we make Fairscheduler detect this situation (cluster under used) and scheduler more tasks in a heartbeat.
> Thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.