You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Milind Bhandarkar (JIRA)" <ji...@apache.org> on 2011/07/26 22:22:09 UTC
[jira] [Commented] (MAPREDUCE-2729) Reducers are always counted
having "pending tasks" even if they can't be scheduled yet because not
enough of their mappers have completed
[ https://issues.apache.org/jira/browse/MAPREDUCE-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13071333#comment-13071333 ]
Milind Bhandarkar commented on MAPREDUCE-2729:
----------------------------------------------
It would be good to have a notion of a "ready" task, which is separate from a pending task.
> Reducers are always counted having "pending tasks" even if they can't be scheduled yet because not enough of their mappers have completed
> -----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2729
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2729
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 0.20.205.0
> Environment: 0.20.1xx-Secondary
> Reporter: Sherry Chen
> Assignee: Sherry Chen
> Fix For: 0.20.205.0
>
>
> In capacity scheduler, number of users in a queue needing slots are calculated based on whether users' jobs have any pending tasks.
> This works fine for map tasks. However, for reduce tasks, jobs do not need reduce slots until the minimum number of map tasks have been completed.
> Here, we add checking whether reduce is ready to schedule (i.e. if a job has completed enough map tasks) when we increment number of users in a queue needing reduce slots.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira