[jira] Updated: (MAPREDUCE-1030) Reduce tasks are getting starved in capacity scheduler

["Vinod K V (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vinod K V updated MAPREDUCE-1030:
---------------------------------

    Fix Version/s: 0.21.0

This is a serious issue. This happens when using capacity-scheduler with TTs in the cluster configured with more number of map slots than reduce slots, for e.g. in 5:2 ratio.

At times, when there is a steady stream of short jobs with much more number of maps than there are reduces across all jobs, jobs start hanging between map and reduce phases. Reduces are starved and (short) maps keep getting finished quickly and new maps keep getting assigned to TT. Reduces of all the jobs hang for hours together, increasing the total jobs' execution time by even 10X sometimes; this _has_ to be fixed.


> Reduce tasks are getting starved in capacity scheduler
> ------------------------------------------------------
>
>                 Key: MAPREDUCE-1030
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1030
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: contrib/capacity-sched
>    Affects Versions: 0.21.0
>            Reporter: rahul k singh
>            Assignee: rahul k singh
>            Priority: Blocker
>             Fix For: 0.21.0
>
>
> reduce tasks are getting starved in capacity scheduler. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.