You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by "Karthik Kambatla (JIRA)" <ji...@apache.org> on 2015/10/05 09:00:35 UTC

[jira] [Created] (MAPREDUCE-6501) Improve reducer preemption

Karthik Kambatla created MAPREDUCE-6501:
-------------------------------------------

             Summary: Improve reducer preemption
                 Key: MAPREDUCE-6501
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6501
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: applicationmaster
    Affects Versions: 2.7.1
            Reporter: Karthik Kambatla
            Assignee: Karthik Kambatla


As discussed on MAPREDUCE-6302, we could improve the reducer preemption as follows:
# preempt enough reducers so there are enough mappers to match slowstart 
# prioritize preempting reducers that are still in SHUFFLE phase
# add an option to not preempt reducers that are past SHUFFLE phase irrespective of slowstart as long as one mapper can run

We could do it all in one patch or create subtasks as necessary. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)