[jira] [Created] (MAPREDUCE-6679) on node failure, only restart mappers whose output is not copied

["Alvin Chyan (JIRA)" <ji...@apache.org>]

Alvin Chyan created MAPREDUCE-6679:
--------------------------------------

             Summary: on node failure, only restart mappers whose output is not copied
                 Key: MAPREDUCE-6679
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6679
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: mrv2
    Affects Versions: 2.7.0
            Reporter: Alvin Chyan
            Priority: Minor


When we detect a bad node, we reschedule all succeeded map tasks on that node in JobImpl.actOnUnusableNode. Wouldn't we be able to get away with only rescheduling the map tasks that have not had their outputs copied to a reducer already?

One consideration could be that the reducer that fetched the mapper output is then killed itself. However, in testing, it seems that once a reducer has moved past the shuffle phase and is reducing, even if the mapper node fails, the mappers don't get rescheduled. The same mechanism that occurs then if a reducer dies can then be applied in this scenario.

This is helpful in general, but is especially beneficial in cloud environments that offer spot/preemptible instances. As long as reducers are running to continually fetch mapper outputs, the job can make progress as long as the preemptible instances stay up long enough for a map task to complete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)