You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-issues@hadoop.apache.org by "Yang Hao (JIRA)" <ji...@apache.org> on 2015/01/28 01:54:34 UTC

[jira] [Commented] (MAPREDUCE-6229) finished MapReduce tasks will be re executed when AM fails on recovery process

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294515#comment-14294515 ] 

Yang Hao commented on MAPREDUCE-6229:
-------------------------------------

The history file is there in hdfs, and we can recover these information. Reduce output will always be right if done, and if some reduce tasks have not finished and some map tasks failed for nodemanager crashed, the reduce will tell AM that it cannot get data from map tasks, so the map will be reexecuted.

> finished MapReduce tasks will be re executed when AM fails on recovery process
> ------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6229
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6229
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.6.0
>            Reporter: Yang Hao
>
> MapReduce AM failovers, and suddently it is killed during recovery. Next time when the AM failovers again, the progress will lose for that it only parses previous history file. As a result,
> * many tasks will be reexecuted.
> * users will not see the history, such as the tasks killed or finished at the first attempt.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)