You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by "yanglongfei (JIRA)" <ji...@apache.org> on 2019/07/01 07:23:00 UTC

[jira] [Created] (MAPREDUCE-7222) Map tasks' outputs can not be recovered when ApplicationMaster relaunched

yanglongfei created MAPREDUCE-7222:
--------------------------------------

             Summary: Map tasks' outputs can not be recovered  when ApplicationMaster relaunched
                 Key: MAPREDUCE-7222
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7222
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mrv2
    Affects Versions: 2.7.3
            Reporter: yanglongfei


When AM crashes, Yarn would launch a new AM instance and recover all its scheduled tasks. However mapper tasks's committed output files are not recovered when the number of reducers > 0. In my application which output files from mapper and make use of reducer to collect statistics not able to fully recover from the AM crash, and resulting in data from the previous completed mapper tasks get lost in the final output dir.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org