You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-issues@hadoop.apache.org by "Jason Lowe (JIRA)" <ji...@apache.org> on 2014/08/19 16:27:18 UTC

[jira] [Assigned] (MAPREDUCE-5891) Improved shuffle error handling across NM restarts

     [ https://issues.apache.org/jira/browse/MAPREDUCE-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Lowe reassigned MAPREDUCE-5891:
-------------------------------------

    Assignee: Junping Du

Thanks, Junping!  I've been busy with other things and haven't started on this yet.

I had a few ideas around either having the AM have a "grace period" for reported errors to cover the NM restart window or have the Fetcher only report shuffle errors after trying for a period of time that would cover the NM restart window.  But I haven't had a chance to prototype those ideas yet.

> Improved shuffle error handling across NM restarts
> --------------------------------------------------
>
>                 Key: MAPREDUCE-5891
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5891
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>    Affects Versions: 2.5.0
>            Reporter: Jason Lowe
>            Assignee: Junping Du
>
> To minimize the number of map fetch failures reported by reducers across an NM restart it would be nice if reducers only reported a fetch failure after trying for at specified period of time to retrieve the data.



--
This message was sent by Atlassian JIRA
(v6.2#6252)