[jira] Commented: (HADOOP-2175) Blacklisted hosts may not be able to serve map outputs

["Amar Kamat (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12576085#action_12576085 ] 

Amar Kamat commented on HADOOP-2175:
------------------------------------

HADOOP-1984 will cause more waiting here. Since the time for sending the fetch failures will be more. The simplest solution seems to reschedule all the maps on the black listed node. The question is can we do better? Can the JT infer and handle this situation?

> Blacklisted hosts may not be able to serve map outputs
> ------------------------------------------------------
>
>                 Key: HADOOP-2175
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2175
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Runping Qi
>            Assignee: Arun C Murthy
>
> After a node fails 4 mappers (tasks), it is added to blacklist thus it will no longer accept tasks.
> But, it will continue serve the map outputs of any mappers that ran successfully there. 
> However, the node may not be able serve the map outputs either. 
> This will cause the reducers to mark the corresponding map outputs as from slow hosts, 
> but continue to try to get the map outputs from that node.
> This may lead to waiting forever.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.