You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Amar Kamat (JIRA)" <ji...@apache.org> on 2008/07/02 06:38:45 UTC
[jira] Commented: (HADOOP-3682) duplicate "Map Completion Events"
in tasktracker fetch status
[ https://issues.apache.org/jira/browse/HADOOP-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12609794#action_12609794 ]
Amar Kamat commented on HADOOP-3682:
------------------------------------
Having duplicate map-completion-events should not stall the reducers. The ReduceTask takes care of duplicate events. A reduce task will keep on fetching map-completion-events until all the map outputs are copied. Duplicate map-completion-events can be the outcome of some bug. Is there any way to reproduce it?
> duplicate "Map Completion Events" in tasktracker fetch status
> -------------------------------------------------------------
>
> Key: HADOOP-3682
> URL: https://issues.apache.org/jira/browse/HADOOP-3682
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.15.3
> Reporter: Zheng Shao
> Assignee: Zheng Shao
>
> we saw duplicate "MapCompletionEvents" in FetchStatus from TaskTracker. This prevents the ReduceTask from getting all the map locations, and the ReduceTask is waiting forever.
> This is a printout of the MapCompletionEvents from a FetchStatus from the TaskTracker:
> 988 0 task_200806270248_6185_m_000980_0 980 Map SUCCEEDED
> 989 0 task_200806270248_6185_m_000787_0 787 Map SUCCEEDED
> 990 0 task_200806270248_6185_m_000749_0 749 Map SUCCEEDED
> 991 0 task_200806270248_6185_m_000848_0 848 Map SUCCEEDED
> 992 0 task_200806270248_6185_m_000919_0 919 Map SUCCEEDED
> 993 0 task_200806270248_6185_m_000836_0 836 Map SUCCEEDED
> 994 0 task_200806270248_6185_m_000838_0 838 Map SUCCEEDED
> 995 0 task_200806270248_6185_m_000936_0 936 Map SUCCEEDED
> 996 0 task_200806270248_6185_m_000933_0 933 Map SUCCEEDED
> 997 0 task_200806270248_6185_m_000856_0 856 Map SUCCEEDED
> 998 0 task_200806270248_6185_m_000885_0 885 Map SUCCEEDED
> 999 0 task_200806270248_6185_m_000804_0 804 Map SUCCEEDED
> 1000 0 task_200806270248_6185_m_000986_0 986 Map SUCCEEDED *first copy begins*
> 1001 0 task_200806270248_6185_m_000999_0 999 Map SUCCEEDED
> 1002 0 task_200806270248_6185_m_000990_0 990 Map SUCCEEDED
> 1003 0 task_200806270248_6185_m_000985_0 985 Map SUCCEEDED
> 1004 0 task_200806270248_6185_m_000968_0 968 Map SUCCEEDED
> 1005 0 task_200806270248_6185_m_001000_0 1000 Map SUCCEEDED
> 1006 0 task_200806270248_6185_m_000996_0 996 Map SUCCEEDED
> 1007 0 task_200806270248_6185_m_000974_0 974 Map SUCCEEDED
> 1008 0 task_200806270248_6185_m_000998_0 998 Map SUCCEEDED
> 1009 0 task_200806270248_6185_m_000982_0 982 Map SUCCEEDED
> 1010 0 task_200806270248_6185_m_001010_0 1010 Map SUCCEEDED
> 1011 0 task_200806270248_6185_m_000959_0 959 Map SUCCEEDED
> 1012 0 task_200806270248_6185_m_000991_0 991 Map SUCCEEDED *first copy ends*
> 1013 0 task_200806270248_6185_m_000986_0 986 Map SUCCEEDED *second copy begins*
> 1014 0 task_200806270248_6185_m_000999_0 999 Map SUCCEEDED
> 1015 0 task_200806270248_6185_m_000990_0 990 Map SUCCEEDED
> 1016 0 task_200806270248_6185_m_000985_0 985 Map SUCCEEDED
> 1017 0 task_200806270248_6185_m_000968_0 968 Map SUCCEEDED
> 1018 0 task_200806270248_6185_m_001000_0 1000 Map SUCCEEDED
> 1019 0 task_200806270248_6185_m_000996_0 996 Map SUCCEEDED
> 1020 0 task_200806270248_6185_m_000974_0 974 Map SUCCEEDED
> 1021 0 task_200806270248_6185_m_000998_0 998 Map SUCCEEDED
> 1022 0 task_200806270248_6185_m_000982_0 982 Map SUCCEEDED
> 1023 0 task_200806270248_6185_m_001010_0 1010 Map SUCCEEDED
> 1024 0 task_200806270248_6185_m_000959_0 959 Map SUCCEEDED
> 1025 0 task_200806270248_6185_m_000991_0 991 Map SUCCEEDED *second copy ends*
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.