You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Hadoop QA (JIRA)" <ji...@apache.org> on 2011/07/13 20:45:00 UTC
[jira] [Commented] (MAPREDUCE-2187) map tasks timeout during
sorting
[ https://issues.apache.org/jira/browse/MAPREDUCE-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064758#comment-13064758 ]
Hadoop QA commented on MAPREDUCE-2187:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12483443/MAPREDUCE-2187-branch-MR-279.patch
against trunk revision 1145889.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/465//console
This message is automatically generated.
> map tasks timeout during sorting
> --------------------------------
>
> Key: MAPREDUCE-2187
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2187
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2, 0.20.205.0
> Reporter: Gianmarco De Francisci Morales
> Assignee: Anupam Seth
> Attachments: MAPREDUCE-2187-20-security.patch, MAPREDUCE-2187-22.patch, MAPREDUCE-2187-branch-MR-279.patch, MAPREDUCE-2187-trunk.patch
>
>
> During the execution of a large job, the map tasks timeout:
> {code}
> INFO mapred.JobClient: Task Id : attempt_201010290414_60974_m_000057_1, Status : FAILED
> Task attempt_201010290414_60974_m_000057_1 failed to report status for 609 seconds. Killing!
> {code}
> The bug is in the fact that the mapper has already finished, and, according to the logs, the timeout occurs during the merge sort phase.
> The intermediate data generated by the map task is quite large. So I think this is the problem.
> The logs show that the merge-sort was running for 10 minutes when the task was killed.
> I think the mapred.Merger should call Reporter.progress() somewhere.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira