You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2011/02/01 02:56:29 UTC
[jira] Updated: (MAPREDUCE-1716) Truncate logs of finished tasks to
prevent node thrash due to excessive logging
[ https://issues.apache.org/jira/browse/MAPREDUCE-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated MAPREDUCE-1716:
-----------------------------------
Attachment: mapreduce-1716-testcase-race.txt
This test case has a bit of a race, since it assumes that the logs will be truncated immediately upon job completion.
This isn't the case, since the logs aren't added to the truncation manager until the JVM finished, which can be several seconds after the last task finishes (eg when sleepTimeBeforeSigKill is 5 seconds). So, I found that this test was flaky as is.
You can show this by adding a 10 second sleep before truncating the logs, for example.
This delta patch has the tests loop for 20 seconds while checking the logs for truncation. It only fails if the logs aren't truncated after 20 seconds.
> Truncate logs of finished tasks to prevent node thrash due to excessive logging
> -------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1716
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1716
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: task, tasktracker
> Affects Versions: 0.20.3, 0.21.0, 0.21.1, 0.22.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1100-20091216.2.txt, mapreduce-1716-testcase-race.txt, patch-1100-fix-ydist.2.txt, patch-log-truncation-bugs-20100514.txt
>
>
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira