You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Allen Wittenauer (JIRA)" <ji...@apache.org> on 2014/07/22 00:04:39 UTC
[jira] [Resolved] (MAPREDUCE-145) DiskChecker$DiskErrorException
when 'reduce > reduce'
[ https://issues.apache.org/jira/browse/MAPREDUCE-145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Allen Wittenauer resolved MAPREDUCE-145.
----------------------------------------
Resolution: Cannot Reproduce
> DiskChecker$DiskErrorException when 'reduce > reduce'
> -----------------------------------------------------
>
> Key: MAPREDUCE-145
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-145
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Environment: hadoop-0.19.1; CentOS 5
> Reporter: Leitao Guo
>
> We have 9900 maptasks and 60 reducetasks in the job. When all the other 59 reducetasks have finished, the last reducetask runs so slow and finally finished after throwing out a lot of DiskErrorExceptions.
> The following is the tasktracker log on which the reducetask is running.
> 2009-03-18 14:39:52,025 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:39:57,028 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:40:00,695 INFO org.apache.hadoop.mapred.TaskTracker: attempt_200903171026_0091_r_000030_0 0.977961% reduce > reduce
> 2009-03-18 14:40:02,032 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:40:07,036 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:40:12,040 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:40:17,045 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:40:22,050 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:40:27,054 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:40:32,058 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:40:37,062 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:40:42,066 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:40:47,136 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:40:52,140 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:40:57,143 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:41:02,147 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:41:06,760 INFO org.apache.hadoop.mapred.TaskTracker: attempt_200903171026_0091_r_000030_0 0.9788374% reduce > reduce
> 2009-03-18 14:41:07,152 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:41:09,762 INFO org.apache.hadoop.mapred.TaskTracker: attempt_200903171026_0091_r_000030_0 0.9788374% reduce > reduce
> 2009-03-18 14:41:12,158 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:41:17,162 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:41:22,168 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
> 2009-03-18 14:41:27,172 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out in any of the configured local directories
--
This message was sent by Atlassian JIRA
(v6.2#6252)