You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "vijayan.B (JIRA)" <ji...@apache.org> on 2009/07/10 12:07:14 UTC
[jira] Created: (HDFS-485) error : too many fetch failures
error : too many fetch failures
-------------------------------
Key: HDFS-485
URL: https://issues.apache.org/jira/browse/HDFS-485
Project: Hadoop HDFS
Issue Type: Bug
Components: data-node
Environment: fedora - 8 , virtual environment using Xen
Reporter: vijayan.B
Fix For: 0.20.1
i have a hadoop cluster configured with 1 physcical machine as name node and 7 data nodes(2 physcical+5 virtual).
When a sort job (hadoop-core-examples) is submitted it completes with the following error:can anyone tell me why and how to solve this issue.
hadoop version:0.18.3
O/P from terminal********************************
[root@hadoop1 hadoop-0.18.3]# bin/hadoop jar hadoop-0.18.3-examples.jar sort input13 output1
Running on 7 nodes to sort from hdfs://hadoop1:8022/user/root/input13 into hdfs://hadoop1:8022/user/root/output1 with 12 reduces.
Job started: Fri Jul 10 14:00:19 IST 2009
09/07/10 14:00:19 INFO mapred.FileInputFormat: Total input paths to process : 1
09/07/10 14:00:19 INFO mapred.FileInputFormat: Total input paths to process : 1
09/07/10 14:00:19 INFO mapred.JobClient: Running job: job_200907101344_0002
09/07/10 14:00:20 INFO mapred.JobClient: map 0% reduce 0%
09/07/10 14:00:24 INFO mapred.JobClient: map 6% reduce 0%
09/07/10 14:00:25 INFO mapred.JobClient: map 12% reduce 0%
09/07/10 14:00:28 INFO mapred.JobClient: map 31% reduce 0%
09/07/10 14:00:29 INFO mapred.JobClient: map 50% reduce 0%
09/07/10 14:00:33 INFO mapred.JobClient: map 66% reduce 0%
09/07/10 14:00:34 INFO mapred.JobClient: map 72% reduce 0%
09/07/10 14:00:35 INFO mapred.JobClient: map 75% reduce 0%
09/07/10 14:00:37 INFO mapred.JobClient: map 75% reduce 1%
09/07/10 14:00:38 INFO mapred.JobClient: map 78% reduce 5%
09/07/10 14:00:39 INFO mapred.JobClient: map 89% reduce 10%
09/07/10 14:00:40 INFO mapred.JobClient: map 89% reduce 11%
09/07/10 14:00:41 INFO mapred.JobClient: map 90% reduce 11%
09/07/10 14:00:42 INFO mapred.JobClient: map 99% reduce 14%
09/07/10 14:00:43 INFO mapred.JobClient: map 99% reduce 16%
09/07/10 14:00:44 INFO mapred.JobClient: map 99% reduce 18%
09/07/10 14:00:45 INFO mapred.JobClient: map 99% reduce 19%
09/07/10 14:00:47 INFO mapred.JobClient: map 99% reduce 22%
09/07/10 14:00:48 INFO mapred.JobClient: map 100% reduce 22%
09/07/10 14:00:50 INFO mapred.JobClient: map 100% reduce 24%
09/07/10 14:00:52 INFO mapred.JobClient: map 100% reduce 25%
09/07/10 14:00:53 INFO mapred.JobClient: map 100% reduce 26%
09/07/10 14:00:54 INFO mapred.JobClient: map 100% reduce 27%
09/07/10 14:00:58 INFO mapred.JobClient: map 100% reduce 33%
09/07/10 14:01:00 INFO mapred.JobClient: map 100% reduce 34%
09/07/10 14:01:03 INFO mapred.JobClient: map 100% reduce 39%
09/07/10 14:03:29 INFO mapred.JobClient: map 100% reduce 40%
09/07/10 14:03:42 INFO mapred.JobClient: map 100% reduce 41%
09/07/10 14:03:50 INFO mapred.JobClient: map 100% reduce 47%
09/07/10 14:03:51 INFO mapred.JobClient: map 100% reduce 50%
09/07/10 14:03:52 INFO mapred.JobClient: map 100% reduce 56%
09/07/10 14:03:57 INFO mapred.JobClient: map 100% reduce 57%
09/07/10 14:04:00 INFO mapred.JobClient: map 100% reduce 58%
09/07/10 14:04:05 INFO mapred.JobClient: map 100% reduce 59%
09/07/10 14:04:07 INFO mapred.JobClient: map 100% reduce 60%
09/07/10 14:04:09 INFO mapred.JobClient: map 100% reduce 66%
09/07/10 14:04:10 INFO mapred.JobClient: map 100% reduce 67%
09/07/10 14:04:12 INFO mapred.JobClient: map 100% reduce 68%
09/07/10 14:04:13 INFO mapred.JobClient: map 100% reduce 69%
09/07/10 14:04:14 INFO mapred.JobClient: map 100% reduce 70%
09/07/10 14:04:20 INFO mapred.JobClient: map 100% reduce 79%
09/07/10 14:04:21 INFO mapred.JobClient: map 100% reduce 80%
09/07/10 14:04:22 INFO mapred.JobClient: map 100% reduce 81%
09/07/10 14:04:23 INFO mapred.JobClient: map 100% reduce 82%
09/07/10 14:04:33 INFO mapred.JobClient: map 100% reduce 87%
09/07/10 14:04:42 INFO mapred.JobClient: Task Id : attempt_200907101344_0002_m_000013_0, Status : FAILED
Too many fetch-failures
09/07/10 14:04:44 INFO mapred.JobClient: map 93% reduce 87%
09/07/10 14:04:50 INFO mapred.JobClient: map 100% reduce 87%
09/07/10 14:05:06 INFO mapred.JobClient: map 100% reduce 93%
09/07/10 14:05:36 INFO mapred.JobClient: map 100% reduce 94%
09/07/10 14:06:17 INFO mapred.JobClient: Job complete: job_200907101344_0002
09/07/10 14:06:17 INFO mapred.JobClient: Counters: 17
09/07/10 14:06:17 INFO mapred.JobClient: File Systems
09/07/10 14:06:17 INFO mapred.JobClient: HDFS bytes read=1077612760
09/07/10 14:06:17 INFO mapred.JobClient: HDFS bytes written=1077285377
09/07/10 14:06:17 INFO mapred.JobClient: Local bytes read=1083539214
09/07/10 14:06:17 INFO mapred.JobClient: Local bytes written=2167083496
09/07/10 14:06:17 INFO mapred.JobClient: Job Counters
09/07/10 14:06:17 INFO mapred.JobClient: Launched reduce tasks=18
09/07/10 14:06:17 INFO mapred.JobClient: Rack-local map tasks=2
09/07/10 14:06:17 INFO mapred.JobClient: Launched map tasks=18
09/07/10 14:06:17 INFO mapred.JobClient: Data-local map tasks=15
09/07/10 14:06:17 INFO mapred.JobClient: Map-Reduce Framework
09/07/10 14:06:17 INFO mapred.JobClient: Reduce input groups=102341
09/07/10 14:06:17 INFO mapred.JobClient: Combine output records=0
09/07/10 14:06:17 INFO mapred.JobClient: Map input records=102341
09/07/10 14:06:17 INFO mapred.JobClient: Reduce output records=102341
09/07/10 14:06:17 INFO mapred.JobClient: Map output bytes=1074564177
09/07/10 14:06:17 INFO mapred.JobClient: Map input bytes=1077284885
09/07/10 14:06:17 INFO mapred.JobClient: Combine input records=0
09/07/10 14:06:17 INFO mapred.JobClient: Map output records=102341
09/07/10 14:06:17 INFO mapred.JobClient: Reduce input records=102341
Job ended: Fri Jul 10 14:06:17 IST 2009
The job took 357 seconds.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HDFS-485) error : too many fetch failures
Posted by "Ravi Phulari (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Phulari resolved HDFS-485.
-------------------------------
Resolution: Invalid
Closing as invalid because error is not due to HDFS.
The fetch is over HTTP, so it has nothing to do with HDFS . and this could be anything from network congestion to a bad disk. The issue is invalid, since the framework- failing to obtain the map output- reexecuted the map and completed the job successfully.
If you have any issues with Map Reduce please write email to mapreduce-user at hadoop.apache.org
.
> error : too many fetch failures
> -------------------------------
>
> Key: HDFS-485
> URL: https://issues.apache.org/jira/browse/HDFS-485
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node
> Environment: fedora - 8 , virtual environment using Xen
> Reporter: vijayan.B
> Fix For: 0.20.1
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> i have a hadoop cluster configured with 1 physcical machine as name node and 7 data nodes(2 physcical+5 virtual).
> When a sort job (hadoop-core-examples) is submitted it completes with the following error:can anyone tell me why and how to solve this issue.
> hadoop version:0.18.3
> O/P from terminal********************************
> [root@hadoop1 hadoop-0.18.3]# bin/hadoop jar hadoop-0.18.3-examples.jar sort input13 output1
> Running on 7 nodes to sort from hdfs://hadoop1:8022/user/root/input13 into hdfs://hadoop1:8022/user/root/output1 with 12 reduces.
> Job started: Fri Jul 10 14:00:19 IST 2009
> 09/07/10 14:00:19 INFO mapred.FileInputFormat: Total input paths to process : 1
> 09/07/10 14:00:19 INFO mapred.FileInputFormat: Total input paths to process : 1
> 09/07/10 14:00:19 INFO mapred.JobClient: Running job: job_200907101344_0002
> 09/07/10 14:00:20 INFO mapred.JobClient: map 0% reduce 0%
> 09/07/10 14:00:24 INFO mapred.JobClient: map 6% reduce 0%
> 09/07/10 14:00:25 INFO mapred.JobClient: map 12% reduce 0%
> 09/07/10 14:00:28 INFO mapred.JobClient: map 31% reduce 0%
> 09/07/10 14:00:29 INFO mapred.JobClient: map 50% reduce 0%
> 09/07/10 14:00:33 INFO mapred.JobClient: map 66% reduce 0%
> 09/07/10 14:00:34 INFO mapred.JobClient: map 72% reduce 0%
> 09/07/10 14:00:35 INFO mapred.JobClient: map 75% reduce 0%
> 09/07/10 14:00:37 INFO mapred.JobClient: map 75% reduce 1%
> 09/07/10 14:00:38 INFO mapred.JobClient: map 78% reduce 5%
> 09/07/10 14:00:39 INFO mapred.JobClient: map 89% reduce 10%
> 09/07/10 14:00:40 INFO mapred.JobClient: map 89% reduce 11%
> 09/07/10 14:00:41 INFO mapred.JobClient: map 90% reduce 11%
> 09/07/10 14:00:42 INFO mapred.JobClient: map 99% reduce 14%
> 09/07/10 14:00:43 INFO mapred.JobClient: map 99% reduce 16%
> 09/07/10 14:00:44 INFO mapred.JobClient: map 99% reduce 18%
> 09/07/10 14:00:45 INFO mapred.JobClient: map 99% reduce 19%
> 09/07/10 14:00:47 INFO mapred.JobClient: map 99% reduce 22%
> 09/07/10 14:00:48 INFO mapred.JobClient: map 100% reduce 22%
> 09/07/10 14:00:50 INFO mapred.JobClient: map 100% reduce 24%
> 09/07/10 14:00:52 INFO mapred.JobClient: map 100% reduce 25%
> 09/07/10 14:00:53 INFO mapred.JobClient: map 100% reduce 26%
> 09/07/10 14:00:54 INFO mapred.JobClient: map 100% reduce 27%
> 09/07/10 14:00:58 INFO mapred.JobClient: map 100% reduce 33%
> 09/07/10 14:01:00 INFO mapred.JobClient: map 100% reduce 34%
> 09/07/10 14:01:03 INFO mapred.JobClient: map 100% reduce 39%
> 09/07/10 14:03:29 INFO mapred.JobClient: map 100% reduce 40%
> 09/07/10 14:03:42 INFO mapred.JobClient: map 100% reduce 41%
> 09/07/10 14:03:50 INFO mapred.JobClient: map 100% reduce 47%
> 09/07/10 14:03:51 INFO mapred.JobClient: map 100% reduce 50%
> 09/07/10 14:03:52 INFO mapred.JobClient: map 100% reduce 56%
> 09/07/10 14:03:57 INFO mapred.JobClient: map 100% reduce 57%
> 09/07/10 14:04:00 INFO mapred.JobClient: map 100% reduce 58%
> 09/07/10 14:04:05 INFO mapred.JobClient: map 100% reduce 59%
> 09/07/10 14:04:07 INFO mapred.JobClient: map 100% reduce 60%
> 09/07/10 14:04:09 INFO mapred.JobClient: map 100% reduce 66%
> 09/07/10 14:04:10 INFO mapred.JobClient: map 100% reduce 67%
> 09/07/10 14:04:12 INFO mapred.JobClient: map 100% reduce 68%
> 09/07/10 14:04:13 INFO mapred.JobClient: map 100% reduce 69%
> 09/07/10 14:04:14 INFO mapred.JobClient: map 100% reduce 70%
> 09/07/10 14:04:20 INFO mapred.JobClient: map 100% reduce 79%
> 09/07/10 14:04:21 INFO mapred.JobClient: map 100% reduce 80%
> 09/07/10 14:04:22 INFO mapred.JobClient: map 100% reduce 81%
> 09/07/10 14:04:23 INFO mapred.JobClient: map 100% reduce 82%
> 09/07/10 14:04:33 INFO mapred.JobClient: map 100% reduce 87%
> 09/07/10 14:04:42 INFO mapred.JobClient: Task Id : attempt_200907101344_0002_m_000013_0, Status : FAILED
> Too many fetch-failures
> 09/07/10 14:04:44 INFO mapred.JobClient: map 93% reduce 87%
> 09/07/10 14:04:50 INFO mapred.JobClient: map 100% reduce 87%
> 09/07/10 14:05:06 INFO mapred.JobClient: map 100% reduce 93%
> 09/07/10 14:05:36 INFO mapred.JobClient: map 100% reduce 94%
> 09/07/10 14:06:17 INFO mapred.JobClient: Job complete: job_200907101344_0002
> 09/07/10 14:06:17 INFO mapred.JobClient: Counters: 17
> 09/07/10 14:06:17 INFO mapred.JobClient: File Systems
> 09/07/10 14:06:17 INFO mapred.JobClient: HDFS bytes read=1077612760
> 09/07/10 14:06:17 INFO mapred.JobClient: HDFS bytes written=1077285377
> 09/07/10 14:06:17 INFO mapred.JobClient: Local bytes read=1083539214
> 09/07/10 14:06:17 INFO mapred.JobClient: Local bytes written=2167083496
> 09/07/10 14:06:17 INFO mapred.JobClient: Job Counters
> 09/07/10 14:06:17 INFO mapred.JobClient: Launched reduce tasks=18
> 09/07/10 14:06:17 INFO mapred.JobClient: Rack-local map tasks=2
> 09/07/10 14:06:17 INFO mapred.JobClient: Launched map tasks=18
> 09/07/10 14:06:17 INFO mapred.JobClient: Data-local map tasks=15
> 09/07/10 14:06:17 INFO mapred.JobClient: Map-Reduce Framework
> 09/07/10 14:06:17 INFO mapred.JobClient: Reduce input groups=102341
> 09/07/10 14:06:17 INFO mapred.JobClient: Combine output records=0
> 09/07/10 14:06:17 INFO mapred.JobClient: Map input records=102341
> 09/07/10 14:06:17 INFO mapred.JobClient: Reduce output records=102341
> 09/07/10 14:06:17 INFO mapred.JobClient: Map output bytes=1074564177
> 09/07/10 14:06:17 INFO mapred.JobClient: Map input bytes=1077284885
> 09/07/10 14:06:17 INFO mapred.JobClient: Combine input records=0
> 09/07/10 14:06:17 INFO mapred.JobClient: Map output records=102341
> 09/07/10 14:06:17 INFO mapred.JobClient: Reduce input records=102341
> Job ended: Fri Jul 10 14:06:17 IST 2009
> The job took 357 seconds.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.