You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by "Erdong (Roger) CHEN" <ro...@gmail.com> on 2007/06/17 21:55:29 UTC
fail to run a modified WordCount example on 20G data [hadoop]

Hi all,
I tried to run word count on a cluster with 30 nodes, and I always get
the same error.  Could someone give me some insights? What type of
errors I may need to investigate? thanks.

07/06/17 11:02:15 INFO mapred.FileInputFormat: Total input paths to process : 98
07/06/17 11:02:19 INFO mapred.JobClient: Running job: job_0001
07/06/17 11:02:20 INFO mapred.JobClient:  map 0% reduce 0%
07/06/17 11:03:25 INFO mapred.JobClient:  map 1% reduce 0%
07/06/17 11:04:15 INFO mapred.JobClient:  map 2% reduce 0%
07/06/17 11:05:19 INFO mapred.JobClient:  map 3% reduce 0%
07/06/17 11:06:30 INFO mapred.JobClient:  map 4% reduce 0%
07/06/17 11:07:40 INFO mapred.JobClient:  map 5% reduce 0%
07/06/17 11:08:40 INFO mapred.JobClient:  map 6% reduce 0%
07/06/17 11:09:48 INFO mapred.JobClient:  map 7% reduce 0%
07/06/17 11:10:51 INFO mapred.JobClient:  map 8% reduce 0%
07/06/17 11:12:01 INFO mapred.JobClient:  map 9% reduce 0%
07/06/17 11:13:11 INFO mapred.JobClient:  map 10% reduce 0%
07/06/17 11:14:18 INFO mapred.JobClient:  map 11% reduce 0%
07/06/17 11:15:30 INFO mapred.JobClient:  map 12% reduce 0%
07/06/17 11:16:39 INFO mapred.JobClient:  map 13% reduce 0%
07/06/17 11:17:48 INFO mapred.JobClient:  map 14% reduce 0%
07/06/17 11:18:53 INFO mapred.JobClient:  map 15% reduce 0%
07/06/17 11:20:01 INFO mapred.JobClient:  map 16% reduce 0%
07/06/17 11:21:04 INFO mapred.JobClient:  map 17% reduce 0%
07/06/17 11:22:12 INFO mapred.JobClient:  map 18% reduce 0%
07/06/17 11:23:13 INFO mapred.JobClient:  map 19% reduce 0%
07/06/17 11:24:18 INFO mapred.JobClient:  map 20% reduce 0%
07/06/17 11:25:21 INFO mapred.JobClient:  map 21% reduce 0%
07/06/17 11:26:25 INFO mapred.JobClient:  map 22% reduce 0%
07/06/17 11:27:25 INFO mapred.JobClient:  map 23% reduce 0%
07/06/17 11:28:33 INFO mapred.JobClient:  map 24% reduce 0%
07/06/17 11:29:35 INFO mapred.JobClient:  map 25% reduce 0%
07/06/17 11:30:36 INFO mapred.JobClient:  map 26% reduce 0%
07/06/17 11:31:34 INFO mapred.JobClient:  map 27% reduce 0%
07/06/17 11:32:33 INFO mapred.JobClient:  map 28% reduce 0%
07/06/17 11:33:37 INFO mapred.JobClient:  map 29% reduce 0%
07/06/17 11:34:37 INFO mapred.JobClient:  map 30% reduce 0%
07/06/17 11:35:36 INFO mapred.JobClient:  map 31% reduce 0%
07/06/17 11:36:35 INFO mapred.JobClient:  map 32% reduce 0%
07/06/17 11:37:14 INFO mapred.JobClient: Task Id :
task_0001_m_000060_0, Status : FAILED
07/06/17 11:37:45 INFO mapred.JobClient:  map 33% reduce 0%
07/06/17 11:38:40 INFO mapred.JobClient:  map 34% reduce 0%
07/06/17 11:39:41 INFO mapred.JobClient:  map 35% reduce 0%
07/06/17 11:40:38 INFO mapred.JobClient:  map 36% reduce 0%
07/06/17 11:41:39 INFO mapred.JobClient:  map 37% reduce 0%
07/06/17 11:42:41 INFO mapred.JobClient:  map 38% reduce 0%
07/06/17 11:43:40 INFO mapred.JobClient:  map 39% reduce 0%
07/06/17 11:44:43 INFO mapred.JobClient:  map 40% reduce 0%
07/06/17 11:45:42 INFO mapred.JobClient:  map 41% reduce 0%
07/06/17 11:46:41 INFO mapred.JobClient:  map 42% reduce 0%
07/06/17 11:47:41 INFO mapred.JobClient:  map 43% reduce 0%
07/06/17 11:48:38 INFO mapred.JobClient:  map 44% reduce 0%
07/06/17 11:49:37 INFO mapred.JobClient:  map 45% reduce 0%
07/06/17 11:50:33 INFO mapred.JobClient:  map 46% reduce 0%
07/06/17 11:51:32 INFO mapred.JobClient:  map 47% reduce 0%
07/06/17 11:52:36 INFO mapred.JobClient:  map 48% reduce 0%
07/06/17 11:53:32 INFO mapred.JobClient:  map 49% reduce 0%
07/06/17 11:54:31 INFO mapred.JobClient:  map 50% reduce 0%
07/06/17 11:55:32 INFO mapred.JobClient:  map 51% reduce 0%
07/06/17 11:56:27 INFO mapred.JobClient:  map 52% reduce 0%
07/06/17 11:57:25 INFO mapred.JobClient:  map 53% reduce 0%
07/06/17 11:58:22 INFO mapred.JobClient:  map 54% reduce 0%
07/06/17 11:59:20 INFO mapred.JobClient:  map 55% reduce 0%
07/06/17 12:00:17 INFO mapred.JobClient:  map 56% reduce 0%
07/06/17 12:01:16 INFO mapred.JobClient:  map 57% reduce 0%
07/06/17 12:02:15 INFO mapred.JobClient:  map 58% reduce 0%
07/06/17 12:03:11 INFO mapred.JobClient:  map 59% reduce 0%
07/06/17 12:04:11 INFO mapred.JobClient:  map 60% reduce 0%
07/06/17 12:05:10 INFO mapred.JobClient:  map 61% reduce 0%
07/06/17 12:06:11 INFO mapred.JobClient:  map 62% reduce 0%
07/06/17 12:06:39 INFO mapred.JobClient:  map 62% reduce 1%
07/06/17 12:07:09 INFO mapred.JobClient:  map 63% reduce 1%
07/06/17 12:08:09 INFO mapred.JobClient:  map 64% reduce 1%
07/06/17 12:09:07 INFO mapred.JobClient:  map 65% reduce 1%
07/06/17 12:10:06 INFO mapred.JobClient:  map 66% reduce 1%
07/06/17 12:11:06 INFO mapred.JobClient:  map 67% reduce 1%
07/06/17 12:12:00 INFO mapred.JobClient:  map 68% reduce 1%
07/06/17 12:12:57 INFO mapred.JobClient:  map 69% reduce 1%
07/06/17 12:13:56 INFO mapred.JobClient:  map 70% reduce 1%
07/06/17 12:14:50 INFO mapred.JobClient:  map 71% reduce 1%
07/06/17 12:15:48 INFO mapred.JobClient:  map 72% reduce 1%
07/06/17 12:16:37 INFO mapred.JobClient:  map 73% reduce 1%
07/06/17 12:17:34 INFO mapred.JobClient:  map 74% reduce 1%
07/06/17 12:18:25 INFO mapred.JobClient:  map 75% reduce 1%
07/06/17 12:19:05 INFO mapred.JobClient:  map 76% reduce 1%
07/06/17 12:19:44 INFO mapred.JobClient:  map 77% reduce 1%
07/06/17 12:20:27 INFO mapred.JobClient:  map 78% reduce 1%
07/06/17 12:21:03 INFO mapred.JobClient:  map 79% reduce 1%
07/06/17 12:21:43 INFO mapred.JobClient:  map 80% reduce 1%
07/06/17 12:22:10 INFO mapred.JobClient:  map 81% reduce 1%
07/06/17 12:22:40 INFO mapred.JobClient:  map 82% reduce 1%
07/06/17 12:23:07 INFO mapred.JobClient:  map 83% reduce 1%
07/06/17 12:23:34 INFO mapred.JobClient:  map 84% reduce 1%
07/06/17 12:24:05 INFO mapred.JobClient:  map 85% reduce 1%
07/06/17 12:24:34 INFO mapred.JobClient:  map 86% reduce 1%
07/06/17 12:24:56 INFO mapred.JobClient:  map 87% reduce 1%
07/06/17 12:25:17 INFO mapred.JobClient:  map 88% reduce 1%
07/06/17 12:25:36 INFO mapred.JobClient:  map 89% reduce 1%
07/06/17 12:25:56 INFO mapred.JobClient:  map 90% reduce 1%
07/06/17 12:26:16 INFO mapred.JobClient:  map 91% reduce 1%
07/06/17 12:26:34 INFO mapred.JobClient:  map 92% reduce 1%
07/06/17 12:26:49 INFO mapred.JobClient:  map 93% reduce 1%
07/06/17 12:27:06 INFO mapred.JobClient:  map 94% reduce 1%
07/06/17 12:27:17 INFO mapred.JobClient:  map 95% reduce 1%
07/06/17 12:27:31 INFO mapred.JobClient:  map 96% reduce 1%
07/06/17 12:27:47 INFO mapred.JobClient:  map 97% reduce 1%
07/06/17 12:28:16 INFO mapred.JobClient:  map 98% reduce 1%
07/06/17 12:29:27 INFO mapred.JobClient:  map 99% reduce 1%
07/06/17 12:33:41 INFO mapred.JobClient: Task Id :
task_0001_m_000060_1, Status : FAILED
07/06/17 12:42:46 INFO mapred.JobClient: Task Id :
task_0001_m_000060_2, Status : FAILED
07/06/17 12:48:54 INFO mapred.JobClient:  map 99% reduce 100%
Exception in thread "main" java.io.IOException: Job failed!
       at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:604)
       at org.apache.hadoop.examples.WordCount3.main(WordCount3.java:170)


Roger