You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by George Pang <p0...@gmail.com> on 2009/08/14 00:47:04 UTC

About hanging at 16% for reducer

Dear all,

A similar thread about reducer hanging at 16% suggested to open ports
50010,50020, ...and so on to solve the problem.  I disable my firewall
"Firstarter" and add rules on the mentioned ports at iptables.   It doesn't
work, still hangs at 16%.  The error message is the same.

After a long time it will finish the job, but there is a hang at the reduce
stage.

Obviously something wrong with the communication between my two nodes.
Because when one is absent everything goes fine.

Please help with any suggestion.
George

Console message:

09/08/13 13:55:41 INFO mapred.FileInputFormat: Total input paths to process
: 1
09/08/13 13:55:41 INFO mapred.FileInputFormat: Total input paths to process
: 1
09/08/13 13:55:42 INFO mapred.JobClient: Running job: job_200908131354_0001
09/08/13 13:55:43 INFO mapred.JobClient:  map 0% reduce 0%
09/08/13 13:55:47 INFO mapred.JobClient:  map 100% reduce 0%
*09/08/13 13:56:01 INFO mapred.JobClient:  map 100% reduce 16%*
*09/08/13 14:08:51 INFO mapred.JobClient: Task Id :
attempt_200908131354_0001_m_000001_0, Status : FAILED
Too many fetch-failures
09/08/13 14:08:51 WARN mapred.JobClient: Error reading task
outputwpang2-laptop
09/08/13 14:08:51 WARN mapred.JobClient: Error reading task
outputwpang2-laptop*
09/08/13 14:09:04 INFO mapred.JobClient: Job complete: job_200908131354_0001
09/08/13 14:09:04 INFO mapred.JobClient: Counters: 16
09/08/13 14:09:04 INFO mapred.JobClient:   File Systems
09/08/13 14:09:04 INFO mapred.JobClient:     HDFS bytes read=489388
09/08/13 14:09:04 INFO mapred.JobClient:     HDFS bytes written=160672
09/08/13 14:09:04 INFO mapred.JobClient:     Local bytes read=220865
09/08/13 14:09:04 INFO mapred.JobClient:     Local bytes written=491194
09/08/13 14:09:04 INFO mapred.JobClient:   Job Counters
09/08/13 14:09:04 INFO mapred.JobClient:     Launched reduce tasks=1
09/08/13 14:09:04 INFO mapred.JobClient:     Launched map tasks=3
09/08/13 14:09:04 INFO mapred.JobClient:     Data-local map tasks=3
09/08/13 14:09:04 INFO mapred.JobClient:   Map-Reduce Framework
09/08/13 14:09:04 INFO mapred.JobClient:     Reduce input groups=14855
09/08/13 14:09:04 INFO mapred.JobClient:     Combine output records=33283
09/08/13 14:09:04 INFO mapred.JobClient:     Map input records=9484
09/08/13 14:09:04 INFO mapred.JobClient:     Reduce output records=14855
09/08/13 14:09:04 INFO mapred.JobClient:     Map output bytes=793632
09/08/13 14:09:04 INFO mapred.JobClient:     Map input bytes=487253
09/08/13 14:09:04 INFO mapred.JobClient:     Combine input records=98456
09/08/13 14:09:04 INFO mapred.JobClient:     Map output records=80028
09/08/13 14:09:04 INFO mapred.JobClient:     Reduce input records=14855