hadoop task child failed

[claytonly <cl...@163.com>]

Hello all,

I got a problem when I run wordcount from hadoop example. Given I run hadoop in xen-based VMs, I want to increase vm's cpu cores. But when I inrease vm's vcpu cores. A reduce task is failed.
The log detail show following:
13/08/01 16:19:27 INFO mapred.JobClient: Task Id : attempt_201308011614_0001_r_000001_0, Status : FAILED
Error: java.lang.IllegalMonitorStateException
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run(ReduceTask.java:1317)
13/08/01 16:19:29 INFO mapred.JobClient:  map 56% reduce 6%
13/08/01 16:19:32 INFO mapred.JobClient:  map 56% reduce 4%
13/08/01 16:19:34 INFO mapred.JobClient:  map 68% reduce 4%
13/08/01 16:19:38 INFO mapred.JobClient:  map 75% reduce 4%
13/08/01 16:19:40 INFO mapred.JobClient: Task Id : attempt_201308011614_0001_r_000000_0, Status : FAILED
java.lang.Throwable: Child Error
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Task process exit with nonzero status of 134.
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)

attempt_201308011614_0001_r_000000_0: #
attempt_201308011614_0001_r_000000_0: # A fatal error has been detected by the Java Runtime Environment:
attempt_201308011614_0001_r_000000_0: #
attempt_201308011614_0001_r_000000_0: #  Internal Error (objectMonitor.cpp:1187), pid=2083, tid=3019979632
attempt_201308011614_0001_r_000000_0: #  guarantee(p->TState == ObjectWaiter::TS_CXQ) failed: Invariant
attempt_201308011614_0001_r_000000_0: #
attempt_201308011614_0001_r_000000_0: # JRE version: 6.0_30-b12
attempt_201308011614_0001_r_000000_0: # Java VM: Java HotSpot(TM) Client VM (20.5-b03 mixed mode, sharing linux-x86 )
attempt_201308011614_0001_r_000000_0: # An error report file with more information is saved as:
attempt_201308011614_0001_r_000000_0: # /home/xcp/hadoop-1.0.0/tmp/mapred/local/taskTracker/xcp/jobcache/job_201308011614_0001/q/work/hs_err_pid2083.log
attempt_201308011614_0001_r_000000_0: #
attempt_201308011614_0001_r_000000_0: # If you would like to submit a bug report, please visit:
attempt_201308011614_0001_r_000000_0: #   http://java.sun.com/webapps/bugreport/crash.jsp
attempt_201308011614_0001_r_000000_0: #

Why is that? Many thanks!

clayton

2013-08-01



claytonly