Re: Mapreduce program reports child error

[Arun C Murthy <ac...@yahoo-inc.com>]

Moving to mapreduce-dev@, please use the right list for questions.

On May 3, 2011, at 12:06 AM, Sudharsan Sampath wrote:

>
> Hi,
>
> Could anyone point me to a summary on why this error would occur?
>
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:471)
> Caused by: java.io.IOException: Task process exit with nonzero  
> status of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:458)
>
> I know of the following two reasons.
>
> 1) Insufficient space in the logs directory
> 2) ulimit threshold that causes insuffucient allocation of memory.
>

Some more (not a comprehensive list):
a) Task has OOM
b) TT, while monitoring the task, realizes it has used more memory  
than allowed and kills it.
c) Task can't write it's temp outputs to local disk (either map- 
outputs or intermediate merges during map-side sort or reduce-side  
shuffle).
...

Arun