Mapper Out of Memory

[Rui Shi <sh...@yahoo.com>]

Hi,

I run hadoop on a BSD4 clusters and each map task is a gzip file (about 10MB). Some tasks finished. But many of them failed due to heap out of memory. I got the following syslogs:

2007-12-06 12:16:50,277 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=MAP, sessionId=
2007-12-06 12:16:53,128 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 256
2007-12-06 12:16:53,638 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2007-12-06 12:18:19,079 WARN org.apache.hadoop.mapred.TaskTracker: Error running child
java.lang.OutOfMemoryError: Java heap space
Does anyone know what is the reason and how should we avoid it?

Thanks,

Rui





      ____________________________________________________________________________________
Never miss a thing.  Make Yahoo your home page. 
http://www.yahoo.com/r/hs

Re: Mapper Out of Memory

["Peter W." <pe...@marketingbrokers.com>]

Hello,

There is a setting in hadoop-0.15.0/bin/rcc

default:

JAVA_HEAP_MAX=-Xmx1000m

For 2GB memory you can set this about:

JAVA_HEAP_MAX=-Xmx1700m

2048m is the highest allowed setting on a mac, linux,
non-solaris unix or windows box.

Peter W.

On Dec 6, 2007, at 12:30 PM, Rui Shi wrote:

>
> Hi,
>
> I run hadoop on a BSD4 clusters and each map task is a gzip file  
> (about 10MB). Some tasks finished. But many of them failed due to  
> heap out of memory. I got the following syslogs:
>
> 2007-12-06 12:16:50,277 INFO  
> org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics  
> with processName=MAP, sessionId=
> 2007-12-06 12:16:53,128 INFO org.apache.hadoop.mapred.MapTask:  
> numReduceTasks: 256
> 2007-12-06 12:16:53,638 WARN  
> org.apache.hadoop.util.NativeCodeLoader: Unable to load native- 
> hadoop library for your platform... using builtin-java classes  
> where applicable
> 2007-12-06 12:18:19,079 WARN org.apache.hadoop.mapred.TaskTracker:  
> Error running child
> java.lang.OutOfMemoryError: Java heap space
> Does anyone know what is the reason and how should we avoid it?
>
> Thanks,
>
> Rui

Re: Mapper Out of Memory

[Owen O'Malley <oo...@yahoo-inc.com>]

On Dec 6, 2007, at 12:30 PM, Rui Shi wrote:

> Does anyone know what is the reason and how should we avoid it?

Java 6 gives a little better information in the form of a stack  
trace. My patch HADOOP-2367 will also help after it is finished and  
committed.  It will allow you to get cpu and heap summaries from  
representative tasks.

-- Owen

RE: Mapper Out of Memory

[Joydeep Sen Sarma <js...@facebook.com>]

Can control heap size using 'mapred.child.java.opts' option.

Check ur program logic though. Personal experience is that running out
of heap space in map task usually suggests some runaway logic somewhere.

-----Original Message-----
From: Rui Shi [mailto:shearershot@yahoo.com] 
Sent: Thursday, December 06, 2007 12:31 PM
To: hadoop-user@lucene.apache.org
Subject: Mapper Out of Memory


Hi,

I run hadoop on a BSD4 clusters and each map task is a gzip file (about
10MB). Some tasks finished. But many of them failed due to heap out of
memory. I got the following syslogs:

2007-12-06 12:16:50,277 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing JVM Metrics with processName=MAP, sessionId=
2007-12-06 12:16:53,128 INFO org.apache.hadoop.mapred.MapTask:
numReduceTasks: 256
2007-12-06 12:16:53,638 WARN org.apache.hadoop.util.NativeCodeLoader:
Unable to load native-hadoop library for your platform... using
builtin-java classes where applicable
2007-12-06 12:18:19,079 WARN org.apache.hadoop.mapred.TaskTracker: Error
running child
java.lang.OutOfMemoryError: Java heap space
Does anyone know what is the reason and how should we avoid it?

Thanks,

Rui





 
________________________________________________________________________
____________
Never miss a thing.  Make Yahoo your home page. 
http://www.yahoo.com/r/hs