You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "SRIKANTH KOMMINENI (RIT Student)" <sx...@rit.edu> on 2012/04/11 01:17:25 UTC
java.lang.OutOfMemoryError: Java heap space No matter the size of the
input its given
Can you please help me resolve this issue.
The following is the error that I get no matter how small the input the
program is fed with.
I tried changing the default heap size of hadoop in the conf/hadoop-env.sh
but that also did not help.
The following is the error that i get.
Wrote input for Map #9
Starting Job
12/04/10 19:12:23 WARN mapred.JobClient: No job jar file set. User classes
may not be found. See JobConf(Class) or JobConf#setJar(String).
12/04/10 19:12:23 INFO mapred.FileInputFormat: Total input paths to process
: 10
12/04/10 19:12:23 INFO mapred.JobClient: Running job: job_local_0001
12/04/10 19:12:23 INFO util.ProcessTree: setsid exited with exit code 0
12/04/10 19:12:23 INFO mapred.Task: Using ResourceCalculatorPlugin :
org.apache.hadoop.util.LinuxResourceCalculatorPlugin@ab444
12/04/10 19:12:23 INFO mapred.MapTask: numReduceTasks: 1
12/04/10 19:12:23 INFO mapred.MapTask: io.sort.mb = 100
12/04/10 19:12:23 WARN mapred.LocalJobRunner: job_local_0001
java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:949)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:428)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
12/04/10 19:12:24 INFO mapred.JobClient: map 0% reduce 0%
12/04/10 19:12:24 INFO mapred.JobClient: Job complete: job_local_0001
12/04/10 19:12:24 INFO mapred.JobClient: Counters: 0
12/04/10 19:12:24 INFO mapred.JobClient: Job Failed: NA
java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1265)
at org.apache.hadoop.examples.PiEstimator.estimate(PiEstimator.java:297)
at org.apache.hadoop.examples.PiEstimator.run(PiEstimator.java:342)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.examples.PiEstimator.main(PiEstimator.java:351)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
--
Srikanth Kommineni,
Graduate Assistant,
Dept of Computer Science,
Rochester Institute of Technology.
Re: java.lang.OutOfMemoryError: Java heap space No matter the size of
the input its given
Posted by madhu phatak <ph...@gmail.com>.
Hi ,
Can you see if any of your logic going for an infinite loop in mapper?
On Wed, Apr 11, 2012 at 4:47 AM, SRIKANTH KOMMINENI (RIT Student) <
sxk7699@rit.edu> wrote:
> Can you please help me resolve this issue.
>
> The following is the error that I get no matter how small the input the
> program is fed with.
> I tried changing the default heap size of hadoop in the conf/hadoop-env.sh
> but that also did not help.
>
> The following is the error that i get.
>
>
>
> Wrote input for Map #9
> Starting Job
> 12/04/10 19:12:23 WARN mapred.JobClient: No job jar file set. User classes
> may not be found. See JobConf(Class) or JobConf#setJar(String).
> 12/04/10 19:12:23 INFO mapred.FileInputFormat: Total input paths to process
> : 10
> 12/04/10 19:12:23 INFO mapred.JobClient: Running job: job_local_0001
> 12/04/10 19:12:23 INFO util.ProcessTree: setsid exited with exit code 0
> 12/04/10 19:12:23 INFO mapred.Task: Using ResourceCalculatorPlugin :
> org.apache.hadoop.util.LinuxResourceCalculatorPlugin@ab444
> 12/04/10 19:12:23 INFO mapred.MapTask: numReduceTasks: 1
> 12/04/10 19:12:23 INFO mapred.MapTask: io.sort.mb = 100
> 12/04/10 19:12:23 WARN mapred.LocalJobRunner: job_local_0001
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:949)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:428)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
> 12/04/10 19:12:24 INFO mapred.JobClient: map 0% reduce 0%
> 12/04/10 19:12:24 INFO mapred.JobClient: Job complete: job_local_0001
> 12/04/10 19:12:24 INFO mapred.JobClient: Counters: 0
> 12/04/10 19:12:24 INFO mapred.JobClient: Job Failed: NA
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1265)
> at org.apache.hadoop.examples.PiEstimator.estimate(PiEstimator.java:297)
> at org.apache.hadoop.examples.PiEstimator.run(PiEstimator.java:342)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.examples.PiEstimator.main(PiEstimator.java:351)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
>
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
> --
> Srikanth Kommineni,
> Graduate Assistant,
> Dept of Computer Science,
> Rochester Institute of Technology.
>
--
https://github.com/zinnia-phatak-dev/Nectar