You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Deepak Diwakar <dd...@gmail.com> on 2009/07/02 20:06:01 UTC
Cannot allocate memory I/O error
Hi,
I use Hadoop-19.0 in standalone mode. I ran an aggregation task over around
2.5 TB of data. This Particular task I run regularly but didn't get error
except this time. And tried thrice to run this task and got same following
error every time:
09/07/01 10:29:34 INFO mapred.MapTask: Starting flush of map output
09/07/01 10:29:34 WARN mapred.LocalJobRunner: job_local_0001
java.io.IOException: Cannot run program "bash": java.io.IOException:
error=12, Cannot allocate memory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
at org.apache.hadoop.util.Shell.run(Shell.java:134)
at org.apache.hadoop.fs.DF.getAvailable(DF.java:73)
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:321)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124)
at
org.apache.hadoop.mapred.MapOutputFile.getSpillFileForWrite(MapOutputFile.java:107)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:920)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:832)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:138)
Caused by: java.io.IOException: java.io.IOException: error=12, Cannot
allocate memory
at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
at java.lang.ProcessImpl.start(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
... 10 more
java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1217)
at
otfa.mapreducetasks.reach.GeoWiseReachH2.run(GeoWiseReachH2.java:223)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at
otfa.mapreducetasks.reach.GeoWiseReachH2.main(GeoWiseReachH2.java:229)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:165)
at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
Can someone explain why it is happening, while my other hadoop-tasks are
running fine and even this particular task was running well in past(I didn't
do any change). Is it java Heap size problem?
Thanks in advance.
--
- Deepak Diwakar,
Re: Cannot allocate memory I/O error
Posted by Deepak Diwakar <dd...@gmail.com>.
I think I got one reason(probably the one): my RAM and Swap-space were
almost fully occupied:
free -m
total used free shared buffers cached
Mem: 4051 4027 23 0 3 1349
-/+ buffers/cache: 2674 1377
Swap: 8189 7752 436
Is there anything else? Or Can it be possible to handle dynamic memory
allocation while hadoop is processing large data?
--
Deepak Diwakar
2009/7/2 Deepak Diwakar <dd...@gmail.com>
> Hi,
>
> I use Hadoop-19.0 in standalone mode. I ran an aggregation task over
> around 2.5 TB of data. This Particular task I run regularly but didn't get
> error except this time. And tried thrice to run this task and got same
> following error every time:
>
> 09/07/01 10:29:34 INFO mapred.MapTask: Starting flush of map output
> 09/07/01 10:29:34 WARN mapred.LocalJobRunner: job_local_0001
> java.io.IOException: Cannot run program "bash": java.io.IOException:
> error=12, Cannot allocate memory
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
> at org.apache.hadoop.util.Shell.run(Shell.java:134)
> at org.apache.hadoop.fs.DF.getAvailable(DF.java:73)
> at
> org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:321)
>
> at
> org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124)
>
> at
> org.apache.hadoop.mapred.MapOutputFile.getSpillFileForWrite(MapOutputFile.java:107)
>
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:920)
>
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:832)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:138)
> Caused by: java.io.IOException: java.io.IOException: error=12, Cannot
> allocate memory
> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
> ... 10 more
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1217)
> at
> otfa.mapreducetasks.reach.GeoWiseReachH2.run(GeoWiseReachH2.java:223)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at
> otfa.mapreducetasks.reach.GeoWiseReachH2.main(GeoWiseReachH2.java:229)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:165)
> at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>
> Can someone explain why it is happening, while my other hadoop-tasks are
> running fine and even this particular task was running well in past(I didn't
> do any change). Is it java Heap size problem?
>
> Thanks in advance.
> --
> - Deepak Diwakar,
>
>
>
--
- Deepak Diwakar,