Getting memory error when starting spark shell but not often

[Divya Gehlot <di...@gmail.com>]

Hi,
I am using EMR 4.7 with Spark 1.6
Sometimes when I start the spark shell I get below error

OpenJDK 64-Bit Server VM warning: INFO:
> os::commit_memory(0x00000005662c0000, 10632822784, 0) failed; error='Cannot
> allocate memory' (errno=12)
> #
> # There is insufficient memory for the Java Runtime Environment to
> continue.
> # Native memory allocation (malloc) failed to allocate 10632822784 bytes
> for committing reserved memory.
> # An error report file with more information is saved as:
> # /tmp/jvm-6066/hs_error.log



Has any body encountered this kind of issue .
Would really appreciate the resolution.


Thanks,
Divya

Re: Getting memory error when starting spark shell but not often

[Terry Hoo <hu...@gmail.com>]

Maybe not enough continues memory (10G?) in your host

Regards,
- Terry

On Wed, Sep 7, 2016 at 10:51 AM, Divya Gehlot <di...@gmail.com>
wrote:

> Hi,
> I am using EMR 4.7 with Spark 1.6
> Sometimes when I start the spark shell I get below error
>
> OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000005662c0000,
>> 10632822784, 0) failed; error='Cannot allocate memory' (errno=12)
>> #
>> # There is insufficient memory for the Java Runtime Environment to
>> continue.
>> # Native memory allocation (malloc) failed to allocate 10632822784 bytes
>> for committing reserved memory.
>> # An error report file with more information is saved as:
>> # /tmp/jvm-6066/hs_error.log
>
>
>
> Has any body encountered this kind of issue .
> Would really appreciate the resolution.
>
>
> Thanks,
> Divya
>