You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Alok Tanna <at...@exertdigital.com> on 2015/04/22 23:30:09 UTC
Memory error
I was trying to load 500 thousand row csv file in Drill and I am getting
below error . Now I am not even able to start the Drill . Any suggestions
.Attached is the error log.
[root@ apache-drill-0.8.0]# bin/sqlline -u jdbc:drill:zk=local
Drill log directory /var/log/drill does not exist or is not writable,
defaulting to /opt/drill/apache-drill-0.8.0/log
OpenJDK 64-Bit Server VM warning: INFO:
os::commit_memory(0x00000006e0000000, 357892096, 0) failed; error='Cannot
allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 357892096 bytes for
committing reserved memory.
# An error report file with more information is saved as:
# /tmp/jvm-31515/hs_error.log
Thanks,
Alok
Re: Memory error
Posted by Alok Tanna <at...@exertdigital.com>.
Thank you Ted next time I will make sure I would select a larger machine. I have tried below suggested changes and I was able to atleast start the Drill for now.
Regards,
Alok Tanna
Sent from my iPhone
> On Apr 22, 2015, at 6:12 PM, Ted Dunning <te...@gmail.com> wrote:
>
> You have allocated 4GB to Java's heap and the rest of the 4GB RAM (i.e.
> zero) you have allocated to data storage.
>
> Try
>
> 1) running on a larger machine. Having >8G memory will make these worries
> go away.
>
> 2) decreasing memory requirements. Here is one possibility that may or may
> not work out well:
>
> DRILL_MAX_DIRECT_MEMORY="2G"
> DRILL_MAX_HEAP="1G"
>
> export DRILL_JAVA_OPTS="-Xms100M -Xmx$DRILL_MAX_HEAP
> -XX:MaxDirectMemorySize=$DRILL_MAX_DIRECT_MEMORY -XX:MaxPermSize= -X200M
> X:ReservedCodeCacheSize=500M -ea"
>
> Note that decreasing these sizes puts you into untested territory.
>
>
>
>> On Wed, Apr 22, 2015 at 5:57 PM, Alok Tanna <at...@exertdigital.com> wrote:
>>
>> OS RAM is 4 GB and below is the drill memory configuration from
>> drill-env.sh
>>
>> DRILL_MAX_DIRECT_MEMORY="8G"
>> DRILL_MAX_HEAP="4G"
>>
>> export DRILL_JAVA_OPTS="-Xms1G -Xmx$DRILL_MAX_HEAP
>> -XX:MaxDirectMemorySize=$DRILL_MAX_DIRECT_MEMORY -XX:MaxPermSize= -X512M
>> X:ReservedCodeCacheSize=1G -ea"
>>
>> # Class unloading is disabled by default in Java 7
>> #
>>
>> http://hg.openjdk.java.net/jdk7u/jdk7u60/hotspot/file/tip/src/share/vm/runtime/globals.hpp#l1622
>> export SERVER_GC_OPTS="-XX:+CMSClassUnloadingEnabled
>> -XX:+UseConcMarkSweepGC"
>>
>> Thanks,
>> Alok
>>
>>> On Wed, Apr 22, 2015 at 5:53 PM, Hao Zhu <hz...@maprtech.com> wrote:
>>>
>>> OS runs OOM.
>>> What is the OS RAMs and Drill memory configurations?
>>>
>>> Thanks,
>>> Hao
>>>
>>> On Wed, Apr 22, 2015 at 2:30 PM, Alok Tanna <at...@exertdigital.com>
>>> wrote:
>>>
>>>> I was trying to load 500 thousand row csv file in Drill and I am
>>> getting
>>>> below error . Now I am not even able to start the Drill . Any
>> suggestions
>>>> .Attached is the error log.
>>>>
>>>> [root@ apache-drill-0.8.0]# bin/sqlline -u jdbc:drill:zk=local
>>>> Drill log directory /var/log/drill does not exist or is not writable,
>>>> defaulting to /opt/drill/apache-drill-0.8.0/log
>>>> OpenJDK 64-Bit Server VM warning: INFO:
>>>> os::commit_memory(0x00000006e0000000, 357892096, 0) failed;
>> error='Cannot
>>>> allocate memory' (errno=12)
>>>> #
>>>> # There is insufficient memory for the Java Runtime Environment to
>>>> continue.
>>>> # Native memory allocation (malloc) failed to allocate 357892096 bytes
>>> for
>>>> committing reserved memory.
>>>> # An error report file with more information is saved as:
>>>> # /tmp/jvm-31515/hs_error.log
>>>>
>>>> Thanks,
>>>> Alok
>>
>>
>>
>> --
>> ----------------------------------------------------
>> *Alok Tanna | **Business Analyst*
>>
>>
>> c 215-410-8236 | e atanna@exertdigital.com | w eXert Digital
>> <http://www.exertdigital.com/>
>> https://www.facebook.com/exertdigital
>> https://www.linkedin.com/company/exert-digital
>>
Re: Memory error
Posted by Ted Dunning <te...@gmail.com>.
You have allocated 4GB to Java's heap and the rest of the 4GB RAM (i.e.
zero) you have allocated to data storage.
Try
1) running on a larger machine. Having >8G memory will make these worries
go away.
2) decreasing memory requirements. Here is one possibility that may or may
not work out well:
DRILL_MAX_DIRECT_MEMORY="2G"
DRILL_MAX_HEAP="1G"
export DRILL_JAVA_OPTS="-Xms100M -Xmx$DRILL_MAX_HEAP
-XX:MaxDirectMemorySize=$DRILL_MAX_DIRECT_MEMORY -XX:MaxPermSize= -X200M
X:ReservedCodeCacheSize=500M -ea"
Note that decreasing these sizes puts you into untested territory.
On Wed, Apr 22, 2015 at 5:57 PM, Alok Tanna <at...@exertdigital.com> wrote:
> OS RAM is 4 GB and below is the drill memory configuration from
> drill-env.sh
>
> DRILL_MAX_DIRECT_MEMORY="8G"
> DRILL_MAX_HEAP="4G"
>
> export DRILL_JAVA_OPTS="-Xms1G -Xmx$DRILL_MAX_HEAP
> -XX:MaxDirectMemorySize=$DRILL_MAX_DIRECT_MEMORY -XX:MaxPermSize= -X512M
> X:ReservedCodeCacheSize=1G -ea"
>
> # Class unloading is disabled by default in Java 7
> #
>
> http://hg.openjdk.java.net/jdk7u/jdk7u60/hotspot/file/tip/src/share/vm/runtime/globals.hpp#l1622
> export SERVER_GC_OPTS="-XX:+CMSClassUnloadingEnabled
> -XX:+UseConcMarkSweepGC"
>
> Thanks,
> Alok
>
> On Wed, Apr 22, 2015 at 5:53 PM, Hao Zhu <hz...@maprtech.com> wrote:
>
> > OS runs OOM.
> > What is the OS RAMs and Drill memory configurations?
> >
> > Thanks,
> > Hao
> >
> > On Wed, Apr 22, 2015 at 2:30 PM, Alok Tanna <at...@exertdigital.com>
> > wrote:
> >
> > > I was trying to load 500 thousand row csv file in Drill and I am
> > getting
> > > below error . Now I am not even able to start the Drill . Any
> suggestions
> > > .Attached is the error log.
> > >
> > > [root@ apache-drill-0.8.0]# bin/sqlline -u jdbc:drill:zk=local
> > > Drill log directory /var/log/drill does not exist or is not writable,
> > > defaulting to /opt/drill/apache-drill-0.8.0/log
> > > OpenJDK 64-Bit Server VM warning: INFO:
> > > os::commit_memory(0x00000006e0000000, 357892096, 0) failed;
> error='Cannot
> > > allocate memory' (errno=12)
> > > #
> > > # There is insufficient memory for the Java Runtime Environment to
> > > continue.
> > > # Native memory allocation (malloc) failed to allocate 357892096 bytes
> > for
> > > committing reserved memory.
> > > # An error report file with more information is saved as:
> > > # /tmp/jvm-31515/hs_error.log
> > >
> > > Thanks,
> > > Alok
> > >
> >
>
>
>
> --
> ----------------------------------------------------
> *Alok Tanna | **Business Analyst*
>
>
> c 215-410-8236 | e atanna@exertdigital.com | w eXert Digital
> <http://www.exertdigital.com/>
> https://www.facebook.com/exertdigital
> https://www.linkedin.com/company/exert-digital
>
Re: Memory error
Posted by Alok Tanna <at...@exertdigital.com>.
OS RAM is 4 GB and below is the drill memory configuration from drill-env.sh
DRILL_MAX_DIRECT_MEMORY="8G"
DRILL_MAX_HEAP="4G"
export DRILL_JAVA_OPTS="-Xms1G -Xmx$DRILL_MAX_HEAP
-XX:MaxDirectMemorySize=$DRILL_MAX_DIRECT_MEMORY -XX:MaxPermSize= -X512M
X:ReservedCodeCacheSize=1G -ea"
# Class unloading is disabled by default in Java 7
#
http://hg.openjdk.java.net/jdk7u/jdk7u60/hotspot/file/tip/src/share/vm/runtime/globals.hpp#l1622
export SERVER_GC_OPTS="-XX:+CMSClassUnloadingEnabled
-XX:+UseConcMarkSweepGC"
Thanks,
Alok
On Wed, Apr 22, 2015 at 5:53 PM, Hao Zhu <hz...@maprtech.com> wrote:
> OS runs OOM.
> What is the OS RAMs and Drill memory configurations?
>
> Thanks,
> Hao
>
> On Wed, Apr 22, 2015 at 2:30 PM, Alok Tanna <at...@exertdigital.com>
> wrote:
>
> > I was trying to load 500 thousand row csv file in Drill and I am
> getting
> > below error . Now I am not even able to start the Drill . Any suggestions
> > .Attached is the error log.
> >
> > [root@ apache-drill-0.8.0]# bin/sqlline -u jdbc:drill:zk=local
> > Drill log directory /var/log/drill does not exist or is not writable,
> > defaulting to /opt/drill/apache-drill-0.8.0/log
> > OpenJDK 64-Bit Server VM warning: INFO:
> > os::commit_memory(0x00000006e0000000, 357892096, 0) failed; error='Cannot
> > allocate memory' (errno=12)
> > #
> > # There is insufficient memory for the Java Runtime Environment to
> > continue.
> > # Native memory allocation (malloc) failed to allocate 357892096 bytes
> for
> > committing reserved memory.
> > # An error report file with more information is saved as:
> > # /tmp/jvm-31515/hs_error.log
> >
> > Thanks,
> > Alok
> >
>
--
----------------------------------------------------
*Alok Tanna | **Business Analyst*
c 215-410-8236 | e atanna@exertdigital.com | w eXert Digital
<http://www.exertdigital.com/>
https://www.facebook.com/exertdigital
https://www.linkedin.com/company/exert-digital
Re: Memory error
Posted by Hao Zhu <hz...@maprtech.com>.
OS runs OOM.
What is the OS RAMs and Drill memory configurations?
Thanks,
Hao
On Wed, Apr 22, 2015 at 2:30 PM, Alok Tanna <at...@exertdigital.com> wrote:
> I was trying to load 500 thousand row csv file in Drill and I am getting
> below error . Now I am not even able to start the Drill . Any suggestions
> .Attached is the error log.
>
> [root@ apache-drill-0.8.0]# bin/sqlline -u jdbc:drill:zk=local
> Drill log directory /var/log/drill does not exist or is not writable,
> defaulting to /opt/drill/apache-drill-0.8.0/log
> OpenJDK 64-Bit Server VM warning: INFO:
> os::commit_memory(0x00000006e0000000, 357892096, 0) failed; error='Cannot
> allocate memory' (errno=12)
> #
> # There is insufficient memory for the Java Runtime Environment to
> continue.
> # Native memory allocation (malloc) failed to allocate 357892096 bytes for
> committing reserved memory.
> # An error report file with more information is saved as:
> # /tmp/jvm-31515/hs_error.log
>
> Thanks,
> Alok
>