You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by jjimeno <jj...@omp.com> on 2021/02/12 15:22:13 UTC
[2.9.1] Out of memory in data region
Hello,
I've got this simple configuration file:
<bean class="org.apache.ignite.configuration.IgniteConfiguration" id="VDS">
<property name="dataStorageConfiguration">
<bean class="org.apache.ignite.configuration.DataStorageConfiguration">
<property name="defaultDataRegionConfiguration">
<bean class="org.apache.ignite.configuration.DataRegionConfiguration">
<property name="initialSize" value="#{1L * 1024 * 1024 * 1024}"/>
<property name="maxSize" value="#{1L * 1024 * 1024 * 1024}"/>
<property name="persistenceEnabled" value="true"/>
</bean>
</property>
<property name="pageSize" value="#{4 * 1024}"/>
</bean>
</property>
<property name="cacheConfiguration">
<list>
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="vds"/>
<property name="cacheMode" value="PARTITIONED"/>
<property name="atomicityMode" value="TRANSACTIONAL"/>
<property name="backups" value="1"/>
<property name="writeSynchronizationMode" value="PRIMARY_SYNC"/>
<property name="onheapCacheEnabled" value="false"/>
</bean>
</list>
</property>
</bean>
and, at some point during a transaction of ~1GB, I get this error:
[15:01:23,559][SEVERE][client-connector-#134][] JVM will be halted
immediately due to the failure: [failureCtx=FailureContext
[type=CRITICAL_ERROR, err=class o.a.i.i.mem.IgniteOutOfMemoryException: Out
of memory in data region [name=default, initSize=1.0 GiB, maxSize=1.0 GiB,
*persistenceEnabled=true*] Try the following:
^-- Increase maximum off-heap memory size
(DataRegionConfiguration.maxSize)
^-- *Enable Ignite persistence*
(DataRegionConfiguration.persistenceEnabled)
^-- Enable eviction or expiration policies]]
Maybe the off-heap memory must be, at least, as large as the transaction we
want to commit?
Thanks in advance!
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: [2.9.1] Out of memory in data region
Posted by jjimeno <jj...@omp.com>.
Hello,
That's what I was thinking of... thanks Ilya
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: [2.9.1] Out of memory in data region
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I think it's quite possible to run out of off-heap if your transactions are
comparable with your data region in time. Ignite needs to do page eviction
to be able to reuse off-heap pages, but I guess that pages relevant to
currently running transactions can't be evicted.
Regards,
--
Ilya Kasnacheev
пн, 15 февр. 2021 г. в 09:49, jjimeno <jj...@omp.com>:
> Hello!
>
> Ok but, according to the error:
>
> [15:01:23,559][SEVERE][client-connector-#134][] JVM will be halted
> immediately due to the failure: [failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=class o.a.i.i.mem.IgniteOutOfMemoryException:
> *Out
> of memory in data region* [name=default, initSize=1.0 GiB, maxSize=1.0 GiB,
> *persistenceEnabled=true*]
>
> the OOM is occurring in data region, and that's off-heap memory, as you
> pointed out.
>
> On the other hand, on-heap memory in the test is 4GB, more than enough for
> a
> 1GB transaction...
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: [2.9.1] Out of memory in data region
Posted by jjimeno <jj...@omp.com>.
Hello!
Ok but, according to the error:
[15:01:23,559][SEVERE][client-connector-#134][] JVM will be halted
immediately due to the failure: [failureCtx=FailureContext
[type=CRITICAL_ERROR, err=class o.a.i.i.mem.IgniteOutOfMemoryException: *Out
of memory in data region* [name=default, initSize=1.0 GiB, maxSize=1.0 GiB,
*persistenceEnabled=true*]
the OOM is occurring in data region, and that's off-heap memory, as you
pointed out.
On the other hand, on-heap memory in the test is 4GB, more than enough for a
1GB transaction...
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: [2.9.1] Out of memory in data region
Posted by Ilya Kazakov <ka...@gmail.com>.
Hello!
Ignite uses off-heap memory, whether the persistence is on or not (it
depends on another parameter [1]), but only for a data storing.
Transaction processing is performing on heap, therefore an OOM may occur
here.
[1].
https://ignite.apache.org/docs/latest/configuring-caches/on-heap-caching
------------------
Ilya
вс, 14 февр. 2021 г. в 22:32, jjimeno <jj...@omp.com>:
> Hello
>
> Thanks for your help.
>
> I know that fixes the problem, but my question was about why I’m getting
> that error when persistence is on.
>
> As far as I know, if persistence is on, off-heap memory holds a subset of
> data when they don’t fit in memory, and no OOM error should be thrown...
> unless there are some requirements as the one I pointed out in my previous
> mail I’m not aware of.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: [2.9.1] Out of memory in data region
Posted by jjimeno <jj...@omp.com>.
Hello
Thanks for your help.
I know that fixes the problem, but my question was about why I’m getting
that error when persistence is on.
As far as I know, if persistence is on, off-heap memory holds a subset of
data when they don’t fit in memory, and no OOM error should be thrown...
unless there are some requirements as the one I pointed out in my previous
mail I’m not aware of.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: [2.9.1] Out of memory in data region
Posted by Charlin S <Ch...@hotelhub.com>.
Hello,
Please increase maxsize value based on your data volume and try.
example: <property name="maxSize" value="#{*5L* * 1024 * 1024 * 1024}"/> --
5GB is not an approximation.
Thanks & Regards,
Charlin
On Fri, 12 Feb 2021 at 20:52, jjimeno <jj...@omp.com> wrote:
> Hello,
>
> I've got this simple configuration file:
>
> <bean class="org.apache.ignite.configuration.IgniteConfiguration" id="VDS">
> <property name="dataStorageConfiguration">
> <bean
> class="org.apache.ignite.configuration.DataStorageConfiguration">
>
> <property name="defaultDataRegionConfiguration">
> <bean
> class="org.apache.ignite.configuration.DataRegionConfiguration">
> <property name="initialSize"
> value="#{1L * 1024 * 1024 * 1024}"/>
> <property name="maxSize"
> value="#{1L * 1024 * 1024 * 1024}"/>
> <property
> name="persistenceEnabled" value="true"/>
> </bean>
> </property>
> <property name="pageSize" value="#{4 * 1024}"/>
> </bean>
> </property>
> <property name="cacheConfiguration">
> <list>
> <bean
> class="org.apache.ignite.configuration.CacheConfiguration">
> <property name="name" value="vds"/>
> <property name="cacheMode"
> value="PARTITIONED"/>
> <property name="atomicityMode"
> value="TRANSACTIONAL"/>
> <property name="backups" value="1"/>
> <property name="writeSynchronizationMode"
> value="PRIMARY_SYNC"/>
> <property name="onheapCacheEnabled"
> value="false"/>
> </bean>
> </list>
> </property>
> </bean>
>
> and, at some point during a transaction of ~1GB, I get this error:
>
> [15:01:23,559][SEVERE][client-connector-#134][] JVM will be halted
> immediately due to the failure: [failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=class o.a.i.i.mem.IgniteOutOfMemoryException: Out
> of memory in data region [name=default, initSize=1.0 GiB, maxSize=1.0 GiB,
> *persistenceEnabled=true*] Try the following:
> ^-- Increase maximum off-heap memory size
> (DataRegionConfiguration.maxSize)
> ^-- *Enable Ignite persistence*
> (DataRegionConfiguration.persistenceEnabled)
> ^-- Enable eviction or expiration policies]]
>
> Maybe the off-heap memory must be, at least, as large as the transaction we
> want to commit?
>
> Thanks in advance!
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>