You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by xiaweidong <52...@qq.com> on 2020/08/17 08:02:58 UTC
Pure memory mode error in Azure Kubernetes
Hello everyone, I have a question that has been bothering me ;I can not
create a Ignite Kubernetes cluster in Azure Kubernetes from the file
'quark-ignite.xml' quark-ignite.xml: <?xml version="1.0"
encoding="UTF-8"?>
I set persistenceEnabled false , When I write the data , ignite will report
an error ;From the error log, the memory resource is requested until the set
maximum memory is reached;Below is the detailed log information:[2020-08-17
07:40:12] [INFO] [QUARK] [202:client-connector-#116%quark%]
[org.apache.ignite.logger.log4j2.Log4J2Logger.info(Log4J2Logger.java:478)]
[] SERVER Allocated next memory segment [plcName=Default_Region,
chunkSize=268.4 MB][2020-08-17 07:40:13] [INFO] [QUARK]
[205:client-connector-#119%quark%]
[org.apache.ignite.logger.log4j2.Log4J2Logger.info(Log4J2Logger.java:478)]
[] SERVER Allocated next memory segment [plcName=Default_Region,
chunkSize=268.4 MB][2020-08-17 07:40:13] [INFO] [QUARK]
[202:client-connector-#116%quark%]
[org.apache.ignite.logger.log4j2.Log4J2Logger.info(Log4J2Logger.java:478)]
[] SERVER Allocated next memory segment [plcName=Default_Region,
chunkSize=268.4 MB][2020-08-17 07:40:14] [INFO] [QUARK]
[204:client-connector-#118%quark%]
[org.apache.ignite.logger.log4j2.Log4J2Logger.info(Log4J2Logger.java:478)]
[] SERVER Allocated next memory segment [plcName=Default_Region,
chunkSize=268.4 MB][2020-08-17 07:40:15] [INFO] [QUARK]
[202:client-connector-#116%quark%]
[org.apache.ignite.logger.log4j2.Log4J2Logger.info(Log4J2Logger.java:478)]
[] SERVER Allocated next memory segment [plcName=Default_Region,
chunkSize=268.4 MB][2020-08-17 07:40:15] [INFO] [QUARK]
[201:client-connector-#115%quark%]
[org.apache.ignite.logger.log4j2.Log4J2Logger.info(Log4J2Logger.java:478)]
[] SERVER Allocated next memory segment [plcName=Default_Region,
chunkSize=268.4 MB][2020-08-17 07:40:16] [INFO] [QUARK]
[201:client-connector-#115%quark%]
[org.apache.ignite.logger.log4j2.Log4J2Logger.info(Log4J2Logger.java:478)]
[] SERVER Allocated next memory segment [plcName=Default_Region,
chunkSize=268.4 MB] ^-- Enable Ignite persistence
(DataRegionConfiguration.persistenceEnabled) at
org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager.ensureFreeSpaceForInsert(IgniteCacheDatabaseSharedManager.java:1063)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:6160)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.invokeClosure(BPlusTree.java:3817)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1955)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1705)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:445)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2557)
[ignite-core-2.8.1.jar:2.8.1] ^-- Enable eviction or expiration policies at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1839)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:2102)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1719)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:300)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:249)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.put0(GridDhtAtomicCache.java:624)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.platform.client.cache.ClientCachePutRequest.process(ClientCachePutRequest.java:40)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:49)
[ignite-core-2.8.1.jar:2.8.1] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_212] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212] ^--
Enable eviction or expiration
policies]]org.apache.ignite.internal.mem.IgniteOutOfMemoryException: Out of
memory in data region [name=Default_Region, initSize=256.0 MiB, maxSize=2.0
GiB, persistenceEnabled=false] Try the following: at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:5849)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1919)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:446)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1299)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:856)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.platform.client.ClientRequestHandler.handle(ClientRequestHandler.java:98)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
[ignite-core-2.8.1.jar:2.8.1] ^-- Increase maximum off-heap memory size
(DataRegionConfiguration.maxSize) at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.update(GridCacheMapEntry.java:6409)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1164)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2580)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:195)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70)
[ignite-core-2.8.1.jar:2.8.1] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_212][2020-08-17 07:40:16] [ERROR] [QUARK]
[210:client-connector-#124%quark%]
[org.apache.ignite.logger.log4j2.Log4J2Logger.error(Log4J2Logger.java:498)]
[] SERVER Critical system error detected. Will be handled accordingly to
configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false,
timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet
[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]],
failureCtx=FailureContext [type=CRITICAL_ERROR, err=class
o.a.i.i.mem.IgniteOutOfMemoryException: Out of memory in data region
[name=Default_Region, initSize=256.0 MiB, maxSize=2.0 GiB,
persistenceEnabled=false] Try the following: ^-- Increase maximum off-heap
memory size (DataRegionConfiguration.maxSize) ^-- Enable Ignite persistence
(DataRegionConfiguration.persistenceEnabled) at
org.apache.ignite.internal.processors.cache.persistence.RowStore.addRow(RowStore.java:106)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.createRow(IgniteCacheOffheapManagerImpl.java:1755)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.access$5700(BPlusTree.java:3711)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1688)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:2314)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2641)
[ignite-core-2.8.1.jar:2.8.1] at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:486)
[ignite-core-2.8.1.jar:2.8.1]Thread=[name=client-connector-#115%quark%,
id=201], state=WAITINGThread=[name=client-connector-#119%quark%, id=205],
state=WAITINGLocked pages log: name=client-connector-#120%quark%
time=(1597650016926, 2020-08-17 07:40:16.926)Locked pages = [] ^-- Enable
eviction or expiration policies]]Locked pages = []Locked pages =
[]Thread=[name=client-connector-#120%quark%, id=206],
state=WAITINGThread=[name=client-connector-#124%quark%, id=210],
state=RUNNABLEThread=[name=exchange-worker-#37%quark%, id=85],
state=TIMED_WAITINGLocked pages = []Locked pages log:
name=client-connector-#116%quark% time=(1597650016926, 2020-08-17
07:40:16.926)Locked pages log: name=client-connector-#117%quark%
time=(1597650016926, 2020-08-17 07:40:16.926)Locked pages = []Locked pages =
[]Thread=[name=client-connector-#123%quark%, id=209], state=WAITINGLocked
pages = []Locked pages = [] ^-- Enable Ignite persistence
(DataRegionConfiguration.persistenceEnabled)Locked pages log:
name=client-connector-#119%quark% time=(1597650016926, 2020-08-17
07:40:16.926)Locked pages log: name=client-connector-#123%quark%
time=(1597650016926, 2020-08-17 07:40:16.926)Locked pages log:
name=client-connector-#124%quark% time=(1597650016926, 2020-08-17
07:40:16.926)Locked pages log: name=exchange-worker-#37%quark%
time=(1597650016926, 2020-08-17 07:40:16.926)[2020-08-17 07:40:16] [WARN]
[QUARK] [210:client-connector-#124%quark%]
[org.apache.ignite.logger.log4j2.Log4J2Logger.warning(Log4J2Logger.java:488)]
[] SERVER Page locks dump:Locked pages = []Locked pages log:
name=client-connector-#115%quark% time=(1597650016926, 2020-08-17
07:40:16.926)Thread=[name=client-connector-#116%quark%, id=202],
state=WAITINGThread=[name=client-connector-#117%quark%, id=203],
state=WAITINGThread=[name=client-connector-#118%quark%, id=204],
state=WAITINGLocked pages log: name=client-connector-#118%quark%
time=(1597650016926, 2020-08-17 07:40:16.926)[2020-08-17 07:40:16] [ERROR]
[QUARK] [210:client-connector-#124%quark%]
[org.apache.ignite.logger.log4j2.Log4J2Logger.error(Log4J2Logger.java:498)]
[] SERVER JVM will be halted immediately due to the failure:
[failureCtx=FailureContext [type=CRITICAL_ERROR, err=class
o.a.i.i.mem.IgniteOutOfMemoryException: Out of memory in data region
[name=Default_Region, initSize=256.0 MiB, maxSize=2.0 GiB,
persistenceEnabled=false] Try the following: ^-- Increase maximum off-heap
memory size (DataRegionConfiguration.maxSize)
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Pure memory mode error in Azure Kubernetes
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Data region size is a purely configuration limitation, it is not affected
by hardware or OS. So you must be doing something different in the scenario
where you are getting it.
I recommend enabling page eviction, see if you no longer see this error
(but discard data), then tune your region size.
Regards,
--
Ilya Kasnacheev
пн, 17 авг. 2020 г. в 12:10, xiaweidong <52...@qq.com>:
> Thank you for your reply
> I'm pretty sure I didn't get more than 2g of memory.
>
> Roughly 7,000 or so pieces of data would fill up 2g of memory, and I'm
> running on Linux first, Memory consumption is far less than 2g.
> When I move program to Aks, The program starts reporting errors.
>
> I've tested, when I run the k8S node above the first time to ignite, is
> able to run perfectly, when I restart the Ignite and write data, write a
> few will start the OOM;
> When I restart the k8S node and running ignite again, It won't report an
> error;
> I think it's a compatibility issue between the two, and that's just my
> personal idea.
>
> ------------------------------
> Sent from the Apache Ignite Users mailing list archive
> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>
Re: Pure memory mode error in Azure Kubernetes
Posted by xiaweidong <52...@qq.com>.
Thank you for your reply
I'm pretty sure I didn't get more than 2g of memory.
Roughly 7,000 or so pieces of data would fill up 2g of memory, and I'm
running on Linux first, Memory consumption is far less than 2g.
When I move program to Aks, The program starts reporting errors.
I've tested,when I run the k8S node above the first time to ignite, is able
to run perfectly, when I restart the Ignite and write data, write a few will
start the OOM;
When I restart the k8S node and running ignite again, It won't report an
error;
I think it's a compatibility issue between the two, and that's just my
personal idea.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Pure memory mode error in Azure Kubernetes
Posted by Stephen Darlington <st...@gridgain.com>.
It’s exactly as the error message says: you ran out of memory.
org.apache.ignite.internal.mem.IgniteOutOfMemoryException: Out of memory in data region [name=Default_Region, initSize=256.0 MiB, maxSize=2.0 GiB, persistenceEnabled=false]
You can:
Allocate more memory (2Gb isn’t very much)
Use persistence
Set an eviction policy, so that “old” data is automatically removed
Regards,
Stephen
> On 17 Aug 2020, at 09:02, xiaweidong <52...@qq.com> wrote:
>
> Hello everyone, I have a question that has been bothering me ; I can not create a Ignite Kubernetes cluster in Azure Kubernetes from the file 'quark-ignite.xml' quark-ignite.xml: I set persistenceEnabled false , When I write the data , ignite will report an error ; From the error log, the memory resource is requested until the set maximum memory is reached; Below is the detailed log information: [2020-08-17 07:40:12] [INFO] [QUARK] [202:client-connector-#116%quark%] [org.apache.ignite.logger.log4j2.Log4J2Logger.info(Log4J2Logger.java:478)] [] SERVER Allocated next memory segment [plcName=Default_Region, chunkSize=268.4 MB] [2020-08-17 07:40:13] [INFO] [QUARK] [205:client-connector-#119%quark%] [org.apache.ignite.logger.log4j2.Log4J2Logger.info(Log4J2Logger.java:478)] [] SERVER Allocated next memory segment [plcName=Default_Region, chunkSize=268.4 MB] [2020-08-17 07:40:13] [INFO] [QUARK] [202:client-connector-#116%quark%] [org.apache.ignite.logger.log4j2.Log4J2Logger.info(Log4J2Logger.java:478)] [] SERVER Allocated next memory segment [plcName=Default_Region, chunkSize=268.4 MB] [2020-08-17 07:40:14] [INFO] [QUARK] [204:client-connector-#118%quark%] [org.apache.ignite.logger.log4j2.Log4J2Logger.info(Log4J2Logger.java:478)] [] SERVER Allocated next memory segment [plcName=Default_Region, chunkSize=268.4 MB] [2020-08-17 07:40:15] [INFO] [QUARK] [202:client-connector-#116%quark%] [org.apache.ignite.logger.log4j2.Log4J2Logger.info(Log4J2Logger.java:478)] [] SERVER Allocated next memory segment [plcName=Default_Region, chunkSize=268.4 MB] [2020-08-17 07:40:15] [INFO] [QUARK] [201:client-connector-#115%quark%] [org.apache.ignite.logger.log4j2.Log4J2Logger.info(Log4J2Logger.java:478)] [] SERVER Allocated next memory segment [plcName=Default_Region, chunkSize=268.4 MB] [2020-08-17 07:40:16] [INFO] [QUARK] [201:client-connector-#115%quark%] [org.apache.ignite.logger.log4j2.Log4J2Logger.info(Log4J2Logger.java:478)] [] SERVER Allocated next memory segment [plcName=Default_Region, chunkSize=268.4 MB] ^-- Enable Ignite persistence (DataRegionConfiguration.persistenceEnabled) at org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager.ensureFreeSpaceForInsert(IgniteCacheDatabaseSharedManager.java:1063) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:6160) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.invokeClosure(BPlusTree.java:3817) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1955) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1705) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:445) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2557) [ignite-core-2.8.1.jar:2.8.1] ^-- Enable eviction or expiration policies at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1839) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:2102) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1719) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:300) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:249) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.put0(GridDhtAtomicCache.java:624) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.platform.client.cache.ClientCachePutRequest.process(ClientCachePutRequest.java:40) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:49) [ignite-core-2.8.1.jar:2.8.1] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212] ^-- Enable eviction or expiration policies]] org.apache.ignite.internal.mem.IgniteOutOfMemoryException: Out of memory in data region [name=Default_Region, initSize=256.0 MiB, maxSize=2.0 GiB, persistenceEnabled=false] Try the following: at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.call(GridCacheMapEntry.java:5849) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1919) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:446) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1299) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:856) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.platform.client.ClientRequestHandler.handle(ClientRequestHandler.java:98) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) [ignite-core-2.8.1.jar:2.8.1] ^-- Increase maximum off-heap memory size (DataRegionConfiguration.maxSize) at org.apache.ignite.internal.processors.cache.GridCacheMapEntry$AtomicCacheUpdateClosure.update(GridCacheMapEntry.java:6409) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1164) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2580) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:195) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70) [ignite-core-2.8.1.jar:2.8.1] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212] [2020-08-17 07:40:16] [ERROR] [QUARK] [210:client-connector-#124%quark%] [org.apache.ignite.logger.log4j2.Log4J2Logger.error(Log4J2Logger.java:498)] [] SERVER Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=CRITICAL_ERROR, err=class o.a.i.i.mem.IgniteOutOfMemoryException: Out of memory in data region [name=Default_Region, initSize=256.0 MiB, maxSize=2.0 GiB, persistenceEnabled=false] Try the following: ^-- Increase maximum off-heap memory size (DataRegionConfiguration.maxSize) ^-- Enable Ignite persistence (DataRegionConfiguration.persistenceEnabled) at org.apache.ignite.internal.processors.cache.persistence.RowStore.addRow(RowStore.java:106) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.createRow(IgniteCacheOffheapManagerImpl.java:1755) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.access$5700(BPlusTree.java:3711) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1688) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:2314) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2641) [ignite-core-2.8.1.jar:2.8.1] at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:486) [ignite-core-2.8.1.jar:2.8.1] Thread=[name=client-connector-#115%quark%, id=201], state=WAITING Thread=[name=client-connector-#119%quark%, id=205], state=WAITING Locked pages log: name=client-connector-#120%quark% time=(1597650016926, 2020-08-17 07:40:16.926) Locked pages = [] ^-- Enable eviction or expiration policies]] Locked pages = [] Locked pages = [] Thread=[name=client-connector-#120%quark%, id=206], state=WAITING Thread=[name=client-connector-#124%quark%, id=210], state=RUNNABLE Thread=[name=exchange-worker-#37%quark%, id=85], state=TIMED_WAITING Locked pages = [] Locked pages log: name=client-connector-#116%quark% time=(1597650016926, 2020-08-17 07:40:16.926) Locked pages log: name=client-connector-#117%quark% time=(1597650016926, 2020-08-17 07:40:16.926) Locked pages = [] Locked pages = [] Thread=[name=client-connector-#123%quark%, id=209], state=WAITING Locked pages = [] Locked pages = [] ^-- Enable Ignite persistence (DataRegionConfiguration.persistenceEnabled) Locked pages log: name=client-connector-#119%quark% time=(1597650016926, 2020-08-17 07:40:16.926) Locked pages log: name=client-connector-#123%quark% time=(1597650016926, 2020-08-17 07:40:16.926) Locked pages log: name=client-connector-#124%quark% time=(1597650016926, 2020-08-17 07:40:16.926) Locked pages log: name=exchange-worker-#37%quark% time=(1597650016926, 2020-08-17 07:40:16.926) [2020-08-17 07:40:16] [WARN] [QUARK] [210:client-connector-#124%quark%] [org.apache.ignite.logger.log4j2.Log4J2Logger.warning(Log4J2Logger.java:488)] [] SERVER Page locks dump: Locked pages = [] Locked pages log: name=client-connector-#115%quark% time=(1597650016926, 2020-08-17 07:40:16.926) Thread=[name=client-connector-#116%quark%, id=202], state=WAITING Thread=[name=client-connector-#117%quark%, id=203], state=WAITING Thread=[name=client-connector-#118%quark%, id=204], state=WAITING Locked pages log: name=client-connector-#118%quark% time=(1597650016926, 2020-08-17 07:40:16.926) [2020-08-17 07:40:16] [ERROR] [QUARK] [210:client-connector-#124%quark%] [org.apache.ignite.logger.log4j2.Log4J2Logger.error(Log4J2Logger.java:498)] [] SERVER JVM will be halted immediately due to the failure: [failureCtx=FailureContext [type=CRITICAL_ERROR, err=class o.a.i.i.mem.IgniteOutOfMemoryException: Out of memory in data region [name=Default_Region, initSize=256.0 MiB, maxSize=2.0 GiB, persistenceEnabled=false] Try the following: ^-- Increase maximum off-heap memory size (DataRegionConfiguration.maxSize)
> Sent from the Apache Ignite Users mailing list archive <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.