You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by "Artem Zinnatullin (JIRA)" <ji...@apache.org> on 2018/10/19 16:57:00 UTC
[jira] [Created] (IGNITE-9952) Failed to process write update
request in FULL_ASYNC mode
Artem Zinnatullin created IGNITE-9952:
-----------------------------------------
Summary: Failed to process write update request in FULL_ASYNC mode
Key: IGNITE-9952
URL: https://issues.apache.org/jira/browse/IGNITE-9952
Project: Ignite
Issue Type: Bug
Affects Versions: 2.6
Environment: * Kubernetes
* Official Ignite 2.6.0 Docker image
Reporter: Artem Zinnatullin
Cluster of 12 Ignite server nodes, 4 client nodes.
Following cache configuration:
{code:java}
<property name="cacheConfiguration">
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="mycache"/>
<property name="dataRegionName" value="Default_Region"/>
<!-- Distribute cache across server nodes. -->
<property name="cacheMode" value="PARTITIONED"/>
<!-- Store a backup of each key/value pair, allows survive node downtime at cost of storing less data. -->
<property name="backups" value="1"/>
<!-- Do not wait for the end of cache write operation(s). -->
<property name="writeSynchronizationMode" value="FULL_ASYNC"/>
<!-- Throttle rebalancing messages by such pause (ms), otherwise it can over-consume CPU/memory. -->
<property name="rebalanceThrottle" value="20"/>
<!-- Size of rebalancing batch in bytes (default is 512 KB). -->
<property name="rebalanceBatchSize" value="#{2L * 1024 * 1024}"/>
</bean>
</property>
{code}
Ignite is only used as key/value cache.
Periodically these exceptions can be found in Ignite server nodes logs:
{code:java}
[23:17:23,586][INFO][grid-timeout-worker-#23][IgniteKernal] FreeList [name=null, buckets=256, dataPages=32070, reusePages=1435]
[23:17:28,586][INFO][grid-timeout-worker-#23][IgniteKernal]
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
^-- Node [id=37ea392d, uptime=51:52:58.760]
^-- H/N/C [hosts=16, nodes=16, CPUs=128]
^-- CPU [cur=0.4%, avg=0.44%, GC=0%]
^-- PageMemory [pages=2827437]
^-- Heap [used=1866MB, free=63.54%, comm=5120MB]
^-- Non heap [used=94MB, free=-1%, comm=95MB]
^-- Outbound messages queue [size=0]
^-- Public thread pool [active=0, idle=2, qSize=0]
^-- System thread pool [active=0, idle=8, qSize=0]
[23:17:28,586][INFO][grid-timeout-worker-#23][IgniteKernal] FreeList [name=null, buckets=256, dataPages=31943, reusePages=1435]
[23:17:30,506][SEVERE][sys-stripe-5-#6][GridDhtAtomicCache] <mycache> Unexpected exception during cache update
java.lang.NullPointerException
[23:17:30,506][SEVERE][sys-stripe-5-#6][GridDhtAtomicCache] <mycache> Failed to process write update request in FULL_ASYNC mode for keys: [KeyCacheObjectImpl [part=637, val=b/cb65d27d2837d6864edb03cce62aa9c29c540669, hasValBytes=true]]
class org.apache.ignite.IgniteCheckedException: Failed to update keys on primary node.
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.UpdateErrors.addFailedKeys(UpdateErrors.java:124)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateResponse.addFailedKeys(GridNearAtomicUpdateResponse.java:342)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1785)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1628)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processNearAtomicUpdateRequest(GridDhtAtomicCache.java:3056)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$400(GridDhtAtomicCache.java:130)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$5.apply(GridDhtAtomicCache.java:266)
at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$5.apply(GridDhtAtomicCache.java:261)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:378)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:304)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:99)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:293)
at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091)
at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:511)
at java.lang.Thread.run(Thread.java:748)
Suppressed: java.lang.NullPointerException
[23:17:33,588][INFO][grid-timeout-worker-#23][IgniteKernal]
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
^-- Node [id=37ea392d, uptime=51:53:03.763]
^-- H/N/C [hosts=16, nodes=16, CPUs=128]
^-- CPU [cur=1.67%, avg=0.44%, GC=0%]
^-- PageMemory [pages=2827437]
^-- Heap [used=2123MB, free=58.52%, comm=5120MB]
^-- Non heap [used=94MB, free=-1%, comm=95MB]
^-- Outbound messages queue [size=0]
^-- Public thread pool [active=0, idle=2, qSize=0]
^-- System thread pool [active=0, idle=8, qSize=0]
[23:17:33,588][INFO][grid-timeout-worker-#23][IgniteKernal] FreeList [name=null, buckets=256, dataPages=32119, reusePages=1435]
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)