You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Aleksey Plekhanov (Jira)" <ji...@apache.org> on 2020/07/22 06:12:00 UTC
[jira] [Updated] (IGNITE-12456) Cluster Data Store grid gets
Corrupted for Load test
[ https://issues.apache.org/jira/browse/IGNITE-12456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aleksey Plekhanov updated IGNITE-12456:
---------------------------------------
Fix Version/s: (was: 2.9)
2.10
> Cluster Data Store grid gets Corrupted for Load test
> ----------------------------------------------------
>
> Key: IGNITE-12456
> URL: https://issues.apache.org/jira/browse/IGNITE-12456
> Project: Ignite
> Issue Type: Bug
> Components: general
> Affects Versions: 2.7
> Reporter: Ravi Kumar Powli
> Assignee: Alexey Goncharuk
> Priority: Critical
> Fix For: 2.10
>
> Attachments: default-config.xml
>
>
> We have Apache Ignite 3 node cluster setup with Amazon S3 Based Discovery.we are into AWS cloud environment with Microservice model with 8 microservices.we are using Ignite for session data store.While performing load test we are facing data grid issues if the number of clients reached 40 , Once data grid got corrupted we will lost the session store data and Application will not respond due to session data also corrupted and all the instances will auto scaled down to initial size 8.We need to restart Apache Ignite to make the Application up.Please find the attached Apache Ignite Configuration for your reference.
> This is Impacting the scalability for the micro-services. It is very evident that the current state based architecture will not scale beyond a certain TPS and the state store, especially Ignite becomes the single point of failure, stalling the full Micro-service cluster.
> Apache Ignite Version : 2.7.0
> {code}
> 07:24:46,678][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G] Blocked system-critical thread has been detected. This can lead to cluster-wide undefined behaviour [threadName=sys-stripe-5, blockedFor=21s]07:24:46,678][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G] Blocked system-critical thread has been detected. This can lead to cluster-wide undefined behaviour [threadName=sys-stripe-5, blockedFor=21s][07:24:46,680][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=sys-stripe-5, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271465499]]]class org.apache.ignite.IgniteException: GridWorker [name=sys-stripe-5, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271465499] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233) at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)[07:24:52,692][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G] Blocked system-critical thread has been detected. This can lead to cluster-wide undefined behaviour [threadName=ttl-cleanup-worker, blockedFor=27s][07:24:52,692][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=ttl-cleanup-worker, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271465044]]]class org.apache.ignite.IgniteException: GridWorker [name=ttl-cleanup-worker, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271465044] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233) at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)[07:24:55,694][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G] Blocked system-critical thread has been detected. This can lead to cluster-wide undefined behaviour [threadName=sys-stripe-1, blockedFor=24s][07:24:55,694][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=sys-stripe-1, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271471538]]]class org.apache.ignite.IgniteException: GridWorker [name=sys-stripe-1, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271471538] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233) at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)[07:24:58,700][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G] Blocked system-critical thread has been detected. This can lead to cluster-wide undefined behaviour [threadName=sys-stripe-4, blockedFor=29s][07:24:58,700][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=sys-stripe-4, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271469397]]]class org.apache.ignite.IgniteException: GridWorker [name=sys-stripe-4, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271469397] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233) at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)[07:24:59,833][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G] Blocked system-critical thread has been detected. This can lead to cluster-wide undefined behaviour [threadName=sys-stripe-7, blockedFor=29s][07:24:59,833][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=sys-stripe-7, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271469973]]]class org.apache.ignite.IgniteException: GridWorker [name=sys-stripe-7, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271469973] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233) at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)[07:25:04,716][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G] Blocked system-critical thread has been detected. This can lead to cluster-wide undefined behaviour [threadName=sys-stripe-0, blockedFor=39s][07:25:04,716][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=sys-stripe-0, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271464782]]]class org.apache.ignite.IgniteException: GridWorker [name=sys-stripe-0, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271464782] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233) at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)[07:25:05,870][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G] Blocked system-critical thread has been detected. This can lead to cluster-wide undefined behaviour [threadName=sys-stripe-3, blockedFor=34s][07:25:05,870][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=sys-stripe-3, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271471528]]]class org.apache.ignite.IgniteException: GridWorker [name=sys-stripe-3, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271471528] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233) at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)[07:25:07,011][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G] Blocked system-critical thread has been detected. This can lead to cluster-wide undefined behaviour [threadName=sys-stripe-6, blockedFor=41s][07:25:07,011][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=sys-stripe-6, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271465428]]]class org.apache.ignite.IgniteException: GridWorker [name=sys-stripe-6, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271465428] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233) at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)[07:25:08,167][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G] Blocked system-critical thread has been detected. This can lead to cluster-wide undefined behaviour [threadName=sys-stripe-5, blockedFor=42s][07:25:08,167][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=sys-stripe-5, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271465499]]]class org.apache.ignite.IgniteException: GridWorker [name=sys-stripe-5, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271465499] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233) at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)[07:25:09,310][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G] Blocked system-critical thread has been detected. This can lead to cluster-wide undefined behaviour [threadName=sys-stripe-2, blockedFor=40s][07:25:09,311][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=sys-stripe-2, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271468397]]]class org.apache.ignite.IgniteException: GridWorker [name=sys-stripe-2, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271468397] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233) at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)[07:25:14,183][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][G] Blocked system-critical thread has been detected. This can lead to cluster-wide undefined behaviour [threadName=ttl-cleanup-worker, blockedFor=49s][07:25:14,183][SEVERE][tcp-disco-msg-worker-#2%DataStoreIgniteCache%|#2%DataStoreIgniteCache%][] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker [name=ttl-cleanup-worker, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271465044]]]class org.apache.ignite.IgniteException: GridWorker [name=ttl-cleanup-worker, igniteInstanceName=DataStoreIgniteCache, finished=false, heartbeatTs=1575271465044] at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826) at org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233) at org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)