You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Taras Ledkov (Jira)" <ji...@apache.org> on 2022/08/24 10:58:00 UTC
[jira] [Commented] (IGNITE-17317) Test IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata is flaky on TC
[ https://issues.apache.org/jira/browse/IGNITE-17317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584177#comment-17584177 ]
Taras Ledkov commented on IGNITE-17317:
---------------------------------------
[~sergeychugunov], looks like resolved [commit|https://github.com/apache/ignite/commit/7aca244b68d81fa1f891d8f767be2a53b29438a3]. Please resolve the issue
> Test IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata is flaky on TC
> -------------------------------------------------------------------------------------------------------------
>
> Key: IGNITE-17317
> URL: https://issues.apache.org/jira/browse/IGNITE-17317
> Project: Ignite
> Issue Type: Bug
> Reporter: Sergey Chugunov
> Assignee: Sergey Chugunov
> Priority: Major
> Fix For: 2.14
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Test history is available here: [TC link|https://ci.ignite.apache.org/test/-1717009344468901983?currentProjectId=IgniteTests24Java8&branch=%3Cdefault%3E]
> Test fails on a cache put operation before a condition being verified is reached:
> {code:java}
> org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys (retry update if possible).: [2]
> at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1260)
> at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2111)
> at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1347)
> at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:870)
> at org.apache.ignite.internal.processors.cache.persistence.IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata(IgnitePdsBinaryMetadataAsyncWritingTest.java:257)
> {code}
> There is also a stack trace in logs indicating there is some problem when writing to WAL:
> {code:java}
> org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys (retry update if possible).: [2]
> at org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1260)
> at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2111)
> at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1347)
> at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:870)
> at org.apache.ignite.internal.processors.cache.persistence.IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata(IgnitePdsBinaryMetadataAsyncWritingTest.java:257)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2428)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: class org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException: Failed to update keys (retry update if possible).: [2]
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.onPrimaryError(GridNearAtomicAbstractUpdateFuture.java:401)
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.onPrimaryResponse(GridNearAtomicSingleUpdateFuture.java:248)
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processNearAtomicUpdateResponse(GridDhtAtomicCache.java:3213)
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$500(GridDhtAtomicCache.java:145)
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$6.apply(GridDhtAtomicCache.java:305)
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$6.apply(GridDhtAtomicCache.java:300)
> at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1151)
> at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:592)
> at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:393)
> at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:319)
> at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:110)
> at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:309)
> at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1907)
> at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1528)
> at org.apache.ignite.internal.managers.communication.GridIoManager.access$5300(GridIoManager.java:243)
> at org.apache.ignite.internal.managers.communication.GridIoManager$9.execute(GridIoManager.java:1421)
> at org.apache.ignite.internal.managers.communication.TraceRunnable.run(TraceRunnable.java:55)
> at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:637)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
> ... 1 more
> Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to update keys.
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.UpdateErrors.addFailedKey(UpdateErrors.java:108)
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateResponse.addFailedKey(GridNearAtomicUpdateResponse.java:327)
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2696)
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:2012)
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1831)
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1704)
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processNearAtomicUpdateRequest(GridDhtAtomicCache.java:3195)
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$400(GridDhtAtomicCache.java:145)
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$5.apply(GridDhtAtomicCache.java:286)
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$5.apply(GridDhtAtomicCache.java:281)
> ... 14 more
> Suppressed: class org.apache.ignite.internal.processors.cache.persistence.StorageException: Failed to perform WAL operation (environment was invalidated by a previous error)
> at org.apache.ignite.internal.processors.cache.persistence.wal.filehandle.FileWriteHandleImpl.checkNode(FileWriteHandleImpl.java:198)
> at org.apache.ignite.internal.processors.cache.persistence.wal.filehandle.FileWriteHandleImpl.addRecord(FileWriteHandleImpl.java:225)
> at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.log(FileWriteAheadLogManager.java:926)
> at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.log(FileWriteAheadLogManager.java:879)
> at org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.allocatePage(PageMemoryImpl.java:617)
> at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.getOrAllocatePartitionMetas(GridCacheOffheapManager.java:2279)
> at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.init0(GridCacheOffheapManager.java:2016)
> at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridCacheOffheapManager.java:2815)
> at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:449)
> at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:2333)
> at org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2552)
> ... 21 more
> {code}
> Most likely the problem is with the test logic itself - some kind of a race between two conditions both of which could fail the node.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)