You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Ivan Bessonov (Jira)" <ji...@apache.org> on 2023/05/10 14:21:00 UTC
[jira] [Assigned] (IGNITE-19439) AssertionError in RemoveWriteOnGcInvokeClosure during insertion
[ https://issues.apache.org/jira/browse/IGNITE-19439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Bessonov reassigned IGNITE-19439:
--------------------------------------
Assignee: Ivan Bessonov
> AssertionError in RemoveWriteOnGcInvokeClosure during insertion
> ---------------------------------------------------------------
>
> Key: IGNITE-19439
> URL: https://issues.apache.org/jira/browse/IGNITE-19439
> Project: Ignite
> Issue Type: Bug
> Reporter: Konstantin Orlov
> Assignee: Ivan Bessonov
> Priority: Major
> Labels: ignite-3
> Attachments: logs.txt
>
>
> I run a test that in an endless loop inserts 10k rows , clears the table, and starts over, but it failed on 4th iteration with AssertionError.
> The test is as follow:
> {code:java}
> // org.apache.ignite.internal.sql.engine.ItDmlTest
> private static final String STRING_PAYLOAD = IgniteTestUtils.randomString(new Random(), 32);
> @Test
> public void test() throws SQLException {
> int TABLE_SIZE = 10_000;
> sql("CREATE TABLE t (id INT PRIMARY KEY, name VARCHAR, surname VARCHAR, age TINYINT)");
> int iteration = 1;
> do {
> System.out.println("Iteration " + (iteration++) + " started");
> long before = System.currentTimeMillis();
> for (int i = 1; i <= TABLE_SIZE; i++) {
> sql("INSERT INTO t VALUES (?, ?, ?, ?)", i, STRING_PAYLOAD, STRING_PAYLOAD, (byte) i);
> if (i % 10 == 0) {
> long after = System.currentTimeMillis();
> System.out.println(i + " rows inserted. dT=" + Duration.ofMillis(after - before));
> before = after;
> }
> }
> sql("DELETE FROM t");
> } while (true);
> } {code}
> The assertion is:
> {code:java}
> 023-05-08 20:17:51:016 +0300 [WARNING][%idt_n_0%JRaft-Request-Processor-0][ReplicaManager] Failed to process replica request [request=ReadWriteMultiRowReplicaRequestImpl [binaryRows=ArrayList [org.apache.ignite.internal.schema.row.Row@2b6ae605], commitPartitionId=f6b9512f-b735-446e-85a3-d6b8636c847c_part_18, groupId=f6b9512f-b735-446e-85a3-d6b8636c847c_part_1, requestType=RW_INSERT_ALL, term=1, timestampLong=110334199137239040, transactionId=0187fc5e-9a26-0000-0000-000063a26a19]]
> java.util.concurrent.CompletionException: org.apache.ignite.internal.storage.StorageException: IGN-STORAGE-1 TraceId:c48307b5-59a0-4895-a79d-63153a9d47c6 Error removing row version from version chain on garbage collection: [rowId=RowId [partitionId=1, uuid=ca451c56-ea47-422e-b228-67c093ae1a10], rowTimestamp=HybridTimestamp [time=110334012513058818], table=T, partitionId=1]
> at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314)
> at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1113)
> at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2235)
> at org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.lambda$processMultiEntryAction$75(PartitionReplicaListener.java:1487)
> at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106)
> at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2235)
> at org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.processMultiEntryAction(PartitionReplicaListener.java:1445)
> at org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.lambda$processRequest$2(PartitionReplicaListener.java:285)
> at org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.appendTxCommand(PartitionReplicaListener.java:1197)
> at org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.processRequest(PartitionReplicaListener.java:285)
> at org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.lambda$invoke$0(PartitionReplicaListener.java:274)
> at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072)
> at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
> at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
> at org.apache.ignite.internal.raft.RaftGroupServiceImpl.lambda$sendWithRetry$39(RaftGroupServiceImpl.java:538)
> at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
> at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
> at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
> at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
> at org.apache.ignite.network.DefaultMessagingService.onInvokeResponse(DefaultMessagingService.java:371)
> at org.apache.ignite.network.DefaultMessagingService.send0(DefaultMessagingService.java:194)
> at org.apache.ignite.network.DefaultMessagingService.respond(DefaultMessagingService.java:137)
> at org.apache.ignite.network.MessagingService.respond(MessagingService.java:89)
> at org.apache.ignite.raft.jraft.rpc.impl.IgniteRpcServer$NetworkRpcContext.sendResponse(IgniteRpcServer.java:233)
> at org.apache.ignite.raft.jraft.rpc.RpcRequestProcessor.handleRequest(RpcRequestProcessor.java:52)
> at org.apache.ignite.raft.jraft.rpc.RpcRequestProcessor.handleRequest(RpcRequestProcessor.java:29)
> at org.apache.ignite.raft.jraft.rpc.impl.IgniteRpcServer$RpcMessageHandler.lambda$onReceived$0(IgniteRpcServer.java:192)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: org.apache.ignite.internal.storage.StorageException: IGN-STORAGE-1 TraceId:c48307b5-59a0-4895-a79d-63153a9d47c6 Error removing row version from version chain on garbage collection: [rowId=RowId [partitionId=1, uuid=ca451c56-ea47-422e-b228-67c093ae1a10], rowTimestamp=HybridTimestamp [time=110334012513058818], table=T, partitionId=1]
> at org.apache.ignite.internal.storage.pagememory.mv.AbstractPageMemoryMvPartitionStorage.removeWriteOnGc(AbstractPageMemoryMvPartitionStorage.java:968)
> at org.apache.ignite.internal.storage.pagememory.mv.AbstractPageMemoryMvPartitionStorage.vacuum(AbstractPageMemoryMvPartitionStorage.java:952)
> at org.apache.ignite.internal.storage.MvPartitionStorage.lambda$pollForVacuum$0(MvPartitionStorage.java:292)
> at org.apache.ignite.internal.storage.pagememory.mv.PersistentPageMemoryMvPartitionStorage.runConsistently(PersistentPageMemoryMvPartitionStorage.java:143)
> at org.apache.ignite.internal.storage.MvPartitionStorage.pollForVacuum(MvPartitionStorage.java:282)
> at org.apache.ignite.internal.table.distributed.raft.snapshot.outgoing.SnapshotAwarePartitionDataStorage.pollForVacuum(SnapshotAwarePartitionDataStorage.java:141)
> at org.apache.ignite.internal.table.distributed.StorageUpdateHandler.internalVacuum(StorageUpdateHandler.java:345)
> at org.apache.ignite.internal.table.distributed.StorageUpdateHandler.lambda$vacuumBatch$4(StorageUpdateHandler.java:332)
> at org.apache.ignite.internal.storage.pagememory.mv.PersistentPageMemoryMvPartitionStorage.lambda$runConsistently$0(PersistentPageMemoryMvPartitionStorage.java:155)
> at org.apache.ignite.internal.storage.pagememory.mv.AbstractPageMemoryMvPartitionStorage.busy(AbstractPageMemoryMvPartitionStorage.java:774)
> at org.apache.ignite.internal.storage.pagememory.mv.PersistentPageMemoryMvPartitionStorage.runConsistently(PersistentPageMemoryMvPartitionStorage.java:145)
> at org.apache.ignite.internal.table.distributed.raft.snapshot.outgoing.SnapshotAwarePartitionDataStorage.runConsistently(SnapshotAwarePartitionDataStorage.java:66)
> at org.apache.ignite.internal.table.distributed.StorageUpdateHandler.vacuumBatch(StorageUpdateHandler.java:332)
> at org.apache.ignite.internal.table.distributed.StorageUpdateHandler.executeBatchGc(StorageUpdateHandler.java:202)
> at org.apache.ignite.internal.table.distributed.StorageUpdateHandler.handleUpdateAll(StorageUpdateHandler.java:192)
> at org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.applyUpdateAllCommand(PartitionReplicaListener.java:1600)
> at org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener.lambda$processMultiEntryAction$72(PartitionReplicaListener.java:1487)
> at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106)
> ... 28 more
> Caused by: org.apache.ignite.internal.pagememory.tree.CorruptedTreeException: IGN-CMN-65535 TraceId:0f815e82-c0df-4f6e-8784-be38e9ed94cc B+Tree is corrupted [groupId=1, pageIds=[562954248388665], groupName=T, msg=Runtime failure on search row: org.apache.ignite.internal.storage.pagememory.mv.VersionChainKey@13e0d6a6]
> at org.apache.ignite.internal.pagememory.tree.BplusTree.corruptedTreeException(BplusTree.java:6724)
> at org.apache.ignite.internal.pagememory.tree.BplusTree.invoke(BplusTree.java:2135)
> at org.apache.ignite.internal.storage.pagememory.mv.AbstractPageMemoryMvPartitionStorage.removeWriteOnGc(AbstractPageMemoryMvPartitionStorage.java:961)
> ... 45 more
> Caused by: java.lang.AssertionError: rowId=RowId [partitionId=1, uuid=ca451c56-ea47-422e-b228-67c093ae1a10], storage=table=T, partitionId=1
> at org.apache.ignite.internal.storage.pagememory.mv.RemoveWriteOnGcInvokeClosure.call(RemoveWriteOnGcInvokeClosure.java:70)
> at org.apache.ignite.internal.storage.pagememory.mv.RemoveWriteOnGcInvokeClosure.call(RemoveWriteOnGcInvokeClosure.java:42)
> at org.apache.ignite.internal.pagememory.tree.BplusTree$Invoke.invokeClosure(BplusTree.java:4298)
> at org.apache.ignite.internal.pagememory.tree.BplusTree.invokeDown(BplusTree.java:2209)
> at org.apache.ignite.internal.pagememory.tree.BplusTree.invokeDown(BplusTree.java:2191)
> at org.apache.ignite.internal.pagememory.tree.BplusTree.invoke(BplusTree.java:2109)
> ... 46 more
> {code}
> Full log is attached below. [^logs.txt]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)