You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by "Cyrill (via GitHub)" <gi...@apache.org> on 2024/03/19 09:52:15 UTC

[PR] IGNITE-18617 Clear rw tx context and cleanup ready futures on tx finish [ignite-3]

Cyrill opened a new pull request, #3436:
URL: https://github.com/apache/ignite-3/pull/3436

   https://issues.apache.org/jira/browse/IGNITE-18617


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-18617 Clear rw tx context and cleanup ready futures on tx finish [ignite-3]

Posted by "Cyrill (via GitHub)" <gi...@apache.org>.
Cyrill commented on code in PR #3436:
URL: https://github.com/apache/ignite-3/pull/3436#discussion_r1534054293


##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1894,18 +1893,23 @@ private <T> CompletableFuture<T> appendTxCommand(UUID txId, RequestType cmdType,
         var cleanupReadyFut = new CompletableFuture<Void>();
 
         txCleanupReadyFutures.compute(txId, (id, txOps) -> {
-            if (txOps == null) {
-                txOps = new TxCleanupReadyFutureList();
-            }
-
+            // First check whether the transaction has already been finished.
+            // And complete cleanupReadyFut with exception if it is the case.
             TxStateMeta txStateMeta = txManager.stateMeta(txId);
 
             if (txStateMeta == null || isFinalState(txStateMeta.txState())) {
                 cleanupReadyFut.completeExceptionally(new Exception());
-            } else {
-                txOps.futures.computeIfAbsent(cmdType, type -> new ArrayList<>()).add(cleanupReadyFut);
+
+                return txOps;

Review Comment:
   Imagine we went into the `if`, block, in other words, the transaction has finished and the request is trying to execute on a finished transaction.
   There are two options - the cleanup has already finished or has not yet started, that means `txCleanupReadyFutures.compute...` from `awaitCleanupReadyFutures` either has already been called or is going to be called later.
   
   If `awaitCleanupReadyFutures` has already been called, then the value of txOps is null and we return null.
   If not - then txOps!=null and we will use it to wait on the corresponding futures later.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-18617 Clear rw tx context and cleanup ready futures on tx finish [ignite-3]

Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #3436:
URL: https://github.com/apache/ignite-3/pull/3436#discussion_r1534060302


##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1894,18 +1893,23 @@ private <T> CompletableFuture<T> appendTxCommand(UUID txId, RequestType cmdType,
         var cleanupReadyFut = new CompletableFuture<Void>();
 
         txCleanupReadyFutures.compute(txId, (id, txOps) -> {
-            if (txOps == null) {
-                txOps = new TxCleanupReadyFutureList();
-            }
-
+            // First check whether the transaction has already been finished.
+            // And complete cleanupReadyFut with exception if it is the case.
             TxStateMeta txStateMeta = txManager.stateMeta(txId);
 
             if (txStateMeta == null || isFinalState(txStateMeta.txState())) {
                 cleanupReadyFut.completeExceptionally(new Exception());
-            } else {
-                txOps.futures.computeIfAbsent(cmdType, type -> new ArrayList<>()).add(cleanupReadyFut);
+
+                return txOps;

Review Comment:
   Make sense.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-18617 Clear rw tx context and cleanup ready futures on tx finish [ignite-3]

Posted by "Cyrill (via GitHub)" <gi...@apache.org>.
Cyrill commented on code in PR #3436:
URL: https://github.com/apache/ignite-3/pull/3436#discussion_r1532172977


##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1746,7 +1745,7 @@ private CompletableFuture<FuturesCleanupResult> awaitCleanupReadyFutures(UUID tx
 
             txOps.futures.clear();
 
-            return txOps;
+            return null;

Review Comment:
   Changed the new operations checker part.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-18617 Clear rw tx context and cleanup ready futures on tx finish [ignite-3]

Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #3436:
URL: https://github.com/apache/ignite-3/pull/3436#discussion_r1531615487


##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TransactionInflights.java:
##########
@@ -99,6 +99,15 @@ public void removeInflight(UUID txId) {
         tuple.onInflightsRemoved();
     }
 
+    /**
+     * Cleanup tx inflights on tx finish.
+     *
+     * @param txId Transaction id.
+     */
+    void clearInflights(UUID txId) {

Review Comment:
   First of all let's be consistent with naming, few lines below there is a `void removeTxContexts(Collection<UUID> txIds)` - let's rename it to removeInflight, please also rename newly proposed methods to removeInflights.
   
   Generally, naming is confusing. TransactionInflights holds the map of TxContext which is semantically wider than inflights. 
   And even more, we treat RO and RW inflights differently. They have different lifecycle, ones closed on tx finish, others on broadcasting FinishedTransactionsBatchMessage. Within last option we actually do not use the context only the txId itself.
   All in all that means that design is inconsistent and should be refactored. Let's discuss this on tx sync.



##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1746,7 +1745,7 @@ private CompletableFuture<FuturesCleanupResult> awaitCleanupReadyFutures(UUID tx
 
             txOps.futures.clear();
 
-            return txOps;
+            return null;

Review Comment:
   Do we use cleanupReadyFutures in order to handle the race between cleanup and adding new operations? Meaning that: 
   
   - we need to await all pending operations prior to cleanup processing - all good here.
   - we need to discard new operation after cleanup was done - do we use cleanupReadyFutures to handle such race?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-18617 Clear rw tx context and cleanup ready futures on tx finish [ignite-3]

Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc merged PR #3436:
URL: https://github.com/apache/ignite-3/pull/3436


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-18617 Clear rw tx context and cleanup ready futures on tx finish [ignite-3]

Posted by "Cyrill (via GitHub)" <gi...@apache.org>.
Cyrill commented on code in PR #3436:
URL: https://github.com/apache/ignite-3/pull/3436#discussion_r1533370444


##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1746,7 +1745,7 @@ private CompletableFuture<FuturesCleanupResult> awaitCleanupReadyFutures(UUID tx
 
             txOps.futures.clear();
 
-            return txOps;
+            return null;

Review Comment:
   Yes we handle this case now - when a new operation comes in, in `compute` we first check the tx state.
   If it's final, we do not create a new one. If it's not final - we proceed as normal.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-18617 Clear rw tx context and cleanup ready futures on tx finish [ignite-3]

Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #3436:
URL: https://github.com/apache/ignite-3/pull/3436#discussion_r1532477285


##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1746,7 +1745,7 @@ private CompletableFuture<FuturesCleanupResult> awaitCleanupReadyFutures(UUID tx
 
             txOps.futures.clear();
 
-            return txOps;
+            return null;

Review Comment:
   Hmm, the question is still on the table.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-18617 Clear rw tx context and cleanup ready futures on tx finish [ignite-3]

Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #3436:
URL: https://github.com/apache/ignite-3/pull/3436#discussion_r1534036451


##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1894,18 +1893,23 @@ private <T> CompletableFuture<T> appendTxCommand(UUID txId, RequestType cmdType,
         var cleanupReadyFut = new CompletableFuture<Void>();
 
         txCleanupReadyFutures.compute(txId, (id, txOps) -> {
-            if (txOps == null) {
-                txOps = new TxCleanupReadyFutureList();
-            }
-
+            // First check whether the transaction has already been finished.
+            // And complete cleanupReadyFut with exception if it is the case.
             TxStateMeta txStateMeta = txManager.stateMeta(txId);
 
             if (txStateMeta == null || isFinalState(txStateMeta.txState())) {
                 cleanupReadyFut.completeExceptionally(new Exception());
-            } else {
-                txOps.futures.computeIfAbsent(cmdType, type -> new ArrayList<>()).add(cleanupReadyFut);
+
+                return txOps;

Review Comment:
   Why do we need to put txOps into map in case of exception?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-18617 Clear rw tx context and cleanup ready futures on tx finish [ignite-3]

Posted by "Cyrill (via GitHub)" <gi...@apache.org>.
Cyrill commented on code in PR #3436:
URL: https://github.com/apache/ignite-3/pull/3436#discussion_r1532188796


##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TransactionInflights.java:
##########
@@ -99,6 +99,15 @@ public void removeInflight(UUID txId) {
         tuple.onInflightsRemoved();
     }
 
+    /**
+     * Cleanup tx inflights on tx finish.
+     *
+     * @param txId Transaction id.
+     */
+    void clearInflights(UUID txId) {

Review Comment:
   I agree that the naming is utterly confusing.
   
   WDYT of renaming `TransactionInflights` to `TransactionContextManager` or `TransactionContexts`, `addInflight`/`removeInflight` will remain the same or we can rename them to `registerRequest`/`unregisterRequest`.
   Both existing `removeTxContexts` and `clearInflights` will become `removeTxContext(s)` or `clearTxContext(s)`.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org