You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by "ascherbakoff (via GitHub)" <gi...@apache.org> on 2023/08/30 16:20:06 UTC
[GitHub] [ignite-3] ascherbakoff opened a new pull request, #2520: IGNITE-20127
ascherbakoff opened a new pull request, #2520:
URL: https://github.com/apache/ignite-3/pull/2520
Implement delayed acks for tx commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340540654
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TxManagerImpl.java:
##########
@@ -202,21 +215,85 @@ public CompletableFuture<Void> finish(
) {
assert groups != null && !groups.isEmpty();
- HybridTimestamp commitTimestamp = commit ? clock.now() : null;
-
- TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
- .txId(txId)
- .timestampLong(clock.nowLong())
- .groupId(commitPartition)
- .groups(groups)
- .commit(commit)
- .commitTimestampLong(hybridTimestampToLong(commitTimestamp))
- .term(term)
- .build();
-
- return replicaService.invoke(recipientNode, req)
- // TODO: IGNITE-20033 TestOnly code, let's consider using Txn state map instead of states.
- .thenRun(() -> changeState(txId, PENDING, commit ? COMMITED : ABORTED));
+ Function<Void, CompletableFuture<Void>> clo = ignored -> {
+ HybridTimestamp commitTimestamp = commit ? clock.now() : null;
+
+ TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
+ .txId(txId)
+ .timestampLong(clock.nowLong())
+ .groupId(commitPartition)
+ .groups(groups)
+ .commit(commit)
+ .commitTimestampLong(hybridTimestampToLong(commitTimestamp))
+ .term(term)
+ .build();
+
+ return replicaService.invoke(recipientNode, req)
+ // TODO: IGNITE-20033 TestOnly code, let's consider using Txn state map instead of states.
+ .thenRun(() -> changeState(txId, PENDING, commit ? COMMITED : ABORTED));
+ };
+
+ if (!commit) {
+ AtomicReference<CompletableFuture<Void>> ref = new AtomicReference<>();
+ TxContext tuple = txCtxMap.compute(txId, (uuid, tuple0) -> {
+ if (tuple0 == null) {
+ tuple0 = new TxContext(); // No writes enlisted.
Review Comment:
Let's be consistent with the naming. enlistment and in-flight operations have different meaning. It's minor of course.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345625149
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1654,10 +1685,13 @@ private CompletableFuture<Object> processMultiEntryAction(ReadWriteMultiRowRepli
}
}
- CompletableFuture<Object> raftFut = rowIdsToDelete.isEmpty() ? completedFuture(null)
- : updateAllCommand(request, rowIdsToDelete).thenCompose(this::applyUpdateAllCommand);
+ if (rowIdsToDelete.isEmpty()) {
+ return completedFuture(new ReplicaResult(result, null));
Review Comment:
Not sure if this is actual after merging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345575025
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1396,42 +1407,61 @@ private <T> CompletableFuture<T> continueResolvingByPk(
* @param cmdType Command type.
* @param full {@code True} if a full transaction and can be immediately committed.
* @param op Operation closure.
- * @param <T> Type of execution result.
* @return A future object representing the result of the given operation.
*/
private <T> CompletableFuture<T> appendTxCommand(UUID txId, RequestType cmdType, boolean full, Supplier<CompletableFuture<T>> op) {
- var fut = new CompletableFuture<T>();
+ if (full) {
+ return op.get().whenComplete((v, th) -> {
+ // Fast unlock.
+ releaseTxLocks(txId);
+ });
+ }
- if (!full) {
- txCleanupReadyFutures.compute(txId, (id, txOps) -> {
- if (txOps == null) {
- txOps = new TxCleanupReadyFutureList();
- }
+ var cleanupReadyFut = new CompletableFuture<Void>();
Review Comment:
This future is used to prevent races between inflight ops and commit/abort, as it was before my patch. No need to change anything here.
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1810,10 +1845,19 @@ private CompletableFuture<Object> applyCmdWithExceptionHandling(Command cmd) {
* Executes an Update command.
*
* @param cmd Update command.
- * @return Raft future, see {@link #applyCmdWithExceptionHandling(Command)}.
+ * @return A local update ready future, possibly having a nested replication future as a result for delayed ack purpose.
*/
- private CompletableFuture<Object> applyUpdateCommand(UpdateCommand cmd) {
+ private CompletableFuture<CompletableFuture<?>> applyUpdateCommand(UpdateCommand cmd) {
if (!cmd.full()) {
+ CompletableFuture<UUID> fut = applyCmdWithExceptionHandling(cmd).thenApply(res -> {
+ // This check guaranties the result will never be lost. Currently always null.
+ assert res == null : "Replication result is lost";
+
+ // Set context for delayed response.
+ return cmd.txId();
+ });
+
+ // TODO error handling similar to replication group.
Review Comment:
No longer actual, removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345626150
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1853,36 +1902,68 @@ private CompletableFuture<Object> applyUpdateCommand(UpdateCommand cmd) {
* @param cmd UpdateAll command.
* @return Raft future, see {@link #applyCmdWithExceptionHandling(Command)}.
*/
- private CompletableFuture<Object> applyUpdateAllCommand(UpdateAllCommand cmd) {
+ private CompletableFuture<CompletableFuture<?>> applyUpdateAllCommand(UpdateAllCommand cmd, boolean skipDelayedAck) {
if (!cmd.full()) {
- storageUpdateHandler.handleUpdateAll(
- cmd.txId(),
- cmd.rowsToUpdate(),
- cmd.tablePartitionId().asTablePartitionId(),
- rowIds -> txsPendingRowIds.compute(cmd.txId(), (k, v) -> {
- if (v == null) {
- v = new TreeSet<>();
- }
+ if (skipDelayedAck) {
+ storageUpdateHandler.handleUpdateAll(
+ cmd.txId(),
+ cmd.rowsToUpdate(),
+ cmd.tablePartitionId().asTablePartitionId(),
+ rowIds -> txsPendingRowIds.compute(cmd.txId(), (k, v) -> {
+ if (v == null) {
+ v = new TreeSet<>();
+ }
- v.addAll(rowIds);
+ v.addAll(rowIds);
- return v;
- }),
- null);
- }
+ return v;
+ }),
+ null);
+
+ return applyCmdWithExceptionHandling(cmd).thenApply(res -> null);
+ } else {
+ CompletableFuture<Object> fut = applyCmdWithExceptionHandling(cmd).thenApply(res -> {
+ // Currently result is always null on a successfull execution of a replication command.
+ // This check guaranties the result will never be lost.
+ assert res == null : "Replication result is lost";
Review Comment:
But the method signature says otherwise. The result can appear later, so I've added some protection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345589457
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TxManagerImpl.java:
##########
@@ -179,16 +195,13 @@ public TxState state(UUID txId) {
@Override
public void changeState(UUID txId, TxState before, TxState after) {
- TxState computeResult = states.compute(txId, (k, v) -> {
+ states.compute(txId, (k, v) -> {
if (v == before) {
return after;
} else {
return v;
}
});
-
- assert computeResult == after : "Unable to change transaction state, expected = [" + before + "],"
Review Comment:
This assertion is no longer in the master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340382854
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1561,6 +1591,7 @@ private CompletableFuture<Object> processMultiEntryAction(ReadWriteMultiRowRepli
int i = 0;
+ // TODO sort keys ?
Review Comment:
Why?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345632736
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TxManagerImpl.java:
##########
@@ -202,21 +215,85 @@ public CompletableFuture<Void> finish(
) {
assert groups != null && !groups.isEmpty();
- HybridTimestamp commitTimestamp = commit ? clock.now() : null;
-
- TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
- .txId(txId)
- .timestampLong(clock.nowLong())
- .groupId(commitPartition)
- .groups(groups)
- .commit(commit)
- .commitTimestampLong(hybridTimestampToLong(commitTimestamp))
- .term(term)
- .build();
-
- return replicaService.invoke(recipientNode, req)
- // TODO: IGNITE-20033 TestOnly code, let's consider using Txn state map instead of states.
- .thenRun(() -> changeState(txId, PENDING, commit ? COMMITED : ABORTED));
+ Function<Void, CompletableFuture<Void>> clo = ignored -> {
+ HybridTimestamp commitTimestamp = commit ? clock.now() : null;
+
+ TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
+ .txId(txId)
+ .timestampLong(clock.nowLong())
+ .groupId(commitPartition)
+ .groups(groups)
+ .commit(commit)
+ .commitTimestampLong(hybridTimestampToLong(commitTimestamp))
+ .term(term)
+ .build();
+
+ return replicaService.invoke(recipientNode, req)
+ // TODO: IGNITE-20033 TestOnly code, let's consider using Txn state map instead of states.
+ .thenRun(() -> changeState(txId, PENDING, commit ? COMMITED : ABORTED));
+ };
+
+ if (!commit) {
+ AtomicReference<CompletableFuture<Void>> ref = new AtomicReference<>();
+ TxContext tuple = txCtxMap.compute(txId, (uuid, tuple0) -> {
+ if (tuple0 == null) {
+ tuple0 = new TxContext(); // No writes enlisted.
Review Comment:
The naming is correct here - inflighs are caused by enlisted writes.
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TxManagerImpl.java:
##########
@@ -202,21 +215,85 @@ public CompletableFuture<Void> finish(
) {
assert groups != null && !groups.isEmpty();
- HybridTimestamp commitTimestamp = commit ? clock.now() : null;
-
- TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
- .txId(txId)
- .timestampLong(clock.nowLong())
- .groupId(commitPartition)
- .groups(groups)
- .commit(commit)
- .commitTimestampLong(hybridTimestampToLong(commitTimestamp))
- .term(term)
- .build();
-
- return replicaService.invoke(recipientNode, req)
- // TODO: IGNITE-20033 TestOnly code, let's consider using Txn state map instead of states.
- .thenRun(() -> changeState(txId, PENDING, commit ? COMMITED : ABORTED));
+ Function<Void, CompletableFuture<Void>> clo = ignored -> {
+ HybridTimestamp commitTimestamp = commit ? clock.now() : null;
+
+ TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
+ .txId(txId)
+ .timestampLong(clock.nowLong())
+ .groupId(commitPartition)
+ .groups(groups)
+ .commit(commit)
+ .commitTimestampLong(hybridTimestampToLong(commitTimestamp))
+ .term(term)
+ .build();
+
+ return replicaService.invoke(recipientNode, req)
+ // TODO: IGNITE-20033 TestOnly code, let's consider using Txn state map instead of states.
+ .thenRun(() -> changeState(txId, PENDING, commit ? COMMITED : ABORTED));
+ };
+
+ if (!commit) {
+ AtomicReference<CompletableFuture<Void>> ref = new AtomicReference<>();
+ TxContext tuple = txCtxMap.compute(txId, (uuid, tuple0) -> {
+ if (tuple0 == null) {
+ tuple0 = new TxContext(); // No writes enlisted.
+ }
+
+ if (tuple0.finishFut == null) {
+ tuple0.finishFut = new CompletableFuture<>();
+ ref.set(tuple0.finishFut);
+ }
+
+ return tuple0;
+ });
+
+ if (ref.get() != null) { // This is aborting thread.
+ clo.apply(null).handle((ignored, err) -> {
+ if (err == null) {
+ tuple.finishFut.complete(null);
+ } else {
+ tuple.finishFut.completeExceptionally(err);
+ }
+ return null;
+ });
+ }
+
+ return tuple.finishFut;
+ }
+
+ // Wait for commit acks first, then proceed with the finish request.
+ AtomicReference<CompletableFuture<Void>> ref = new AtomicReference<>();
+ TxContext tuple = txCtxMap.compute(txId, (uuid, tuple0) -> {
+ if (tuple0 == null) {
+ tuple0 = new TxContext(); // No writes enlisted.
+ }
+
+ if (tuple0.finishFut == null) {
+ tuple0.finishFut = new CompletableFuture<>();
+ ref.set(tuple0.finishFut);
+ }
+
+ return tuple0;
+ });
+
+ if (ref.get() != null) { // This is committing thread.
+ // All inflights have been completed before the finish.
+ if (tuple.inflights == 0) {
+ tuple.waitRepFut.complete(null);
+ }
+
+ return tuple.waitRepFut.thenCompose(clo).handle((ignored, err) -> {
Review Comment:
No we do not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340552476
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TxManagerImpl.java:
##########
@@ -202,21 +215,85 @@ public CompletableFuture<Void> finish(
) {
assert groups != null && !groups.isEmpty();
- HybridTimestamp commitTimestamp = commit ? clock.now() : null;
-
- TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
- .txId(txId)
- .timestampLong(clock.nowLong())
- .groupId(commitPartition)
- .groups(groups)
- .commit(commit)
- .commitTimestampLong(hybridTimestampToLong(commitTimestamp))
- .term(term)
- .build();
-
- return replicaService.invoke(recipientNode, req)
- // TODO: IGNITE-20033 TestOnly code, let's consider using Txn state map instead of states.
- .thenRun(() -> changeState(txId, PENDING, commit ? COMMITED : ABORTED));
+ Function<Void, CompletableFuture<Void>> clo = ignored -> {
+ HybridTimestamp commitTimestamp = commit ? clock.now() : null;
+
+ TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
+ .txId(txId)
+ .timestampLong(clock.nowLong())
+ .groupId(commitPartition)
+ .groups(groups)
+ .commit(commit)
+ .commitTimestampLong(hybridTimestampToLong(commitTimestamp))
+ .term(term)
+ .build();
+
+ return replicaService.invoke(recipientNode, req)
+ // TODO: IGNITE-20033 TestOnly code, let's consider using Txn state map instead of states.
+ .thenRun(() -> changeState(txId, PENDING, commit ? COMMITED : ABORTED));
+ };
+
+ if (!commit) {
+ AtomicReference<CompletableFuture<Void>> ref = new AtomicReference<>();
+ TxContext tuple = txCtxMap.compute(txId, (uuid, tuple0) -> {
+ if (tuple0 == null) {
+ tuple0 = new TxContext(); // No writes enlisted.
+ }
+
+ if (tuple0.finishFut == null) {
+ tuple0.finishFut = new CompletableFuture<>();
+ ref.set(tuple0.finishFut);
+ }
+
+ return tuple0;
+ });
+
+ if (ref.get() != null) { // This is aborting thread.
+ clo.apply(null).handle((ignored, err) -> {
+ if (err == null) {
+ tuple.finishFut.complete(null);
+ } else {
+ tuple.finishFut.completeExceptionally(err);
+ }
+ return null;
+ });
+ }
+
+ return tuple.finishFut;
+ }
+
+ // Wait for commit acks first, then proceed with the finish request.
+ AtomicReference<CompletableFuture<Void>> ref = new AtomicReference<>();
+ TxContext tuple = txCtxMap.compute(txId, (uuid, tuple0) -> {
+ if (tuple0 == null) {
+ tuple0 = new TxContext(); // No writes enlisted.
+ }
+
+ if (tuple0.finishFut == null) {
+ tuple0.finishFut = new CompletableFuture<>();
+ ref.set(tuple0.finishFut);
+ }
+
+ return tuple0;
+ });
+
+ if (ref.get() != null) { // This is committing thread.
+ // All inflights have been completed before the finish.
+ if (tuple.inflights == 0) {
+ tuple.waitRepFut.complete(null);
+ }
+
+ return tuple.waitRepFut.thenCompose(clo).handle((ignored, err) -> {
Review Comment:
In order to be on the same page. We do wait for all in-flights to complete even in case of rollback, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340203900
##########
modules/network-api/src/main/java/org/apache/ignite/network/MessagingService.java:
##########
@@ -42,6 +42,7 @@ public interface MessagingService {
* <li>Messages send to same receiver will be delivered in the same order as they were sent;</li>
* <li>If a message N has been successfully delivered to a member implies that all messages to same receiver
* preceding N have also been successfully delivered.</li>
+ * <li>A message is delivered only once</li>
Review Comment:
Full stop is missing in the end.
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -261,6 +261,35 @@ private void onReplicaMessageReceived(NetworkMessage message, String senderConsi
}
}
+ if (res.repFuture() != null) {
+ assert request instanceof PrimaryReplicaRequest;
+
+ res.repFuture().handle((res0, ex0) -> {
+ NetworkMessage msg0;
+
+ LOG.debug("Sending delayed response for replica request [request={}]", request);
+
+ if (ex == null) {
+ msg0 = prepareReplicaResponse(finalSendTimestamp, res0);
Review Comment:
What kind of result you expect in repFut (res0). I believe that it is void in any case because it basically tells us that replication is finished or throws an exception.
##########
modules/network/src/main/java/org/apache/ignite/network/DefaultMessagingService.java:
##########
@@ -115,6 +115,8 @@ public DefaultMessagingService(
this.marshaller = marshaller;
this.outboundExecutor = Executors.newSingleThreadExecutor(NamedThreadFactory.create(nodeName, "MessagingService-outbound-", LOG));
+ // TODO asch the implementation of delayed acks relies on absence of reordering on subsequent messages delivery.
Review Comment:
Why?
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -261,6 +261,35 @@ private void onReplicaMessageReceived(NetworkMessage message, String senderConsi
}
}
+ if (res.repFuture() != null) {
+ assert request instanceof PrimaryReplicaRequest;
+
+ res.repFuture().handle((res0, ex0) -> {
+ NetworkMessage msg0;
+
+ LOG.debug("Sending delayed response for replica request [request={}]", request);
+
+ if (ex == null) {
+ msg0 = prepareReplicaResponse(finalSendTimestamp, res0);
+ } else {
+ LOG.warn("Failed to process delayed response [request={}]", ex, request);
+
+ msg0 = prepareReplicaErrorResponse(finalSendTimestamp, ex);
Review Comment:
I don't think that it's valid to use same finalSendTimestamp with both fromPrimary and delayed responses.
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/UpdatableTableImpl.java:
##########
@@ -176,6 +177,7 @@ public <RowT> CompletableFuture<?> upsertAll(
.term(nodeWithTerm.term())
.requestType(RequestType.RW_UPSERT_ALL)
.timestampLong(clock.nowLong())
+ .skipDelayedAck(true)
Review Comment:
What does it mean and why do we skip delayed ack for _ALL operations?
##########
modules/table/src/testFixtures/java/org/apache/ignite/internal/table/TxAbstractTest.java:
##########
@@ -1541,7 +1580,7 @@ public void run() {
}
while (!stop.get() && firstErr.get() == null) {
- InternalTransaction tx = txManager(accounts).begin();
+ InternalTransaction tx = clientTxManager().begin();
Review Comment:
Is an intended change? I mean that, despite the fact that we still have txManager() we want use client ones, right?
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -326,12 +331,18 @@ public PartitionReplicaListener(
}
@Override
- public CompletableFuture<?> invoke(ReplicaRequest request) {
+ public CompletableFuture<ReplicaResult> invoke(ReplicaRequest request) {
if (request instanceof TxStateReplicaRequest) {
- return processTxStateReplicaRequest((TxStateReplicaRequest) request);
+ return processTxStateReplicaRequest((TxStateReplicaRequest) request).thenApply(res -> new ReplicaResult(res, null));
}
- return ensureReplicaIsPrimary(request).thenCompose(isPrimary -> processRequest(request, isPrimary));
+ return ensureReplicaIsPrimary(request).thenCompose(isPrimary -> processRequest(request, isPrimary).thenApply(res -> {
+ if (res instanceof ReplicaResult) {
+ return (ReplicaResult) res;
+ } else {
+ return new ReplicaResult(res, null);
Review Comment:
I'd rather introduce overloaded version with one param.
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -261,6 +261,35 @@ private void onReplicaMessageReceived(NetworkMessage message, String senderConsi
}
}
+ if (res.repFuture() != null) {
Review Comment:
How do we handle single noded raft groups? I didn't check the code below, maybe it's already implemented. But for now, my questions/expectations are following:
- In case of single noded raft group and if primary is collocated with that single node, we should skip primary storage updates, and just send the command for the replication, synchronously await the result and return the future with completed res.repFuture and thus send single response to the client side.
- Otherwise use common logic with two responses.
- In all cases we should check whether repFuture is completed when we are ready to send response from the primary and send only one response if it is.
BTW if items 1 and 2 aren't implemented yet, I'd rather do it in a separate ticket.
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -261,6 +261,35 @@ private void onReplicaMessageReceived(NetworkMessage message, String senderConsi
}
}
+ if (res.repFuture() != null) {
+ assert request instanceof PrimaryReplicaRequest;
+
+ res.repFuture().handle((res0, ex0) -> {
+ NetworkMessage msg0;
+
+ LOG.debug("Sending delayed response for replica request [request={}]", request);
+
+ if (ex == null) {
+ msg0 = prepareReplicaResponse(finalSendTimestamp, res0);
+ } else {
+ LOG.warn("Failed to process delayed response [request={}]", ex, request);
+
+ msg0 = prepareReplicaErrorResponse(finalSendTimestamp, ex);
+ }
+
+ ClusterNode sender = clusterNetSvc.topologyService().getByConsistentId(senderConsistentId);
Review Comment:
I'd rather add overloaded version of send with recipientConsistentId as we discussed with Ivan and just skip the clusterNodeResolver part.
We already have one with channelType param
`CompletableFuture<Void> send(String recipientConsistentId, ChannelType channelType, NetworkMessage msg);`
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/UpdatableTableImpl.java:
##########
@@ -237,14 +239,15 @@ public <RowT> CompletableFuture<?> insertAll(
TablePartitionId partGroupId = new TablePartitionId(tableId, partToRows.getIntKey());
NodeWithTerm nodeWithTerm = ectx.description().mapping().updatingTableAssignments().get(partToRows.getIntKey());
- ReplicaRequest request = MESSAGES_FACTORY.readWriteMultiRowReplicaRequest()
+ ReadWriteMultiRowReplicaRequest request = MESSAGES_FACTORY.readWriteMultiRowReplicaRequest()
Review Comment:
Why?
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -261,6 +261,35 @@ private void onReplicaMessageReceived(NetworkMessage message, String senderConsi
}
}
+ if (res.repFuture() != null) {
+ assert request instanceof PrimaryReplicaRequest;
+
+ res.repFuture().handle((res0, ex0) -> {
+ NetworkMessage msg0;
+
+ LOG.debug("Sending delayed response for replica request [request={}]", request);
+
+ if (ex == null) {
+ msg0 = prepareReplicaResponse(finalSendTimestamp, res0);
+ } else {
+ LOG.warn("Failed to process delayed response [request={}]", ex, request);
+
+ msg0 = prepareReplicaErrorResponse(finalSendTimestamp, ex);
Review Comment:
As was mentioned above, it's either void or exception, no need to send the result one more time.
##########
modules/table/src/integrationTest/java/org/apache/ignite/distributed/ItTxDistributedTestSingleNode.java:
##########
@@ -365,6 +370,7 @@ public void before() throws Exception {
if (startClient()) {
clientTxManager = new TxManagerImpl(clientReplicaSvc, new HeapLockManager(), clientClock, new TransactionIdGenerator(-1));
+ clientTxManager.start();
Review Comment:
Do we also call stop()? BTW I believe that we do in main, so it worth merging it first.
##########
modules/table/src/test/java/org/apache/ignite/internal/table/distributed/replication/PartitionReplicaListenerTest.java:
##########
@@ -1167,9 +1168,9 @@ private void testWriteIntentOnPrimaryReplica(
raftClientFutureClosure = cmd -> writeFut;
try {
- CompletableFuture<?> replicaWriteFut = partitionReplicaListener.invoke(updatingRequestSupplier.get());
+ CompletableFuture<ReplicaResult> replicaWriteFut = partitionReplicaListener.invoke(updatingRequestSupplier.get());
- assertFalse(replicaWriteFut.isDone());
+ assertTrue(replicaWriteFut.isDone());
Review Comment:
Why?
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaResult.java:
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.replicator;
+
+import java.util.concurrent.CompletableFuture;
+import org.jetbrains.annotations.Nullable;
+
+/**
+ * Represents replica execution result.
+ */
+public class ReplicaResult {
Review Comment:
According to the comments above, I don't think that ReplicaResult should have given structure.
##########
modules/table/src/integrationTest/java/org/apache/ignite/distributed/ItTxDistributedTestThreeNodesThreeReplicas.java:
##########
@@ -59,4 +69,36 @@ public void after() throws Exception {
super.after();
}
}
+
+ @Test
+ public void testPrimaryReplicaDirectUpdateForExplicitTxn() throws InterruptedException {
+ Peer leader = accRaftClients.get(0).leader();
+ JraftServerImpl server = (JraftServerImpl) raftServers.get(leader.consistentId()).server();
+ var groupId = new TablePartitionId(accounts.tableId(), 0);
+
+ // BLock replication messages to both replicas.
+ server.blockMessages(new RaftNodeId(groupId, leader), (msg, peerId) -> {
+ if (msg instanceof RpcRequests.AppendEntriesRequest) {
+ RpcRequests.AppendEntriesRequest tmp = (AppendEntriesRequest) msg;
+
+ if (tmp.entriesList() != null && !tmp.entriesList().isEmpty()) {
+ return true;
+ }
+ }
+ return false;
+ });
+
+ assertTrue(IgniteTestUtils.waitForCondition(() -> server.blockedMessages(new RaftNodeId(groupId, leader)).size() == 2, 10000),
+ "Failed to wait for blocked messages");
+
+ ReadWriteTransactionImpl tx = (ReadWriteTransactionImpl) igniteTransactions.begin();
+ CompletableFuture<Void> fut = accounts.recordView().upsertAsync(tx, makeValue(1, 100.));
+ // Update must complete now despite the blocked replication protocol.
+ assertTrue(IgniteTestUtils.waitForCondition(fut::isDone, 5_000), "The update future is not completed within timeout");
+
Review Comment:
Do you have an idea of how to check that transaction won't finish until unblocking?
##########
modules/table/src/integrationTest/java/org/apache/ignite/distributed/ItTxDistributedTestThreeNodesThreeReplicas.java:
##########
@@ -59,4 +69,36 @@ public void after() throws Exception {
super.after();
}
}
+
+ @Test
+ public void testPrimaryReplicaDirectUpdateForExplicitTxn() throws InterruptedException {
+ Peer leader = accRaftClients.get(0).leader();
Review Comment:
It's a bit unreliable. client cache leader, so that actual leader may be on another node. The chance isn't that big, however it's extremely difficult to debug such failures. So I believe that we can ensure that leader is actually the one that we've blocked after the block and either fail the test with meaningful message in order to ease the analysis or even re-block with true leader in a while loop until success. It's also required to check whether the leader was changed or not during the test. Crappy stuff.
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/LockManager.java:
##########
@@ -33,7 +33,6 @@ public interface LockManager {
* @param lockKey The key.
* @param lockMode Lock mode, for example shared, exclusive, intention-shared etc.
* @return The future with gained lock that will be completed when a lock is successfully acquired.
- * @throws LockException When a lock can't be taken due to possible deadlock.
Review Comment:
Why?
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -261,6 +261,37 @@ private void onReplicaMessageReceived(NetworkMessage message, String senderConsi
}
}
+ if (res.repFuture() != null) {
+ assert request instanceof PrimaryReplicaRequest;
+
+ LOG.info("Procesing delayed response for replica request [request={}]", request);
+
+ res.repFuture().handle((res0, ex0) -> {
+ NetworkMessage msg0;
+
+ LOG.info("Sending delayed response for replica request [request={}]", request);
+
+ if (ex == null) {
+ msg0 = prepareReplicaResponse(finalSendTimestamp, res0);
+ } else {
+ LOG.warn("Failed to process delayed response [request={}]", ex, request);
+
+ msg0 = prepareReplicaErrorResponse(finalSendTimestamp, ex);
+ }
+
+ ClusterNode sender = clusterNetSvc.topologyService().getByConsistentId(senderConsistentId);
+
+ if (sender != null) {
+ // Using strong send here is important to avoid a reordering with a normal response.
+ clusterNetSvc.messagingService().send(sender, msg0);
Review Comment:
Agree. I'd rather add operation id or similar same for both fromPrimary and delayed responses in order to match each other. Such ids should be unique within transaction, so basically it's operation counter or similar. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345616762
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaResult.java:
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.replicator;
+
+import java.util.concurrent.CompletableFuture;
+import org.jetbrains.annotations.Nullable;
+
+/**
+ * Represents replica execution result.
+ */
+public class ReplicaResult {
Review Comment:
I don't understand this. Which structure are you talking about ? ReplicaResult represent immediate and delayed results, as I previously replied.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340413890
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1853,36 +1902,68 @@ private CompletableFuture<Object> applyUpdateCommand(UpdateCommand cmd) {
* @param cmd UpdateAll command.
* @return Raft future, see {@link #applyCmdWithExceptionHandling(Command)}.
*/
- private CompletableFuture<Object> applyUpdateAllCommand(UpdateAllCommand cmd) {
+ private CompletableFuture<CompletableFuture<?>> applyUpdateAllCommand(UpdateAllCommand cmd, boolean skipDelayedAck) {
if (!cmd.full()) {
- storageUpdateHandler.handleUpdateAll(
- cmd.txId(),
- cmd.rowsToUpdate(),
- cmd.tablePartitionId().asTablePartitionId(),
- rowIds -> txsPendingRowIds.compute(cmd.txId(), (k, v) -> {
- if (v == null) {
- v = new TreeSet<>();
- }
+ if (skipDelayedAck) {
+ storageUpdateHandler.handleUpdateAll(
+ cmd.txId(),
+ cmd.rowsToUpdate(),
+ cmd.tablePartitionId().asTablePartitionId(),
+ rowIds -> txsPendingRowIds.compute(cmd.txId(), (k, v) -> {
+ if (v == null) {
+ v = new TreeSet<>();
+ }
- v.addAll(rowIds);
+ v.addAll(rowIds);
- return v;
- }),
- null);
- }
+ return v;
+ }),
+ null);
+
+ return applyCmdWithExceptionHandling(cmd).thenApply(res -> null);
+ } else {
+ CompletableFuture<Object> fut = applyCmdWithExceptionHandling(cmd).thenApply(res -> {
+ // Currently result is always null on a successfull execution of a replication command.
+ // This check guaranties the result will never be lost.
+ assert res == null : "Replication result is lost";
Review Comment:
Well, from my point of view it will always be null, because basically it's void.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345578543
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1810,10 +1845,19 @@ private CompletableFuture<Object> applyCmdWithExceptionHandling(Command cmd) {
* Executes an Update command.
*
* @param cmd Update command.
- * @return Raft future, see {@link #applyCmdWithExceptionHandling(Command)}.
+ * @return A local update ready future, possibly having a nested replication future as a result for delayed ack purpose.
*/
- private CompletableFuture<Object> applyUpdateCommand(UpdateCommand cmd) {
+ private CompletableFuture<CompletableFuture<?>> applyUpdateCommand(UpdateCommand cmd) {
Review Comment:
This signature is required to implement correct chaining both for implicit and explicit tx flows.
The result of a call either the immediate result with a delayed replication future, or immediate replication future.
Actually this is the simplest way to implement it.
Can you propose simpler variant, which is also correct ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345605486
##########
modules/network/src/main/java/org/apache/ignite/network/DefaultMessagingService.java:
##########
@@ -115,6 +115,8 @@ public DefaultMessagingService(
this.marshaller = marshaller;
this.outboundExecutor = Executors.newSingleThreadExecutor(NamedThreadFactory.create(nodeName, "MessagingService-outbound-", LOG));
+ // TODO asch the implementation of delayed acks relies on absence of reordering on subsequent messages delivery.
Review Comment:
Because if it is not held the commit can be called with some write operation in progress.
But we need to enforce the following invariant on commit - all inflight operations are finished and all corresponding replication is completed.
Actually, looks like we can relax this invariant to "all corresponding replication is completed", because it implies that all updates on primary replica is also applied.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345631832
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/storage/InternalTableImpl.java:
##########
@@ -486,13 +476,54 @@ private <R> CompletableFuture<R> enlistWithRetry(
return result;
}
+ private <R> CompletableFuture<R> trackingInvoke(
+ InternalTransaction tx,
+ int partId,
+ Function<Long, ReplicaRequest> mapFunc,
+ boolean full,
+ IgniteBiTuple<ClusterNode, Long> primaryReplicaAndTerm,
+ @Nullable BiPredicate<R, ReplicaRequest> noOpChecker
+ ) {
+ ReplicaRequest request = mapFunc.apply(primaryReplicaAndTerm.get2());
+
+ boolean write = request instanceof SingleRowReplicaRequest && ((SingleRowReplicaRequest) request).requestType() != RW_GET
+ || request instanceof MultipleRowReplicaRequest && ((MultipleRowReplicaRequest) request).requestType() != RW_GET_ALL;
+
+ if (write && !full) {
+ // Track only write requests from explicit transactions.
Review Comment:
Tracking is required only for wait replication on commit purposes.
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/storage/InternalTableImpl.java:
##########
@@ -486,13 +476,54 @@ private <R> CompletableFuture<R> enlistWithRetry(
return result;
}
+ private <R> CompletableFuture<R> trackingInvoke(
+ InternalTransaction tx,
+ int partId,
+ Function<Long, ReplicaRequest> mapFunc,
+ boolean full,
+ IgniteBiTuple<ClusterNode, Long> primaryReplicaAndTerm,
+ @Nullable BiPredicate<R, ReplicaRequest> noOpChecker
+ ) {
+ ReplicaRequest request = mapFunc.apply(primaryReplicaAndTerm.get2());
+
+ boolean write = request instanceof SingleRowReplicaRequest && ((SingleRowReplicaRequest) request).requestType() != RW_GET
+ || request instanceof MultipleRowReplicaRequest && ((MultipleRowReplicaRequest) request).requestType() != RW_GET_ALL;
+
+ if (write && !full) {
+ // Track only write requests from explicit transactions.
+ if (!txManager.addInflight(tx.id())) {
+ return failedFuture(
+ new TransactionException(TX_UNEXPECTED_STATE_ERR, IgniteStringFormatter.format(
+ "Failed to enlist a write operation into a transaction, tx is locked for updates "
+ + "[tableName={}, partId={}, txState={}]",
+ tableName,
+ partId,
+ tx.state()
+ )));
+ }
+
+ return replicaSvc.<R>invoke(primaryReplicaAndTerm.get1(), request).thenApply(res -> {
+ assert noOpChecker != null;
+
+ // Remove inflight if no replication was scheduled, otherwise inflight will be removed by delayed response.
+ if (noOpChecker.test(res, request)) {
Review Comment:
Renamed to noWriteCheck
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345622165
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/LockManager.java:
##########
@@ -33,7 +33,6 @@ public interface LockManager {
* @param lockKey The key.
* @param lockMode Lock mode, for example shared, exclusive, intention-shared etc.
* @return The future with gained lock that will be completed when a lock is successfully acquired.
- * @throws LockException When a lock can't be taken due to possible deadlock.
Review Comment:
Because this exception is never thrown.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340385206
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1615,12 +1646,12 @@ private CompletableFuture<Object> processMultiEntryAction(ReadWriteMultiRowRepli
}
if (rowIdsToDelete.isEmpty()) {
- return completedFuture(result);
+ return completedFuture(new ReplicaResult(result, null));
}
return updateAllCommand(request, rowIdsToDelete)
- .thenCompose(this::applyUpdateAllCommand)
- .thenApply(ignored -> result);
+ .thenCompose(cmd -> applyUpdateAllCommand(cmd, request.skipDelayedAck()))
Review Comment:
Same as above. Feel free to answer in one topic only. Why we skipDelayedAck in case of _ALL?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340532400
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/storage/InternalTableImpl.java:
##########
@@ -486,13 +476,54 @@ private <R> CompletableFuture<R> enlistWithRetry(
return result;
}
+ private <R> CompletableFuture<R> trackingInvoke(
+ InternalTransaction tx,
+ int partId,
+ Function<Long, ReplicaRequest> mapFunc,
+ boolean full,
+ IgniteBiTuple<ClusterNode, Long> primaryReplicaAndTerm,
+ @Nullable BiPredicate<R, ReplicaRequest> noOpChecker
+ ) {
+ ReplicaRequest request = mapFunc.apply(primaryReplicaAndTerm.get2());
+
+ boolean write = request instanceof SingleRowReplicaRequest && ((SingleRowReplicaRequest) request).requestType() != RW_GET
+ || request instanceof MultipleRowReplicaRequest && ((MultipleRowReplicaRequest) request).requestType() != RW_GET_ALL;
+
+ if (write && !full) {
+ // Track only write requests from explicit transactions.
+ if (!txManager.addInflight(tx.id())) {
+ return failedFuture(
+ new TransactionException(TX_UNEXPECTED_STATE_ERR, IgniteStringFormatter.format(
+ "Failed to enlist a write operation into a transaction, tx is locked for updates "
+ + "[tableName={}, partId={}, txState={}]",
+ tableName,
+ partId,
+ tx.state()
+ )));
+ }
+
+ return replicaSvc.<R>invoke(primaryReplicaAndTerm.get1(), request).thenApply(res -> {
+ assert noOpChecker != null;
+
+ // Remove inflight if no replication was scheduled, otherwise inflight will be removed by delayed response.
+ if (noOpChecker.test(res, request)) {
Review Comment:
noOpCheck naming seems confusing to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345620324
##########
modules/table/src/integrationTest/java/org/apache/ignite/distributed/ItTxDistributedTestThreeNodesThreeReplicas.java:
##########
@@ -59,4 +69,36 @@ public void after() throws Exception {
super.after();
}
}
+
+ @Test
+ public void testPrimaryReplicaDirectUpdateForExplicitTxn() throws InterruptedException {
+ Peer leader = accRaftClients.get(0).leader();
Review Comment:
You are complicating things. Leader should not change during the test. There are many tests relaying on a stable leader.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345618083
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/UpdatableTableImpl.java:
##########
@@ -176,6 +177,7 @@ public <RowT> CompletableFuture<?> upsertAll(
.term(nodeWithTerm.term())
.requestType(RequestType.RW_UPSERT_ALL)
.timestampLong(clock.nowLong())
+ .skipDelayedAck(true)
Review Comment:
SQL doesn't support this optimization yet. Without the flag, all sql txns will hang forever.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345585730
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TxManagerImpl.java:
##########
@@ -202,21 +215,85 @@ public CompletableFuture<Void> finish(
) {
assert groups != null && !groups.isEmpty();
- HybridTimestamp commitTimestamp = commit ? clock.now() : null;
-
- TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
- .txId(txId)
- .timestampLong(clock.nowLong())
- .groupId(commitPartition)
- .groups(groups)
- .commit(commit)
- .commitTimestampLong(hybridTimestampToLong(commitTimestamp))
- .term(term)
- .build();
-
- return replicaService.invoke(recipientNode, req)
- // TODO: IGNITE-20033 TestOnly code, let's consider using Txn state map instead of states.
- .thenRun(() -> changeState(txId, PENDING, commit ? COMMITED : ABORTED));
+ Function<Void, CompletableFuture<Void>> clo = ignored -> {
Review Comment:
Because Supplier can not be used in completableFuture.thenCompose
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 Implement delayed replication acks [ignite-3]
Posted by "asfgit (via GitHub)" <gi...@apache.org>.
asfgit closed pull request #2520: IGNITE-20127 Implement delayed replication acks
URL: https://github.com/apache/ignite-3/pull/2520
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340535960
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TxManagerImpl.java:
##########
@@ -179,16 +195,13 @@ public TxState state(UUID txId) {
@Override
public void changeState(UUID txId, TxState before, TxState after) {
- TxState computeResult = states.compute(txId, (k, v) -> {
+ states.compute(txId, (k, v) -> {
if (v == before) {
return after;
} else {
return v;
}
});
-
- assert computeResult == after : "Unable to change transaction state, expected = [" + before + "],"
Review Comment:
Agree, assert seems useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340532705
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/storage/InternalTableImpl.java:
##########
@@ -509,16 +540,16 @@ private <T> CompletableFuture<T> postEnlist(CompletableFuture<T> fut, boolean au
if (e != null) {
RuntimeException e0 = wrapReplicationException(e);
+ //LOGGER.info("DBG: before auto rollback {}", tx0.id());
Review Comment:
Please remove, here and below.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345623672
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -326,12 +331,18 @@ public PartitionReplicaListener(
}
@Override
- public CompletableFuture<?> invoke(ReplicaRequest request) {
+ public CompletableFuture<ReplicaResult> invoke(ReplicaRequest request) {
if (request instanceof TxStateReplicaRequest) {
- return processTxStateReplicaRequest((TxStateReplicaRequest) request);
+ return processTxStateReplicaRequest((TxStateReplicaRequest) request).thenApply(res -> new ReplicaResult(res, null));
}
- return ensureReplicaIsPrimary(request).thenCompose(isPrimary -> processRequest(request, isPrimary));
+ return ensureReplicaIsPrimary(request).thenCompose(isPrimary -> processRequest(request, isPrimary).thenApply(res -> {
+ if (res instanceof ReplicaResult) {
+ return (ReplicaResult) res;
+ } else {
+ return new ReplicaResult(res, null);
Review Comment:
This will not work, because ReplicaResult represents two results. See my previous comment about it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345624309
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1561,6 +1591,7 @@ private CompletableFuture<Object> processMultiEntryAction(ReadWriteMultiRowRepli
int i = 0;
+ // TODO sort keys ?
Review Comment:
Not actual
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345588367
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TxManagerImpl.java:
##########
@@ -57,7 +65,12 @@
*
* <p>Uses 2PC for atomic commitment and 2PL for concurrency control.
*/
-public class TxManagerImpl implements TxManager {
+public class TxManagerImpl implements TxManager, NetworkMessageHandler {
Review Comment:
Here org.apache.ignite.internal.tx.impl.TxManagerImpl#start
Inflight calculation is required for safe committing a transaction, see ticket description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345615616
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -261,6 +261,35 @@ private void onReplicaMessageReceived(NetworkMessage message, String senderConsi
}
}
+ if (res.repFuture() != null) {
+ assert request instanceof PrimaryReplicaRequest;
+
+ res.repFuture().handle((res0, ex0) -> {
+ NetworkMessage msg0;
+
+ LOG.debug("Sending delayed response for replica request [request={}]", request);
+
+ if (ex == null) {
+ msg0 = prepareReplicaResponse(finalSendTimestamp, res0);
+ } else {
+ LOG.warn("Failed to process delayed response [request={}]", ex, request);
+
+ msg0 = prepareReplicaErrorResponse(finalSendTimestamp, ex);
Review Comment:
It doesn't matter here and can not break anything.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345621116
##########
modules/table/src/testFixtures/java/org/apache/ignite/internal/table/TxAbstractTest.java:
##########
@@ -1541,7 +1580,7 @@ public void run() {
}
while (!stop.get() && firstErr.get() == null) {
- InternalTransaction tx = txManager(accounts).begin();
+ InternalTransaction tx = clientTxManager().begin();
Review Comment:
Yes it is. Previously a transaction has been started using server txManager, instead of an client.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345608125
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -261,6 +261,35 @@ private void onReplicaMessageReceived(NetworkMessage message, String senderConsi
}
}
+ if (res.repFuture() != null) {
Review Comment:
No special handling required, it just works as is. But you are correct - it may be additionally optimized. I suggest a separate ticket.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340405353
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1810,10 +1845,19 @@ private CompletableFuture<Object> applyCmdWithExceptionHandling(Command cmd) {
* Executes an Update command.
*
* @param cmd Update command.
- * @return Raft future, see {@link #applyCmdWithExceptionHandling(Command)}.
+ * @return A local update ready future, possibly having a nested replication future as a result for delayed ack purpose.
*/
- private CompletableFuture<Object> applyUpdateCommand(UpdateCommand cmd) {
+ private CompletableFuture<CompletableFuture<?>> applyUpdateCommand(UpdateCommand cmd) {
Review Comment:
Agree. Proposed API is overcomplicated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340519809
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/TxManager.java:
##########
@@ -149,4 +149,19 @@ CompletableFuture<Void> cleanup(
* @return Future of all read-only transactions with read timestamp less or equals the given new low watermark.
*/
CompletableFuture<Void> updateLowWatermark(HybridTimestamp newLowWatermark);
+
+ /**
+ * Registers the infligh update for a transaction.
+ *
+ * @param tx The transaction id.
Review Comment:
txId not tx, but as mentioned below probably not relevant because I believe we should move given method to InternalTransaction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340521828
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/storage/InternalTableImpl.java:
##########
@@ -434,41 +437,28 @@ private CompletableFuture<Collection<BinaryRow>> enlistCursorInTx(
* @param partId Partition number.
* @param mapFunc Function to create replica request with new raft term.
* @param attempts Number of attempts.
+ * @param full {@code True} if is a full transaction.
+ * @param noOpChecker Used to handle no-op operations (producing no updates).
* @return The future.
*/
private <R> CompletableFuture<R> enlistWithRetry(
InternalTransaction tx,
int partId,
Function<Long, ReplicaRequest> mapFunc,
- int attempts
+ int attempts,
+ boolean full,
+ @Nullable BiPredicate<R, ReplicaRequest> noOpChecker
) {
+ // LOGGER.info("enlistWithRetry {} {}", partId, tx.id().toString());
Review Comment:
Please remove.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340381326
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1396,42 +1407,61 @@ private <T> CompletableFuture<T> continueResolvingByPk(
* @param cmdType Command type.
* @param full {@code True} if a full transaction and can be immediately committed.
* @param op Operation closure.
- * @param <T> Type of execution result.
* @return A future object representing the result of the given operation.
*/
private <T> CompletableFuture<T> appendTxCommand(UUID txId, RequestType cmdType, boolean full, Supplier<CompletableFuture<T>> op) {
- var fut = new CompletableFuture<T>();
+ if (full) {
+ return op.get().whenComplete((v, th) -> {
+ // Fast unlock.
+ releaseTxLocks(txId);
+ });
+ }
- if (!full) {
- txCleanupReadyFutures.compute(txId, (id, txOps) -> {
- if (txOps == null) {
- txOps = new TxCleanupReadyFutureList();
- }
+ var cleanupReadyFut = new CompletableFuture<Void>();
- if (txOps.state == TxState.ABORTED || txOps.state == TxState.COMMITED) {
- fut.completeExceptionally(
- new TransactionException(TX_FAILED_READ_WRITE_OPERATION_ERR, "Transaction is already finished."));
- } else {
- txOps.futures.computeIfAbsent(cmdType, type -> new ArrayList<>()).add(fut);
- }
+ txCleanupReadyFutures.compute(txId, (id, txOps) -> {
+ if (txOps == null) {
+ txOps = new TxCleanupReadyFutureList();
+ }
- return txOps;
- });
+ if (txOps.state == TxState.ABORTED || txOps.state == TxState.COMMITED) {
+ cleanupReadyFut.completeExceptionally(new Exception());
Review Comment:
new Exception()? That doesn't sounds meaningful enough.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340398250
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1654,10 +1685,13 @@ private CompletableFuture<Object> processMultiEntryAction(ReadWriteMultiRowRepli
}
}
- CompletableFuture<Object> raftFut = rowIdsToDelete.isEmpty() ? completedFuture(null)
- : updateAllCommand(request, rowIdsToDelete).thenCompose(this::applyUpdateAllCommand);
+ if (rowIdsToDelete.isEmpty()) {
+ return completedFuture(new ReplicaResult(result, null));
Review Comment:
That's a tricky part. We still may want to update safeTime even if there are no rows to delete, otherwise we will need to await idle safe time propagation. We may cover this with separate optimization ticket, but for now I'd prefer calling applyUpdateAllCommand even for empty rows for the simplicity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345614102
##########
modules/table/src/integrationTest/java/org/apache/ignite/distributed/ItTxDistributedTestThreeNodesThreeReplicas.java:
##########
@@ -59,4 +69,36 @@ public void after() throws Exception {
super.after();
}
}
+
+ @Test
+ public void testPrimaryReplicaDirectUpdateForExplicitTxn() throws InterruptedException {
+ Peer leader = accRaftClients.get(0).leader();
+ JraftServerImpl server = (JraftServerImpl) raftServers.get(leader.consistentId()).server();
+ var groupId = new TablePartitionId(accounts.tableId(), 0);
+
+ // BLock replication messages to both replicas.
+ server.blockMessages(new RaftNodeId(groupId, leader), (msg, peerId) -> {
+ if (msg instanceof RpcRequests.AppendEntriesRequest) {
+ RpcRequests.AppendEntriesRequest tmp = (AppendEntriesRequest) msg;
+
+ if (tmp.entriesList() != null && !tmp.entriesList().isEmpty()) {
+ return true;
+ }
+ }
+ return false;
+ });
+
+ assertTrue(IgniteTestUtils.waitForCondition(() -> server.blockedMessages(new RaftNodeId(groupId, leader)).size() == 2, 10000),
+ "Failed to wait for blocked messages");
+
+ ReadWriteTransactionImpl tx = (ReadWriteTransactionImpl) igniteTransactions.begin();
+ CompletableFuture<Void> fut = accounts.recordView().upsertAsync(tx, makeValue(1, 100.));
+ // Update must complete now despite the blocked replication protocol.
+ assertTrue(IgniteTestUtils.waitForCondition(fut::isDone, 5_000), "The update future is not completed within timeout");
+
Review Comment:
I haven't
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345606827
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -261,6 +261,35 @@ private void onReplicaMessageReceived(NetworkMessage message, String senderConsi
}
}
+ if (res.repFuture() != null) {
+ assert request instanceof PrimaryReplicaRequest;
+
+ res.repFuture().handle((res0, ex0) -> {
+ NetworkMessage msg0;
+
+ LOG.debug("Sending delayed response for replica request [request={}]", request);
+
+ if (ex == null) {
+ msg0 = prepareReplicaResponse(finalSendTimestamp, res0);
Review Comment:
It is not always void. In delayed ack response txId is set as a result. See applyUpdateCommand
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345627477
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/TxManager.java:
##########
@@ -149,4 +149,19 @@ CompletableFuture<Void> cleanup(
* @return Future of all read-only transactions with read timestamp less or equals the given new low watermark.
*/
CompletableFuture<Void> updateLowWatermark(HybridTimestamp newLowWatermark);
+
+ /**
+ * Registers the infligh update for a transaction.
+ *
+ * @param tx The transaction id.
+ * @return {@code True} if the inflight was registered. The update must be failed on false.
+ */
+ boolean addInflight(UUID txId);
Review Comment:
I disagree. It makes sense only for RW transactions, so it conflicts with RO implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345570866
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -261,6 +261,37 @@ private void onReplicaMessageReceived(NetworkMessage message, String senderConsi
}
}
+ if (res.repFuture() != null) {
+ assert request instanceof PrimaryReplicaRequest;
+
+ LOG.info("Procesing delayed response for replica request [request={}]", request);
+
+ res.repFuture().handle((res0, ex0) -> {
+ NetworkMessage msg0;
+
+ LOG.info("Sending delayed response for replica request [request={}]", request);
+
+ if (ex == null) {
+ msg0 = prepareReplicaResponse(finalSendTimestamp, res0);
+ } else {
+ LOG.warn("Failed to process delayed response [request={}]", ex, request);
+
+ msg0 = prepareReplicaErrorResponse(finalSendTimestamp, ex);
+ }
+
+ ClusterNode sender = clusterNetSvc.topologyService().getByConsistentId(senderConsistentId);
+
+ if (sender != null) {
+ // Using strong send here is important to avoid a reordering with a normal response.
+ clusterNetSvc.messagingService().send(sender, msg0);
Review Comment:
The absence of reordering is guaranteed by current IO implementation. Both messages will use the same channel for sending a message, and a delayed message is send strictly after the initial.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] vldpyatkov commented on a diff in pull request #2520: IGNITE-20127
Posted by "vldpyatkov (via GitHub)" <gi...@apache.org>.
vldpyatkov commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1336998435
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaResult.java:
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.replicator;
+
+import java.util.concurrent.CompletableFuture;
+import org.jetbrains.annotations.Nullable;
+
+/**
+ * Represents replica execution result.
+ */
+public class ReplicaResult {
+ /** The result. */
+ private final Object res;
+
+ /** The replication future. */
+ private final CompletableFuture<?> repFut;
+
+ /**
+ * Construct a replica result.
+ *
+ * @param res The result.
+ * @param repFut The replication future.
+ */
+ public ReplicaResult(@Nullable Object res, @Nullable CompletableFuture<?> repFut) {
+ this.res = res;
+ this.repFut = repFut;
+ }
+
+ /**
+ * Get the result.
+ *
+ * @return The result.
+ */
+ public @Nullable Object result() {
+ return res;
+ }
+
+ /**
+ * Get the replication future.
+ *
+ * @return The replication future.
+ */
+ public @Nullable CompletableFuture<?> repFuture() {
Review Comment:
Just name it as it really is `replicationFuture`.
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1396,42 +1407,61 @@ private <T> CompletableFuture<T> continueResolvingByPk(
* @param cmdType Command type.
* @param full {@code True} if a full transaction and can be immediately committed.
* @param op Operation closure.
- * @param <T> Type of execution result.
* @return A future object representing the result of the given operation.
*/
private <T> CompletableFuture<T> appendTxCommand(UUID txId, RequestType cmdType, boolean full, Supplier<CompletableFuture<T>> op) {
- var fut = new CompletableFuture<T>();
+ if (full) {
+ return op.get().whenComplete((v, th) -> {
+ // Fast unlock.
+ releaseTxLocks(txId);
+ });
+ }
- if (!full) {
- txCleanupReadyFutures.compute(txId, (id, txOps) -> {
- if (txOps == null) {
- txOps = new TxCleanupReadyFutureList();
- }
+ var cleanupReadyFut = new CompletableFuture<Void>();
- if (txOps.state == TxState.ABORTED || txOps.state == TxState.COMMITED) {
- fut.completeExceptionally(
- new TransactionException(TX_FAILED_READ_WRITE_OPERATION_ERR, "Transaction is already finished."));
- } else {
- txOps.futures.computeIfAbsent(cmdType, type -> new ArrayList<>()).add(fut);
- }
+ txCleanupReadyFutures.compute(txId, (id, txOps) -> {
+ if (txOps == null) {
+ txOps = new TxCleanupReadyFutureList();
+ }
- return txOps;
- });
+ if (txOps.state == TxState.ABORTED || txOps.state == TxState.COMMITED) {
Review Comment:
Use here `TxState#isFinalState`
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -261,6 +261,37 @@ private void onReplicaMessageReceived(NetworkMessage message, String senderConsi
}
}
+ if (res.repFuture() != null) {
+ assert request instanceof PrimaryReplicaRequest;
+
+ LOG.info("Procesing delayed response for replica request [request={}]", request);
+
+ res.repFuture().handle((res0, ex0) -> {
+ NetworkMessage msg0;
+
+ LOG.info("Sending delayed response for replica request [request={}]", request);
+
+ if (ex == null) {
+ msg0 = prepareReplicaResponse(finalSendTimestamp, res0);
+ } else {
+ LOG.warn("Failed to process delayed response [request={}]", ex, request);
+
+ msg0 = prepareReplicaErrorResponse(finalSendTimestamp, ex);
+ }
+
+ ClusterNode sender = clusterNetSvc.topologyService().getByConsistentId(senderConsistentId);
+
+ if (sender != null) {
+ // Using strong send here is important to avoid a reordering with a normal response.
+ clusterNetSvc.messagingService().send(sender, msg0);
Review Comment:
How will we avoid reordering if the message may be in another thread?
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1810,10 +1845,19 @@ private CompletableFuture<Object> applyCmdWithExceptionHandling(Command cmd) {
* Executes an Update command.
*
* @param cmd Update command.
- * @return Raft future, see {@link #applyCmdWithExceptionHandling(Command)}.
+ * @return A local update ready future, possibly having a nested replication future as a result for delayed ack purpose.
*/
- private CompletableFuture<Object> applyUpdateCommand(UpdateCommand cmd) {
+ private CompletableFuture<CompletableFuture<?>> applyUpdateCommand(UpdateCommand cmd) {
if (!cmd.full()) {
+ CompletableFuture<UUID> fut = applyCmdWithExceptionHandling(cmd).thenApply(res -> {
+ // This check guaranties the result will never be lost. Currently always null.
+ assert res == null : "Replication result is lost";
+
+ // Set context for delayed response.
+ return cmd.txId();
+ });
+
+ // TODO error handling similar to replication group.
Review Comment:
All TODO comments should a ticket number.
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/raft/PartitionListener.java:
##########
@@ -149,6 +149,8 @@ public void onWrite(Iterator<CommandClosure<WriteCommand>> iterator) {
iterator.forEachRemaining((CommandClosure<? extends WriteCommand> clo) -> {
Command command = clo.command();
+ // LOG.info("CMD {}", command.getClass().getName());
Review Comment:
Debugging commnets.
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -156,6 +159,8 @@
/** Partition replication listener. */
public class PartitionReplicaListener implements ReplicaListener {
+ private static final IgniteLogger LOGGER = Loggers.forClass(PartitionReplicaListener.class);
Review Comment:
We name it LOG, just because we use the same name everywhere.
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1810,10 +1845,19 @@ private CompletableFuture<Object> applyCmdWithExceptionHandling(Command cmd) {
* Executes an Update command.
*
* @param cmd Update command.
- * @return Raft future, see {@link #applyCmdWithExceptionHandling(Command)}.
+ * @return A local update ready future, possibly having a nested replication future as a result for delayed ack purpose.
*/
- private CompletableFuture<Object> applyUpdateCommand(UpdateCommand cmd) {
+ private CompletableFuture<CompletableFuture<?>> applyUpdateCommand(UpdateCommand cmd) {
Review Comment:
I don't see it necessary to use this unclear logic. This method returns either a result or a replication future. A future wrapper is never required.
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1396,42 +1407,61 @@ private <T> CompletableFuture<T> continueResolvingByPk(
* @param cmdType Command type.
* @param full {@code True} if a full transaction and can be immediately committed.
* @param op Operation closure.
- * @param <T> Type of execution result.
* @return A future object representing the result of the given operation.
*/
private <T> CompletableFuture<T> appendTxCommand(UUID txId, RequestType cmdType, boolean full, Supplier<CompletableFuture<T>> op) {
- var fut = new CompletableFuture<T>();
+ if (full) {
+ return op.get().whenComplete((v, th) -> {
+ // Fast unlock.
+ releaseTxLocks(txId);
+ });
+ }
- if (!full) {
- txCleanupReadyFutures.compute(txId, (id, txOps) -> {
- if (txOps == null) {
- txOps = new TxCleanupReadyFutureList();
- }
+ var cleanupReadyFut = new CompletableFuture<Void>();
Review Comment:
I was trying to understand how this future connected with a cleanup procedure, and I realized the future does not connect with the cleanup. I would rename this variable to 'replicationReadyFut'.
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1853,36 +1902,68 @@ private CompletableFuture<Object> applyUpdateCommand(UpdateCommand cmd) {
* @param cmd UpdateAll command.
* @return Raft future, see {@link #applyCmdWithExceptionHandling(Command)}.
*/
- private CompletableFuture<Object> applyUpdateAllCommand(UpdateAllCommand cmd) {
+ private CompletableFuture<CompletableFuture<?>> applyUpdateAllCommand(UpdateAllCommand cmd, boolean skipDelayedAck) {
Review Comment:
The same is before. The future wrapper is not need here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] vldpyatkov commented on a diff in pull request #2520: IGNITE-20127
Posted by "vldpyatkov (via GitHub)" <gi...@apache.org>.
vldpyatkov commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1339258771
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TxManagerImpl.java:
##########
@@ -57,7 +65,12 @@
*
* <p>Uses 2PC for atomic commitment and 2PL for concurrency control.
*/
-public class TxManagerImpl implements TxManager {
+public class TxManagerImpl implements TxManager, NetworkMessageHandler {
+ private static final IgniteLogger LOGGER = Loggers.forClass(TxManagerImpl.class);
Review Comment:
LOG
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/storage/InternalTableImpl.java:
##########
@@ -106,6 +117,8 @@
* Storage of table rows.
*/
public class InternalTableImpl implements InternalTable {
+ private static final IgniteLogger LOGGER = Loggers.forClass(InternalTableImpl.class);
Review Comment:
We name it LOG, just because we use the same name everywhere.
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TxManagerImpl.java:
##########
@@ -202,21 +215,85 @@ public CompletableFuture<Void> finish(
) {
assert groups != null && !groups.isEmpty();
- HybridTimestamp commitTimestamp = commit ? clock.now() : null;
-
- TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
- .txId(txId)
- .timestampLong(clock.nowLong())
- .groupId(commitPartition)
- .groups(groups)
- .commit(commit)
- .commitTimestampLong(hybridTimestampToLong(commitTimestamp))
- .term(term)
- .build();
-
- return replicaService.invoke(recipientNode, req)
- // TODO: IGNITE-20033 TestOnly code, let's consider using Txn state map instead of states.
- .thenRun(() -> changeState(txId, PENDING, commit ? COMMITED : ABORTED));
+ Function<Void, CompletableFuture<Void>> clo = ignored -> {
Review Comment:
Why do not use `Supplier<CompletableFuture<Void>>`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340518920
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/TxManager.java:
##########
@@ -149,4 +149,19 @@ CompletableFuture<Void> cleanup(
* @return Future of all read-only transactions with read timestamp less or equals the given new low watermark.
*/
CompletableFuture<Void> updateLowWatermark(HybridTimestamp newLowWatermark);
+
+ /**
+ * Registers the infligh update for a transaction.
+ *
+ * @param tx The transaction id.
+ * @return {@code True} if the inflight was registered. The update must be failed on false.
+ */
+ boolean addInflight(UUID txId);
Review Comment:
I'd say that add/removeInflight should be a part of InternalTransaction and not TxManager.
Besides that javadoc requires clarification. And what's more important sql currently totally miss the thing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340522218
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/storage/InternalTableImpl.java:
##########
@@ -486,13 +476,54 @@ private <R> CompletableFuture<R> enlistWithRetry(
return result;
}
+ private <R> CompletableFuture<R> trackingInvoke(
Review Comment:
Given method deserves javadoc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340259341
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/ReplicaManager.java:
##########
@@ -261,6 +261,37 @@ private void onReplicaMessageReceived(NetworkMessage message, String senderConsi
}
}
+ if (res.repFuture() != null) {
+ assert request instanceof PrimaryReplicaRequest;
+
+ LOG.info("Procesing delayed response for replica request [request={}]", request);
+
+ res.repFuture().handle((res0, ex0) -> {
+ NetworkMessage msg0;
+
+ LOG.info("Sending delayed response for replica request [request={}]", request);
+
+ if (ex == null) {
+ msg0 = prepareReplicaResponse(finalSendTimestamp, res0);
+ } else {
+ LOG.warn("Failed to process delayed response [request={}]", ex, request);
+
+ msg0 = prepareReplicaErrorResponse(finalSendTimestamp, ex);
+ }
+
+ ClusterNode sender = clusterNetSvc.topologyService().getByConsistentId(senderConsistentId);
+
+ if (sender != null) {
+ // Using strong send here is important to avoid a reordering with a normal response.
+ clusterNetSvc.messagingService().send(sender, msg0);
Review Comment:
Agree. I'd rather add operation id or similar same for both fromPrimary and delayed responses in order to match each other. Such ids should be unique within transaction, so basically it's operation counter or similar. From one point of view that means that it won't be possible to use simple inflights counter, however on the other hand it'll be possible to implement more reliable engine including aforementioned optimizations like single response for single noded groups. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345618805
##########
modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/UpdatableTableImpl.java:
##########
@@ -237,14 +239,15 @@ public <RowT> CompletableFuture<?> insertAll(
TablePartitionId partGroupId = new TablePartitionId(tableId, partToRows.getIntKey());
NodeWithTerm nodeWithTerm = ectx.description().mapping().updatingTableAssignments().get(partToRows.getIntKey());
- ReplicaRequest request = MESSAGES_FACTORY.readWriteMultiRowReplicaRequest()
+ ReadWriteMultiRowReplicaRequest request = MESSAGES_FACTORY.readWriteMultiRowReplicaRequest()
Review Comment:
Because of skipDelayedAck.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20127 [ignite-3]
Posted by "ascherbakoff (via GitHub)" <gi...@apache.org>.
ascherbakoff commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1345586337
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TxManagerImpl.java:
##########
@@ -202,21 +215,85 @@ public CompletableFuture<Void> finish(
) {
assert groups != null && !groups.isEmpty();
- HybridTimestamp commitTimestamp = commit ? clock.now() : null;
-
- TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
- .txId(txId)
- .timestampLong(clock.nowLong())
- .groupId(commitPartition)
- .groups(groups)
- .commit(commit)
- .commitTimestampLong(hybridTimestampToLong(commitTimestamp))
- .term(term)
- .build();
-
- return replicaService.invoke(recipientNode, req)
- // TODO: IGNITE-20033 TestOnly code, let's consider using Txn state map instead of states.
- .thenRun(() -> changeState(txId, PENDING, commit ? COMMITED : ABORTED));
+ Function<Void, CompletableFuture<Void>> clo = ignored -> {
+ HybridTimestamp commitTimestamp = commit ? clock.now() : null;
+
+ TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
+ .txId(txId)
+ .timestampLong(clock.nowLong())
+ .groupId(commitPartition)
+ .groups(groups)
+ .commit(commit)
+ .commitTimestampLong(hybridTimestampToLong(commitTimestamp))
+ .term(term)
+ .build();
+
+ return replicaService.invoke(recipientNode, req)
+ // TODO: IGNITE-20033 TestOnly code, let's consider using Txn state map instead of states.
+ .thenRun(() -> changeState(txId, PENDING, commit ? COMMITED : ABORTED));
+ };
+
+ if (!commit) {
Review Comment:
I've removed code duplication.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340381326
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1396,42 +1407,61 @@ private <T> CompletableFuture<T> continueResolvingByPk(
* @param cmdType Command type.
* @param full {@code True} if a full transaction and can be immediately committed.
* @param op Operation closure.
- * @param <T> Type of execution result.
* @return A future object representing the result of the given operation.
*/
private <T> CompletableFuture<T> appendTxCommand(UUID txId, RequestType cmdType, boolean full, Supplier<CompletableFuture<T>> op) {
- var fut = new CompletableFuture<T>();
+ if (full) {
+ return op.get().whenComplete((v, th) -> {
+ // Fast unlock.
+ releaseTxLocks(txId);
+ });
+ }
- if (!full) {
- txCleanupReadyFutures.compute(txId, (id, txOps) -> {
- if (txOps == null) {
- txOps = new TxCleanupReadyFutureList();
- }
+ var cleanupReadyFut = new CompletableFuture<Void>();
- if (txOps.state == TxState.ABORTED || txOps.state == TxState.COMMITED) {
- fut.completeExceptionally(
- new TransactionException(TX_FAILED_READ_WRITE_OPERATION_ERR, "Transaction is already finished."));
- } else {
- txOps.futures.computeIfAbsent(cmdType, type -> new ArrayList<>()).add(fut);
- }
+ txCleanupReadyFutures.compute(txId, (id, txOps) -> {
+ if (txOps == null) {
+ txOps = new TxCleanupReadyFutureList();
+ }
- return txOps;
- });
+ if (txOps.state == TxState.ABORTED || txOps.state == TxState.COMMITED) {
+ cleanupReadyFut.completeExceptionally(new Exception());
Review Comment:
new Exception()? That doesn't sounds meaningful enough.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340523274
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/storage/InternalTableImpl.java:
##########
@@ -486,13 +476,54 @@ private <R> CompletableFuture<R> enlistWithRetry(
return result;
}
+ private <R> CompletableFuture<R> trackingInvoke(
+ InternalTransaction tx,
+ int partId,
+ Function<Long, ReplicaRequest> mapFunc,
+ boolean full,
+ IgniteBiTuple<ClusterNode, Long> primaryReplicaAndTerm,
+ @Nullable BiPredicate<R, ReplicaRequest> noOpChecker
+ ) {
+ ReplicaRequest request = mapFunc.apply(primaryReplicaAndTerm.get2());
+
+ boolean write = request instanceof SingleRowReplicaRequest && ((SingleRowReplicaRequest) request).requestType() != RW_GET
+ || request instanceof MultipleRowReplicaRequest && ((MultipleRowReplicaRequest) request).requestType() != RW_GET_ALL;
+
+ if (write && !full) {
+ // Track only write requests from explicit transactions.
Review Comment:
Why we track only write requests. AFAIK we should fail reading operations within finished transactions as well as modifications ones. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] sanpwc commented on a diff in pull request #2520: IGNITE-20127
Posted by "sanpwc (via GitHub)" <gi...@apache.org>.
sanpwc commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1340411360
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replicator/PartitionReplicaListener.java:
##########
@@ -1853,36 +1902,68 @@ private CompletableFuture<Object> applyUpdateCommand(UpdateCommand cmd) {
* @param cmd UpdateAll command.
* @return Raft future, see {@link #applyCmdWithExceptionHandling(Command)}.
*/
- private CompletableFuture<Object> applyUpdateAllCommand(UpdateAllCommand cmd) {
+ private CompletableFuture<CompletableFuture<?>> applyUpdateAllCommand(UpdateAllCommand cmd, boolean skipDelayedAck) {
Review Comment:
Code seems reasonably complicated. It worth adding comments explaining the flow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite-3] vldpyatkov commented on a diff in pull request #2520: IGNITE-20127
Posted by "vldpyatkov (via GitHub)" <gi...@apache.org>.
vldpyatkov commented on code in PR #2520:
URL: https://github.com/apache/ignite-3/pull/2520#discussion_r1339941012
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TxManagerImpl.java:
##########
@@ -307,4 +384,74 @@ public CompletableFuture<Void> updateLowWatermark(HybridTimestamp newLowWatermar
lowWatermarkReadWriteLock.writeLock().unlock();
}
}
+
+ @Override
+ public boolean addInflight(UUID txId) {
+ boolean[] res = {true};
+
+ txCtxMap.compute(txId, (uuid, tuple) -> {
+ if (tuple == null) {
+ tuple = new TxContext();
+ }
+
+ if (tuple.finishFut != null) {
+ res[0] = false;
+ return tuple;
+ } else {
+ //noinspection NonAtomicOperationOnVolatileField
+ tuple.inflights++;
+ }
+
+ return tuple;
+ });
+
+ //LOGGER.info("DBG: add {} {} {}", txId.toString(), txCtxMap.size(), this.hashCode());
+
+ return res[0];
+ }
+
+ @Override
+ public void removeInflight(UUID txId) {
+ TxContext tuple = txCtxMap.compute(txId, (uuid, ctx) -> {
+ assert ctx != null;
Review Comment:
Add a check that the counter of the flight operations is greater than 0.
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TxManagerImpl.java:
##########
@@ -57,7 +65,12 @@
*
* <p>Uses 2PC for atomic commitment and 2PL for concurrency control.
*/
-public class TxManagerImpl implements TxManager {
+public class TxManagerImpl implements TxManager, NetworkMessageHandler {
Review Comment:
I don't see where TxManager is used as a NetworkMessageHandler. Also, I do not understand why it is necessary to calculate inflight operations for the algorithm.
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TxManagerImpl.java:
##########
@@ -202,21 +215,85 @@ public CompletableFuture<Void> finish(
) {
assert groups != null && !groups.isEmpty();
- HybridTimestamp commitTimestamp = commit ? clock.now() : null;
-
- TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
- .txId(txId)
- .timestampLong(clock.nowLong())
- .groupId(commitPartition)
- .groups(groups)
- .commit(commit)
- .commitTimestampLong(hybridTimestampToLong(commitTimestamp))
- .term(term)
- .build();
-
- return replicaService.invoke(recipientNode, req)
- // TODO: IGNITE-20033 TestOnly code, let's consider using Txn state map instead of states.
- .thenRun(() -> changeState(txId, PENDING, commit ? COMMITED : ABORTED));
+ Function<Void, CompletableFuture<Void>> clo = ignored -> {
+ HybridTimestamp commitTimestamp = commit ? clock.now() : null;
+
+ TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
+ .txId(txId)
+ .timestampLong(clock.nowLong())
+ .groupId(commitPartition)
+ .groups(groups)
+ .commit(commit)
+ .commitTimestampLong(hybridTimestampToLong(commitTimestamp))
+ .term(term)
+ .build();
+
+ return replicaService.invoke(recipientNode, req)
+ // TODO: IGNITE-20033 TestOnly code, let's consider using Txn state map instead of states.
+ .thenRun(() -> changeState(txId, PENDING, commit ? COMMITED : ABORTED));
+ };
+
+ if (!commit) {
Review Comment:
The logic for rollback and commit operations are duplicated. You can just make this on the commit flag:
```
// All inflights have been completed before the finish.
if (commit && tuple.inflights == 0) {
tuple.waitRepFut.complete(null);
}
```
##########
modules/transactions/src/main/java/org/apache/ignite/internal/tx/impl/TxManagerImpl.java:
##########
@@ -179,16 +195,13 @@ public TxState state(UUID txId) {
@Override
public void changeState(UUID txId, TxState before, TxState after) {
- TxState computeResult = states.compute(txId, (k, v) -> {
+ states.compute(txId, (k, v) -> {
if (v == before) {
return after;
} else {
return v;
}
});
-
- assert computeResult == after : "Unable to change transaction state, expected = [" + before + "],"
Review Comment:
I'm not sure if it is right to delete this asset because the redundant state changes invoked are not expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org