You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@activemq.apache.org by "gtully (via GitHub)" <gi...@apache.org> on 2023/10/31 15:54:08 UTC
[PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
gtully opened a new pull request, #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659
… context completion callbacks are sequentially ordered to support exclusive consumer isolation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "gtully (via GitHub)" <gi...@apache.org>.
gtully commented on code in PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#discussion_r1395759813
##########
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/amq/AMQConsumer.java:
##########
@@ -517,7 +517,6 @@ public void removeRolledback(MessageReference messageReference) {
}
public void addRolledback(MessageReference messageReference) {
- currentWindow.decrementAndGet();
Review Comment:
I played with doing the session cancelRef in a new transaction, but multiple transactions can interact with IO and become out of sync due to any operation context task clearing the current context. When the delivery count is persisted, there is IO. That was to root cause of the original investigation, so I don't want to introduce that possibility again. I think what i have now is good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "AntonRoskvist (via GitHub)" <gi...@apache.org>.
AntonRoskvist commented on code in PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#discussion_r1391405403
##########
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/amq/AMQConsumer.java:
##########
@@ -517,7 +517,6 @@ public void removeRolledback(MessageReference messageReference) {
}
public void addRolledback(MessageReference messageReference) {
- currentWindow.decrementAndGet();
Review Comment:
@gtully unless adressed elsewhere, this will still cause a regression on the openwire consumers prefetch-limit...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "AntonRoskvist (via GitHub)" <gi...@apache.org>.
AntonRoskvist commented on code in PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#discussion_r1379278847
##########
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/amq/AMQConsumer.java:
##########
@@ -517,7 +517,6 @@ public void removeRolledback(MessageReference messageReference) {
}
public void addRolledback(MessageReference messageReference) {
- currentWindow.decrementAndGet();
Review Comment:
...yes, that is indeed the case. Verified by adding messages to rollback in "testRedeliveredMessageNotOverflowingPrefetch" and printing deliveryCount instead of asserting.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "clebertsuconic (via GitHub)" <gi...@apache.org>.
clebertsuconic commented on PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#issuecomment-1808471503
@gtully this looks good to me.
But can you rebase and run the testsuite again please? if all good I think you should merge this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "gtully (via GitHub)" <gi...@apache.org>.
gtully commented on PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#issuecomment-1810370356
doing some more work to get to the bottom of the currentWindow issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "gtully (via GitHub)" <gi...@apache.org>.
gtully merged PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "gtully (via GitHub)" <gi...@apache.org>.
gtully commented on PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#issuecomment-1795331969
part of my difficulty was misreading the clearContext at: https://github.com/apache/activemq-artemis/blob/a0784e3b8a75116424dd345037426e38a36f85d0/artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/OperationContextImpl.java#L319 in terms of the clear method on the context itself, which I have renamed reset! For a future self or other. The clear is clearContext refers to the thread local only, which is ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "gtully (via GitHub)" <gi...@apache.org>.
gtully commented on code in PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#discussion_r1378941324
##########
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/amq/AMQConsumer.java:
##########
@@ -517,7 +517,6 @@ public void removeRolledback(MessageReference messageReference) {
}
public void addRolledback(MessageReference messageReference) {
- currentWindow.decrementAndGet();
Review Comment:
there is a prefetch rollback test in openwire that was off by one, and that was the reason
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "AntonRoskvist (via GitHub)" <gi...@apache.org>.
AntonRoskvist commented on code in PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#discussion_r1393218874
##########
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/amq/AMQConsumer.java:
##########
@@ -517,7 +517,6 @@ public void removeRolledback(MessageReference messageReference) {
}
public void addRolledback(MessageReference messageReference) {
- currentWindow.decrementAndGet();
Review Comment:
Quite interesting... I have actually noticed queue counters (metrics) being incorrect from time to time on queues using OW consumers... in particular ones using XA-transactions. This type of application is getting exceedingly rare in the envs I manage, and it has not happened often enough to warrant further investigation, but it is something I have seen multiple times. Guessing it could have a common source to this as well.
For the added test you mentioned though, I found it to behave pretty much as I would expect it to, besides prefetch being of-by-one (which I guess might be expected given the rollback, not sure).
The error I am getting is caused by a null value returned from the receive-call, which happens because some message gets moved over to DLQ after exceeding "maximumRedeliveries"... so this is the expected result given the queue is empty.
Setting `connection.getRedeliveryPolicy().setMaximumRedeliveries(-1);` makes the test pass.
Additionally I ran with `queueControl.getDeliveringCount()` printed for each loop iteration and prefetch never exceeds prefetchPolicy + 1.
I'm not sure about this next part at all:
From what I recall from last time I looked at this, I think I interpreted the incorrect `currentWindow` and `prefetch` values to be caused by from some difference in how OW handled redeliveries compared to Core... something about the OW client holding on to the message for each redelivery but also notifying the broker each time about the redelivery, whereas for Core the message actually "bounces" back and forth for each delivery.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "AntonRoskvist (via GitHub)" <gi...@apache.org>.
AntonRoskvist commented on code in PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#discussion_r1379061456
##########
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/amq/AMQConsumer.java:
##########
@@ -517,7 +517,6 @@ public void removeRolledback(MessageReference messageReference) {
}
public void addRolledback(MessageReference messageReference) {
- currentWindow.decrementAndGet();
Review Comment:
It was a while since I looked at that, but I think this change will enable the consumer to fetch messages past the prefetchLimit. It happens when doing redeliveries, such that it can theoretically prefetch twice as many messages as it's supposed to.
Again, saw this quite some time back so I could be mistaken...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "clebertsuconic (via GitHub)" <gi...@apache.org>.
clebertsuconic commented on code in PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#discussion_r1378905454
##########
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/OpenWireConnection.java:
##########
@@ -1338,7 +1339,11 @@ public Response processRemoveSubscription(RemoveSubscriptionInfo subInfo) throws
@Override
public Response processRollbackTransaction(TransactionInfo info) throws Exception {
- Transaction tx = lookupTX(info.getTransactionId(), null, true);
+ Transaction tx = lookupTX(info.getTransactionId(), null);
+
+ if (tx == null) {
+ throw new IllegalStateException("Transaction not started, " + info.getTransactionId());
Review Comment:
I would use a logger-id from the MessageBundle.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "gtully (via GitHub)" <gi...@apache.org>.
gtully commented on code in PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#discussion_r1392644963
##########
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/amq/AMQConsumer.java:
##########
@@ -517,7 +517,6 @@ public void removeRolledback(MessageReference messageReference) {
}
public void addRolledback(MessageReference messageReference) {
- currentWindow.decrementAndGet();
Review Comment:
will investigate a bit more here, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "gtully (via GitHub)" <gi...@apache.org>.
gtully commented on code in PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#discussion_r1383250630
##########
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/OpenWireConnection.java:
##########
@@ -1338,7 +1339,11 @@ public Response processRemoveSubscription(RemoveSubscriptionInfo subInfo) throws
@Override
public Response processRollbackTransaction(TransactionInfo info) throws Exception {
- Transaction tx = lookupTX(info.getTransactionId(), null, true);
+ Transaction tx = lookupTX(info.getTransactionId(), null);
+
+ if (tx == null) {
+ throw new IllegalStateException("Transaction not started, " + info.getTransactionId());
Review Comment:
there is currently no use of the bundle in any of the exceptions, that can be another PR if really necessary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "gtully (via GitHub)" <gi...@apache.org>.
gtully commented on code in PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#discussion_r1392707143
##########
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/amq/AMQConsumer.java:
##########
@@ -517,7 +517,6 @@ public void removeRolledback(MessageReference messageReference) {
}
public void addRolledback(MessageReference messageReference) {
- currentWindow.decrementAndGet();
Review Comment:
in testRedeliveredMessageNotOverflowingPrefetch: currentWindow goes negative and message 14 does not arrive! the logic that puts messaged back on the delivered list and pulls from the transaction is in the mix here. That list has 11 messages, prefetch + the one rolled back message. I think currentWindow should only be decremented by the 1 rolledback message.
This is very tricky, but it the root problem may be that there is a single transaction now, doing the rollback of the client message and the prefetched messages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "gtully (via GitHub)" <gi...@apache.org>.
gtully commented on code in PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#discussion_r1393018970
##########
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/amq/AMQConsumer.java:
##########
@@ -517,7 +517,6 @@ public void removeRolledback(MessageReference messageReference) {
}
public void addRolledback(MessageReference messageReference) {
- currentWindow.decrementAndGet();
Review Comment:
@AntonRoskvist I pushed a new test that fails with a revert of line 520, the decrement. I don't yet know what is at fault, but the root cause is the combination of rolledback and delivered messages in a single transaction.
we decrement 11 times in stead of 1 if the session transaction is in play.
it may be that cancelRefs needs its own transaction all the time, the comments at: https://github.com/apache/activemq-artemis/blob/6e28e4683458490ab0e3990ba8f35e9c6531c461/artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/ServerSessionImpl.java#L2234
may have a hint - it speaks to the same problem.
it may be that cancleRefs should not have a tx at all, or a new one all the time that has no IO may work ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "gtully (via GitHub)" <gi...@apache.org>.
gtully commented on PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#issuecomment-1788788977
sorted some xa related tests, tx in prepared state not being active, pointed to an unnecessary change. investigating some transacted mirror issues now related to closed sessions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "gtully (via GitHub)" <gi...@apache.org>.
gtully commented on PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#issuecomment-1789418139
I need to go back to the drawing board, the ordered executor should keep completions sequential, there must be something else at play
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "clebertsuconic (via GitHub)" <gi...@apache.org>.
clebertsuconic commented on code in PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#discussion_r1378905989
##########
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/amq/AMQConsumer.java:
##########
@@ -517,7 +517,6 @@ public void removeRolledback(MessageReference messageReference) {
}
public void addRolledback(MessageReference messageReference) {
- currentWindow.decrementAndGet();
Review Comment:
why? were you having issues here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "gtully (via GitHub)" <gi...@apache.org>.
gtully commented on code in PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#discussion_r1395756119
##########
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/amq/AMQConsumer.java:
##########
@@ -517,7 +517,6 @@ public void removeRolledback(MessageReference messageReference) {
}
public void addRolledback(MessageReference messageReference) {
- currentWindow.decrementAndGet();
Review Comment:
Thanks. There was something weird, but messages going to the DLQ was part of it.
The other part is the returnReferences that puts messages back in delivering rather than back on the queue on a tx rollback. With the core session in play, that also had prefetched messages in the tx, and that was breaking the returnReferenced logic and causing to many decrements on the currentWindow.
Thanks for picking up on that change :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ARTEMIS-4480: rationalise openwire session tx usage, ensure operation… [activemq-artemis]
Posted by "gtully (via GitHub)" <gi...@apache.org>.
gtully commented on PR #4659:
URL: https://github.com/apache/activemq-artemis/pull/4659#issuecomment-1814703857
full test run looks good
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@activemq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org