You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/01/28 20:21:37 UTC
[GitHub] [pulsar] congbobo184 opened a new pull request #14020: [Transaction] Fix individual ack with transaction decrease unAckMessageCounnt
congbobo184 opened a new pull request #14020:
URL: https://github.com/apache/pulsar/pull/14020
link https://github.com/apache/pulsar/pull/13383
## Motivation
#13383 has fixed the batch message ack does not decrease the unacked-msg count, but ack with transaction don't fix
because decrease unAckMessageCount move to another method. ack with transaction can't decrease unackMessageCount.
### Modifications
change ack with transaction decrease unackMessageCount to normal ack(ack no transaction).
### Verifying this change
Add the tests for it
Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes
Dependencies (does it add or upgrade a dependency): (no)
The public API: (no)
The schema: (no)
The default values of configurations: (no)
The wire protocol: (no)
The rest endpoints: (no)
The admin cli options: (no)
Anything that affects deployment: (no)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] congbobo184 commented on a change in pull request #14020: [Transaction] Fix individual ack with transaction decrease unAckMessageCounnt
Posted by GitBox <gi...@apache.org>.
congbobo184 commented on a change in pull request #14020:
URL: https://github.com/apache/pulsar/pull/14020#discussion_r795114396
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/client/impl/TransactionEndToEndTest.java
##########
@@ -249,6 +246,63 @@ public void produceAbortTest() throws Exception {
log.info("finished test partitionAbortTest");
}
+ @Test(dataProvider="enableBatch")
+ private void testAckWithTransactionReduceUnAckMessageCount(boolean enableBatch) throws Exception {
Review comment:
TransactionTestBase has enable batch index ack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #14020: [Transaction] Fix individual ack with transaction decrease unAckMessageCounnt
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14020:
URL: https://github.com/apache/pulsar/pull/14020#issuecomment-1024608435
@congbobo184:Thanks for providing doc info!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on a change in pull request #14020: [Transaction] Fix individual ack with transaction decrease unAckMessageCounnt
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on a change in pull request #14020:
URL: https://github.com/apache/pulsar/pull/14020#discussion_r795110792
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/client/impl/TransactionEndToEndTest.java
##########
@@ -249,6 +246,63 @@ public void produceAbortTest() throws Exception {
log.info("finished test partitionAbortTest");
}
+ @Test(dataProvider="enableBatch")
+ private void testAckWithTransactionReduceUnAckMessageCount(boolean enableBatch) throws Exception {
Review comment:
The test should enable batch index ack for the broker and the consumer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaoran10 commented on a change in pull request #14020: [Transaction] Fix individual ack with transaction decrease unAckMessageCounnt
Posted by GitBox <gi...@apache.org>.
gaoran10 commented on a change in pull request #14020:
URL: https://github.com/apache/pulsar/pull/14020#discussion_r795004636
##########
File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
##########
@@ -464,14 +464,28 @@ public void doUnsubscribe(final long requestId) {
for (int i = 0; i < ack.getMessageIdsCount(); i++) {
MessageIdData msgId = ack.getMessageIdAt(i);
PositionImpl position;
+ long ackedCount = 0;
+ long batchSize = getBatchSize(msgId);
+ Consumer ackOwnerConsumer = getAckOwnerConsumer(msgId.getLedgerId(), msgId.getEntryId());
if (msgId.getAckSetsCount() > 0) {
- long[] acksSets = new long[msgId.getAckSetsCount()];
+ long[] ackSets = new long[msgId.getAckSetsCount()];
for (int j = 0; j < msgId.getAckSetsCount(); j++) {
- acksSets[j] = msgId.getAckSetAt(j);
+ ackSets[j] = msgId.getAckSetAt(j);
}
- position = PositionImpl.get(msgId.getLedgerId(), msgId.getEntryId(), acksSets);
+ position = PositionImpl.get(msgId.getLedgerId(), msgId.getEntryId(), ackSets);
+ ackedCount = getAckedCountForBatchIndexLevelEnabled(position, batchSize, ackSets);
} else {
position = PositionImpl.get(msgId.getLedgerId(), msgId.getEntryId());
+ if (isAcknowledgmentAtBatchIndexLevelEnabled) {
+ long[] cursorAckSet = getCursorAckSet(position);
+ if (cursorAckSet != null) {
+ ackedCount = batchSize - BitSet.valueOf(cursorAckSet).cardinality();
+ } else {
+ ackedCount = batchSize;
+ }
+ } else {
+ ackedCount = batchSize;
+ }
Review comment:
LGTM, a little suggestion. Could we reuse this code block for methods `individualAckNormal` and `individualAckWithTransaction`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #14020: [Transaction] Fix individual ack with transaction decrease unAckMessageCounnt
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14020:
URL: https://github.com/apache/pulsar/pull/14020#issuecomment-1024608195
@congbobo184:Thanks for your contribution. For this PR, do we need to update docs?
(The [PR template contains info about doc](https://github.com/apache/pulsar/blob/master/.github/PULL_REQUEST_TEMPLATE.md#documentation), which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui merged pull request #14020: [Transaction] Fix individual ack with transaction decrease unAckMessageCounnt
Posted by GitBox <gi...@apache.org>.
codelipenghui merged pull request #14020:
URL: https://github.com/apache/pulsar/pull/14020
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaoran10 commented on a change in pull request #14020: [Transaction] Fix individual ack with transaction decrease unAckMessageCounnt
Posted by GitBox <gi...@apache.org>.
gaoran10 commented on a change in pull request #14020:
URL: https://github.com/apache/pulsar/pull/14020#discussion_r795004636
##########
File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
##########
@@ -464,14 +464,28 @@ public void doUnsubscribe(final long requestId) {
for (int i = 0; i < ack.getMessageIdsCount(); i++) {
MessageIdData msgId = ack.getMessageIdAt(i);
PositionImpl position;
+ long ackedCount = 0;
+ long batchSize = getBatchSize(msgId);
+ Consumer ackOwnerConsumer = getAckOwnerConsumer(msgId.getLedgerId(), msgId.getEntryId());
if (msgId.getAckSetsCount() > 0) {
- long[] acksSets = new long[msgId.getAckSetsCount()];
+ long[] ackSets = new long[msgId.getAckSetsCount()];
for (int j = 0; j < msgId.getAckSetsCount(); j++) {
- acksSets[j] = msgId.getAckSetAt(j);
+ ackSets[j] = msgId.getAckSetAt(j);
}
- position = PositionImpl.get(msgId.getLedgerId(), msgId.getEntryId(), acksSets);
+ position = PositionImpl.get(msgId.getLedgerId(), msgId.getEntryId(), ackSets);
+ ackedCount = getAckedCountForBatchIndexLevelEnabled(position, batchSize, ackSets);
} else {
position = PositionImpl.get(msgId.getLedgerId(), msgId.getEntryId());
+ if (isAcknowledgmentAtBatchIndexLevelEnabled) {
+ long[] cursorAckSet = getCursorAckSet(position);
+ if (cursorAckSet != null) {
+ ackedCount = batchSize - BitSet.valueOf(cursorAckSet).cardinality();
+ } else {
+ ackedCount = batchSize;
+ }
+ } else {
+ ackedCount = batchSize;
+ }
Review comment:
LGTM, left a trivial suggestion. Could we reuse this code block for methods `individualAckNormal` and `individualAckWithTransaction`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org