You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/10/17 00:30:12 UTC
[GitHub] [pulsar] liangyepianzhou opened a new pull request #12386: Fix maxReadPosition with normal publish
liangyepianzhou opened a new pull request #12386:
URL: https://github.com/apache/pulsar/pull/12386
Fixes https://github.com/apache/pulsar/issues/12382
### Motivation
We should not change maxReadPosition before TransactionBuffer recovers completely.
### Modifications
It will change when the state of TB is NoSnapshot or the state of TB is Ready but the ongoingTxn is empty.
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
Add test in TransactionTest.
### Does this pull request potentially affect one of the following parts:
*If `yes` was chosen, please highlight the changes*
- Dependencies (does it add or upgrade a dependency): (yes / no)
- The public API: (yes / no)
- The schema: (yes / no / don't know)
- The default values of configurations: (yes / no)
- The wire protocol: (yes / no)
- The rest endpoints: (yes / no)
- The admin cli options: (yes / no)
- Anything that affects deployment: (yes / no / don't know)
### Documentation
Check the box below and label this PR (if you have committer privilege).
Need to update docs?
- [x] no-need-doc
This is a bug fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on pull request #12386: [Transaction]Fix maxReadPosition with normal publish
Posted by GitBox <gi...@apache.org>.
liangyepianzhou commented on pull request #12386:
URL: https://github.com/apache/pulsar/pull/12386#issuecomment-947197375
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] congbobo184 merged pull request #12386: [Transaction]Fix maxReadPosition with normal publish
Posted by GitBox <gi...@apache.org>.
congbobo184 merged pull request #12386:
URL: https://github.com/apache/pulsar/pull/12386
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] congbobo184 merged pull request #12386: [Transaction]Fix maxReadPosition with normal publish
Posted by GitBox <gi...@apache.org>.
congbobo184 merged pull request #12386:
URL: https://github.com/apache/pulsar/pull/12386
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on pull request #12386: [Transaction]Fix maxReadPosition with normal publish
Posted by GitBox <gi...@apache.org>.
liangyepianzhou commented on pull request #12386:
URL: https://github.com/apache/pulsar/pull/12386#issuecomment-946556119
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] congbobo184 merged pull request #12386: [Transaction]Fix maxReadPosition with normal publish
Posted by GitBox <gi...@apache.org>.
congbobo184 merged pull request #12386:
URL: https://github.com/apache/pulsar/pull/12386
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on a change in pull request #12386: [Transaction]Fix maxReadPosition with normal publish
Posted by GitBox <gi...@apache.org>.
liangyepianzhou commented on a change in pull request #12386:
URL: https://github.com/apache/pulsar/pull/12386#discussion_r731793677
##########
File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/buffer/impl/TopicTransactionBuffer.java
##########
@@ -455,9 +455,14 @@ public void syncMaxReadPositionForNormalPublish(PositionImpl position) {
// when ongoing transaction is empty, proved that lastAddConfirm is can read max position, because callback
// thread is the same tread, in this time the lastAddConfirm don't content transaction message.
synchronized (TopicTransactionBuffer.this) {
- if (ongoingTxns.isEmpty()) {
+ if (checkIfNoSnapshot()) {
maxReadPosition = position;
changeMaxReadPositionAndAddAbortTimes.incrementAndGet();
+ } else if (checkIfReady()) {
+ if (ongoingTxns.isEmpty()) {
Review comment:
When transaction aborted or committed, maxReadPosition will be updated. In addition, the transaction has a timeout mechanism, and the transaction will be aborted after the timeout.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaoran10 commented on a change in pull request #12386: [Transaction]Fix maxReadPosition with normal publish
Posted by GitBox <gi...@apache.org>.
gaoran10 commented on a change in pull request #12386:
URL: https://github.com/apache/pulsar/pull/12386#discussion_r731959296
##########
File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/buffer/impl/TopicTransactionBuffer.java
##########
@@ -455,9 +455,14 @@ public void syncMaxReadPositionForNormalPublish(PositionImpl position) {
// when ongoing transaction is empty, proved that lastAddConfirm is can read max position, because callback
// thread is the same tread, in this time the lastAddConfirm don't content transaction message.
synchronized (TopicTransactionBuffer.this) {
- if (ongoingTxns.isEmpty()) {
+ if (checkIfNoSnapshot()) {
maxReadPosition = position;
changeMaxReadPositionAndAddAbortTimes.incrementAndGet();
+ } else if (checkIfReady()) {
+ if (ongoingTxns.isEmpty()) {
Review comment:
I think this is ok because that if each transaction message may update the `maxReadPosition`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on a change in pull request #12386: [Transaction]Fix maxReadPosition with normal publish
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on a change in pull request #12386:
URL: https://github.com/apache/pulsar/pull/12386#discussion_r731764535
##########
File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/buffer/impl/TopicTransactionBuffer.java
##########
@@ -455,9 +455,14 @@ public void syncMaxReadPositionForNormalPublish(PositionImpl position) {
// when ongoing transaction is empty, proved that lastAddConfirm is can read max position, because callback
// thread is the same tread, in this time the lastAddConfirm don't content transaction message.
synchronized (TopicTransactionBuffer.this) {
- if (ongoingTxns.isEmpty()) {
+ if (checkIfNoSnapshot()) {
maxReadPosition = position;
changeMaxReadPositionAndAddAbortTimes.incrementAndGet();
+ } else if (checkIfReady()) {
+ if (ongoingTxns.isEmpty()) {
Review comment:
If we always have a few ongoing txns, how to update the maxReadPosition?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaoran10 commented on a change in pull request #12386: [Transaction]Fix maxReadPosition with normal publish
Posted by GitBox <gi...@apache.org>.
gaoran10 commented on a change in pull request #12386:
URL: https://github.com/apache/pulsar/pull/12386#discussion_r731974599
##########
File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/buffer/impl/TopicTransactionBuffer.java
##########
@@ -455,9 +455,14 @@ public void syncMaxReadPositionForNormalPublish(PositionImpl position) {
// when ongoing transaction is empty, proved that lastAddConfirm is can read max position, because callback
// thread is the same tread, in this time the lastAddConfirm don't content transaction message.
synchronized (TopicTransactionBuffer.this) {
- if (ongoingTxns.isEmpty()) {
+ if (checkIfNoSnapshot()) {
maxReadPosition = position;
changeMaxReadPositionAndAddAbortTimes.incrementAndGet();
+ } else if (checkIfReady()) {
Review comment:
Maybe the condition `checkIfReady` is enough?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] congbobo184 merged pull request #12386: [Transaction]Fix maxReadPosition with normal publish
Posted by GitBox <gi...@apache.org>.
congbobo184 merged pull request #12386:
URL: https://github.com/apache/pulsar/pull/12386
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaoran10 commented on a change in pull request #12386: [Transaction]Fix maxReadPosition with normal publish
Posted by GitBox <gi...@apache.org>.
gaoran10 commented on a change in pull request #12386:
URL: https://github.com/apache/pulsar/pull/12386#discussion_r731959296
##########
File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/buffer/impl/TopicTransactionBuffer.java
##########
@@ -455,9 +455,14 @@ public void syncMaxReadPositionForNormalPublish(PositionImpl position) {
// when ongoing transaction is empty, proved that lastAddConfirm is can read max position, because callback
// thread is the same tread, in this time the lastAddConfirm don't content transaction message.
synchronized (TopicTransactionBuffer.this) {
- if (ongoingTxns.isEmpty()) {
+ if (checkIfNoSnapshot()) {
maxReadPosition = position;
changeMaxReadPositionAndAddAbortTimes.incrementAndGet();
+ } else if (checkIfReady()) {
+ if (ongoingTxns.isEmpty()) {
Review comment:
I think this is ok because that each transaction message may update the `maxReadPosition`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on pull request #12386: [Transaction]Fix maxReadPosition with normal publish
Posted by GitBox <gi...@apache.org>.
liangyepianzhou commented on pull request #12386:
URL: https://github.com/apache/pulsar/pull/12386#issuecomment-944365269
@codelipenghui @congbobo184 @gaoran10 Please help review, THX.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on a change in pull request #12386: [Transaction]Fix maxReadPosition with normal publish
Posted by GitBox <gi...@apache.org>.
liangyepianzhou commented on a change in pull request #12386:
URL: https://github.com/apache/pulsar/pull/12386#discussion_r732002084
##########
File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/buffer/impl/TopicTransactionBuffer.java
##########
@@ -455,9 +455,14 @@ public void syncMaxReadPositionForNormalPublish(PositionImpl position) {
// when ongoing transaction is empty, proved that lastAddConfirm is can read max position, because callback
// thread is the same tread, in this time the lastAddConfirm don't content transaction message.
synchronized (TopicTransactionBuffer.this) {
- if (ongoingTxns.isEmpty()) {
+ if (checkIfNoSnapshot()) {
maxReadPosition = position;
changeMaxReadPositionAndAddAbortTimes.incrementAndGet();
+ } else if (checkIfReady()) {
Review comment:
The state of TB may be NoSnapshot and Ready after TC recovered completely. We allow producer to send common messages and update MaxReadPosition when the state is NoSnapshot .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org