You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/06/27 10:30:57 UTC
[GitHub] [pulsar] poorbarcode opened a new pull request, #16240: [fix] [transaction] infinite loop task that pengding ack log replay
poorbarcode opened a new pull request, #16240:
URL: https://github.com/apache/pulsar/pull/16240
### Motivation
`pending-ack-cursor` will reload by `MLPendingAckStoreProvider` when created `persistent subscription`, it will check the cache firstly, and create a new one if the cache does not exist.
https://github.com/apache/pulsar/blob/eeb22ba9631d86528bbca8f1825feff7e70272d2/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L928-L935
The managed ledger and cursor which already exist may be like this:
```
cursor.markDeletePosition: 60:1
cursor.readPosition: 60:6
managedLedger.getLastConfirmedEntry: 60:5
```
So `MLPendingAckStore` will initialize like this:
```
currentLoadPosition: 60:1
lastConfirmedEntry: 60:5
```
At this point, we expected four logs to be read, but none were. And then it goes into an infinite loop.
https://github.com/apache/pulsar/blob/877795ead640039a0bcb5ef0b9aa190c3536ca1e/pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/pendingack/impl/MLPendingAckStore.java#L332-L355
And we should not lose these four logs while executing A, because they may be in a transaction that has not yet handle, so we should reset the read position of the pending-ack-log cursor.
### Modifications
Consistently execute 'cursor.rewind' after pending-ack-log cursor initialize.
### Documentation
- [ ] `doc-required`
- [x] `doc-not-needed`
- [ ] `doc`
- [ ] `doc-complete`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode closed pull request #16240: [fix] [transaction] infinite loop task that pengding ack log replay
Posted by GitBox <gi...@apache.org>.
poorbarcode closed pull request #16240: [fix] [transaction] infinite loop task that pengding ack log replay
URL: https://github.com/apache/pulsar/pull/16240
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on pull request #16240: [fix] [transaction] infinite loop task that pengding ack log replay
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on PR #16240:
URL: https://github.com/apache/pulsar/pull/16240#issuecomment-1169977643
https://github.com/apache/pulsar/pull/16247 can solve this problem, close this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on pull request #16240: [fix] [transaction] infinite loop task that pengding ack log replay
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on PR #16240:
URL: https://github.com/apache/pulsar/pull/16240#issuecomment-1167181724
@congbobo184 @liangyepianzhou Could you take a look, thanks. ^_^
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on pull request #16240: [fix] [transaction] infinite loop task that pengding ack log replay
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on PR #16240:
URL: https://github.com/apache/pulsar/pull/16240#issuecomment-1167327049
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #16240: [fix] [transaction] infinite loop task that pengding ack log replay
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #16240:
URL: https://github.com/apache/pulsar/pull/16240#issuecomment-1167316659
@poorbarcode Please provide a correct documentation label for your PR.
Instructions see [Pulsar Documentation Label Guide](https://docs.google.com/document/d/1Qw7LHQdXWBW9t2-r-A7QdFDBwmZh6ytB4guwMoXHqc0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on pull request #16240: [fix] [transaction] infinite loop task that pengding ack log replay
Posted by GitBox <gi...@apache.org>.
liangyepianzhou commented on PR #16240:
URL: https://github.com/apache/pulsar/pull/16240#issuecomment-1167365277
The cursor of the pending ack log is only used when it is created and recovered, so it will not go to the cache.
Are there any scenarios I haven't considered?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on pull request #16240: [fix] [transaction] infinite loop task that pengding ack log replay
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on PR #16240:
URL: https://github.com/apache/pulsar/pull/16240#issuecomment-1167633068
> The cursor of the pending ack log is only used when it is created and recovered, so it will not go to the cache. Are there any scenarios I haven't considered?
Yes, It is #16247
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #16240: [fix] [transaction] infinite loop task that pengding ack log replay
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #16240:
URL: https://github.com/apache/pulsar/pull/16240#issuecomment-1167310095
@poorbarcode Please provide a correct documentation label for your PR.
Instructions see [Pulsar Documentation Label Guide](https://docs.google.com/document/d/1Qw7LHQdXWBW9t2-r-A7QdFDBwmZh6ytB4guwMoXHqc0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on a diff in pull request #16240: [fix] [transaction] infinite loop task that pengding ack log replay
Posted by GitBox <gi...@apache.org>.
liangyepianzhou commented on code in PR #16240:
URL: https://github.com/apache/pulsar/pull/16240#discussion_r907401718
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/pendingack/impl/MLPendingAckStoreProvider.java:
##########
@@ -74,6 +74,9 @@ public void openLedgerComplete(ManagedLedger ledger, Object ctx) {
InitialPosition.Earliest, new AsyncCallbacks.OpenCursorCallback() {
@Override
public void openCursorComplete(ManagedCursor cursor, Object ctx) {
+ // Why "rewind" ?
+ // see: https://github.com/apache/pulsar/pull/16240
+ cursor.rewind();
Review Comment:
Is it better to delete the cursor after recovery is complete?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #16240: [fix] [transaction] infinite loop task that pengding ack log replay
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #16240:
URL: https://github.com/apache/pulsar/pull/16240#issuecomment-1167328351
@poorbarcode Please provide a correct documentation label for your PR.
Instructions see [Pulsar Documentation Label Guide](https://docs.google.com/document/d/1Qw7LHQdXWBW9t2-r-A7QdFDBwmZh6ytB4guwMoXHqc0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on a diff in pull request #16240: [fix] [transaction] infinite loop task that pengding ack log replay
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on code in PR #16240:
URL: https://github.com/apache/pulsar/pull/16240#discussion_r907616835
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/pendingack/impl/MLPendingAckStoreProvider.java:
##########
@@ -74,6 +74,9 @@ public void openLedgerComplete(ManagedLedger ledger, Object ctx) {
InitialPosition.Earliest, new AsyncCallbacks.OpenCursorCallback() {
@Override
public void openCursorComplete(ManagedCursor cursor, Object ctx) {
+ // Why "rewind" ?
+ // see: https://github.com/apache/pulsar/pull/16240
+ cursor.rewind();
Review Comment:
> Is it better to delete the cursor after recovery is complete?
No, that won't solve this problem. #16247 can solve this problem, I am thinking of closing this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on pull request #16240: [fix] [transaction] infinite loop task that pengding ack log replay
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on PR #16240:
URL: https://github.com/apache/pulsar/pull/16240#issuecomment-1167315560
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org