You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/08/20 07:16:45 UTC
[GitHub] [pulsar] MarvinCai opened a new pull request #7858: Add config to lazily recover cursors when recovering a managed ledger.
MarvinCai opened a new pull request #7858:
URL: https://github.com/apache/pulsar/pull/7858
Fixes #7404
### Motivation
Currently, we have to wait until all cursors to be recovered before completing opening a managed ledger. This results in very long recovery time for the managed ledger. Pulsar is a messaging/streaming system. Write availability is critical to a lot of core business applications. We need to make it always writable. Hence we should decouple loading cursors from loading managed ledger. This ensures producers can move forward immediately once the data ledgers are ready.
### Modifications
Add config to lazily recover cursors when recovering a managed ledger.
Will initial cursor recovery and put it into uninitializedCursors map.
This can speed up managed ledger recovery and write availability,
but with the caveat that when topic is recovered we're not sure if all old cursor can be recovered or not.
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
*(Please pick either of the following options)*
This change added tests and can be verified as follows:
- *Added unit test to verify ledger is ready before cursor fully recovered*
### Does this pull request potentially affect one of the following parts:
*If `yes` was chosen, please highlight the changes*
- Dependencies (does it add or upgrade a dependency): (no)
- The public API: (no)
- The schema: (no)
- The default values of configurations: (no)
- The wire protocol: (no)
- The rest endpoints: (no)
- The admin cli options: (no)
- Anything that affects deployment: (no)
### Documentation
- Does this pull request introduce a new feature? (yes)
- If yes, how is the feature documented? ( docs / JavaDocs )
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] MarvinCai commented on pull request #7858: Add config to lazily recover cursors when recovering a managed ledger.
Posted by GitBox <gi...@apache.org>.
MarvinCai commented on pull request #7858:
URL: https://github.com/apache/pulsar/pull/7858#issuecomment-686497096
/pulsarbot rerun unit-tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiazhai merged pull request #7858: Add config to lazily recover cursors when recovering a managed ledger.
Posted by GitBox <gi...@apache.org>.
jiazhai merged pull request #7858:
URL: https://github.com/apache/pulsar/pull/7858
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] MarvinCai commented on pull request #7858: Add config to lazily recover cursors when recovering a managed ledger.
Posted by GitBox <gi...@apache.org>.
MarvinCai commented on pull request #7858:
URL: https://github.com/apache/pulsar/pull/7858#issuecomment-677717058
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] MarvinCai commented on pull request #7858: Add config to lazily recover cursors when recovering a managed ledger.
Posted by GitBox <gi...@apache.org>.
MarvinCai commented on pull request #7858:
URL: https://github.com/apache/pulsar/pull/7858#issuecomment-686592307
/pulsarbot rerun unit-tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] MarvinCai commented on pull request #7858: Add config to lazily recover cursors when recovering a managed ledger.
Posted by GitBox <gi...@apache.org>.
MarvinCai commented on pull request #7858:
URL: https://github.com/apache/pulsar/pull/7858#issuecomment-686493089
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] MarvinCai commented on pull request #7858: Add config to lazily recover cursors when recovering a managed ledger.
Posted by GitBox <gi...@apache.org>.
MarvinCai commented on pull request #7858:
URL: https://github.com/apache/pulsar/pull/7858#issuecomment-686272888
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org