You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/08/23 03:28:02 UTC

[GitHub] [pulsar] codelipenghui opened a new issue, #17220: Make the autoSkipNonRecoverableData can work for cursor ledger

codelipenghui opened a new issue, #17220:
URL: https://github.com/apache/pulsar/issues/17220

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
   
   
   ### Motivation
   
   After we enabled autoSkipNonRecoverableData, the topic will skip the non-recoverable ledger which will lose some data, but the topic is able to load and continues to provide messaging service. This usually happens when multiple bookie replicas are corrupted, or the Ledger has been deleted This usually happens when multiple bookie copies are corrupted
   
   But the Ledger used by the cursor is not able to be skipped even if `autoSkipNonRecoverableData=true`
   
   ### Solution
   
   Make the cursor can skip the non-recoverable Ledger if `autoSkipNonRecoverableData=true`
   
   ### Alternatives
   
   No
   
   ### Anything else?
   
   No
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] poorbarcode commented on issue #17220: Make the autoSkipNonRecoverableData can work for cursor ledger

Posted by GitBox <gi...@apache.org>.

poorbarcode commented on issue #17220:
URL: https://github.com/apache/pulsar/issues/17220#issuecomment-1223686603

   assign to me


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] github-actions[bot] commented on issue #17220: Make the autoSkipNonRecoverableData can work for cursor ledger

Posted by GitBox <gi...@apache.org>.

github-actions[bot] commented on issue #17220:
URL: https://github.com/apache/pulsar/issues/17220#issuecomment-1256835683

   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] poorbarcode commented on issue #17220: Make the autoSkipNonRecoverableData can work for cursor ledger

Posted by GitBox <gi...@apache.org>.

poorbarcode commented on issue #17220:
URL: https://github.com/apache/pulsar/issues/17220#issuecomment-1225987441

   Current implementation: `ManagedCursor` will skip the invalid data during `recover`, even if the feature `autoSkipNonRecoverableData ` is disabled.
   
   Ledger data invalid occurs in two cases: 
   
   - ledger could not be found
   - some entries could not be found
   
   ----
   
   #### Ledger could not be found
   
   When the ledger could not be found, we can recover data from ZK. But when switching ledger, data stored in ZK does not contain `individualDeletedMessages`, so we can only recover `markDeletedPosition` from ZK. The current code already implements this.
   
   ----
   
   ####  (<strong>High light</strong>) Some entries could not be found
   
   When the last entry is could not be found, we can trace the entry forward until one available entry is found, so as to recover data as much as possible. PR #17264 resolves this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] github-actions[bot] commented on issue #17220: Make the autoSkipNonRecoverableData can work for cursor ledger

Posted by GitBox <gi...@apache.org>.

github-actions[bot] commented on issue #17220:
URL: https://github.com/apache/pulsar/issues/17220#issuecomment-1312319493

   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org