You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "isminex (via GitHub)" <gi...@apache.org> on 2024/06/13 11:23:29 UTC
[I] [Bug] Major compaction is not recovered automatically after the disk is writable again [pulsar]
isminex opened a new issue, #22901:
URL: https://github.com/apache/pulsar/issues/22901
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Read release policy
- [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.
### Version
v2.11.2
### Minimal reproduce step
1. Major compaction task is suspended because the disk is almost full.
2. Minor compaction task is also suspended because the disk is full-filled.
3. Disk becomes writable again.
**Problem1**: Only `fulfilledDirs` will be recovered. If the disks(dirs) are just almost full(not full-filled), there is no change their major compaction or minor compaction task can be recovered.
**Problem2**:
Because the disk's usage is still not under warn threshold (0.9) when they are writable, only minor compaction task can be recovered and then the disks(dirs) are removed from `fulfilledDirs`, So there is no change their major compaction compaction task can be recovered because of `Problem1`.
3. Major compaction task can not be recovered even the disk is idle.
### What did you expect to see?
Major compaction is recoveryed automatically when the dist is idle again.
### What did you see instead?
Major compaction cannot be recoveryed automatically when the dist is idle again.
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Major compaction is not recovered automatically after the disk is writable again [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat closed issue #22901: [Bug] Major compaction is not recovered automatically after the disk is writable again
URL: https://github.com/apache/pulsar/issues/22901
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Major compaction is not recovered automatically after the disk is writable again [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on issue #22901:
URL: https://github.com/apache/pulsar/issues/22901#issuecomment-6381000960
Thanks @isminex for the clear analysis. The suspend and resume of bookie major and minor compaction is BookKeeper code (`GarbageCollectorThread` and the ledger-dirs listener), so the fix has to land in apache/bookkeeper. That problem, where major GC stays suspended when disk usage drops back below the warn threshold, is tracked there by https://github.com/apache/bookkeeper/pull/4018 (a related change, https://github.com/apache/bookkeeper/pull/4661, is in BookKeeper 4.17.3). Pulsar picks the fix up when it upgrades BookKeeper. I'm closing this in favour of the BookKeeper PR. Please reopen if it still happens on a currently supported release (4.0.x, 4.2.x or 5.0.x) with a BookKeeper version that includes the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org