You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/12/25 08:44:38 UTC

[GitHub] [pulsar] wangjialing218 opened a new issue #9060: Close current ledger when retention reached

wangjialing218 opened a new issue #9060:
URL: https://github.com/apache/pulsar/issues/9060

**Is your enhancement request related to a problem? Please describe.**
Currently task `internalTrimConsumedLedgers` will delete all closed ledgers which have reached retention except current ledger.
If the current ledger does not closed, the ledger will not deleted even retention reached.

Current ledger will be close in the follow condition:
1. entries in the ledger reaches `managedLedgerMaxEntriesPerLedger` (default is 50000).
2. total size of the ledger reached `managedLedgerMaxSizePerLedgerMbytes` (default is 2048MB)
3. `managedLedgerMaxLedgerRolloverTimeMinutes` (default is 4 hours)

That means each partitioned topic could retain 2GB storage size at most in current ledger for 4 hours, which could not be deleted after retention reached due to not closed. And in some situation like using shared bookkeeper, we hope the disk storage could be reclaimed more quickly, but retention time is not working. (related issue: #6935 #6639)

**Describe the solution you'd like**
Lower `managedLedgerMaxLedgerRolloverTimeMinutes` could be a solution, but this will roll over all current ledgers even retention is not reached.
Could we close current ledger when it reach retention? Add a judgement in `internalTrimConsumedLedgers`, if current ledger reach retention time or size, close current ledger and create new ledger, so that the closed ledger could be deleted in the next trim task. Of coure `managedLedgerMinLedgerRolloverTimeMinutes` could take effect to avoid close ledger too frequently.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] codelipenghui commented on issue #9060: Close current ledger when retention reached

Posted by GitBox <gi...@apache.org>.

codelipenghui commented on issue #9060:
URL: https://github.com/apache/pulsar/issues/9060#issuecomment-1058894448


   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] MarvinCai commented on issue #9060: Close current ledger when retention reached

Posted by GitBox <gi...@apache.org>.

MarvinCai commented on issue #9060:
URL: https://github.com/apache/pulsar/issues/9060#issuecomment-803169690


   this pr already add logic to close currentledger if it pass retention size or time: https://github.com/apache/pulsar/pull/7111


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org