You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/05/06 15:57:28 UTC
[GitHub] [pulsar] Nicklee007 opened a new issue, #15473: [Bug][broker] expired ledger cannot be cleanup when ledger consumed completely
Nicklee007 opened a new issue, #15473:
URL: https://github.com/apache/pulsar/issues/15473
**Describe the bug**
In some case, expired ledger data cannot be cleanup when the topic has not new message and ledger consumed completely, because the slowestReaderPosition which set to the new ledger will be cover to the old ledger by some cursor operator which like flink reset the Durable Cursor and the auto flushed dirty mark-delete position.
Also, the last expired ledger can not be clean if the topic has not active consumer and expired the message by MessageExpiryMonitor or the cursor markDeletePosition are in some old ledger when stop produce message.
<img width="1218" alt="image" src="https://user-images.githubusercontent.com/84127069/167107771-5390825d-45e1-40d9-b58f-c73776097281.png">
**To Reproduce**
Steps to reproduce the behavior:
1. For quickly reproduce we change the config managedLedgerMaxLedgerRolloverTimeMinutes and managedLedgerMinLedgerRolloverTimeMinutes to few minutes.
2. start broker, create topic and set-message-ttl and set-retention to 4 * managedLedgerMaxLedgerRolloverTimeMinutes;
3. create Subscription and produce some message longer than set-message-ttl;
4. stop producer and wait the message expired by MessageExpiryMonitor;
5. the are two ledger. The has data ledger's all message are expired but can not be cleanup.
**Expected behavior**
All expired ledger data should be cleanup. There are many expired ledger's data can not be clean , If we have many not new message's topic. we also need migrate the ledger data when we offline one bookie.
**Additional context**
The bug can be reproduce before the pulsar version 2.10.1, the current master pr [#14672](https://github.com/apache/pulsar/pull/14672) change the behavior of rollCurrentLedgerIfFull when roll full ledger could not create new ledger, I think should resume the behavior to create new ledger.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][broker] expired ledger cannot be cleanup when ledger consumed completely [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat closed issue #15473: [Bug][broker] expired ledger cannot be cleanup when ledger consumed completely
URL: https://github.com/apache/pulsar/issues/15473
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #15473: [Bug][broker] expired ledger cannot be cleanup when ledger consumed completely
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #15473:
URL: https://github.com/apache/pulsar/issues/15473#issuecomment-1146963145
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][broker] expired ledger cannot be cleanup when ledger consumed completely [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on issue #15473:
URL: https://github.com/apache/pulsar/issues/15473#issuecomment-6386776588
Thanks @Nicklee007 for the report. This was reported on a Pulsar release before 3.0, which is no longer supported; the supported releases are 4.0.x, 4.2.x and 5.0.x. Much of the code involved has changed since, and the problem hasn't been confirmed on a current release, so I'm closing it as part of cleaning up old reports. If you still see it on a supported release, please open a new issue with the version and steps to reproduce, and link this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org