You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/12/11 08:04:59 UTC

[GitHub] [pulsar] hangc0276 opened a new issue #13238: Should introduce deleting state for ledger deletion

hangc0276 opened a new issue #13238:
URL: https://github.com/apache/pulsar/issues/13238


   **Is your enhancement request related to a problem? Please describe.**
   In current ledger deletion, we divided it into two separate steps.
   1. Remove all the waiting to delete ledgers from ledger list and update the newest ledger list into meta store.
   2.  In the meta store update callback operation, delete the waiting to delete ledgers from storage system, such as BookKeeper or Tiered storage.
   
   Due to the separate step, we can't ensure the ledger deletion transaction. If the first step succeed and the second step failed, it will lead to ledgers can't be deleted from storage system forever. The second step maybe fail by broker restart or storage system deletion failed.
   
   https://github.com/apache/pulsar/blob/bd68b6f05f9749328701c59bdaf3cddda2254d39/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L2532-L2561
   
   In our customer's environment, we have found many orphan ledgers cause by the above reason.
   
   **Describe the solution you'd like**
   We'd better introduce a `deleting` state for the waiting to delete ledgers. The desired ledger deletion should follow the three steps.
   1. Remove all the waiting to delete ledgers from ledger list and update the newest ledger list into meta store.
   2. Write the waiting to delete ledgers into meta store marked as `deleting` state.
   3. Use another thread to periodically check the `deleting` state ledgers, and do ledger delete operation on storage system. Each topic's owner control their own topics' ledger deletion.
   4. After the ledger deletion complete, remove the ledger from meta store. 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] hangc0276 commented on issue #13238: Should introduce deleting state for ledger deletion

Posted by GitBox <gi...@apache.org>.
hangc0276 commented on issue #13238:
URL: https://github.com/apache/pulsar/issues/13238#issuecomment-991525734


   @wuzhanpeng Thanks for your help, I have assigned this issue to you. If you have any questions or need any help, please ping me at any time.  You'd better write a proposal first.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] wuzhanpeng commented on issue #13238: Should introduce deleting state for ledger deletion

Posted by GitBox <gi...@apache.org>.
wuzhanpeng commented on issue #13238:
URL: https://github.com/apache/pulsar/issues/13238#issuecomment-991524211


   I can help with this issue~


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] github-actions[bot] commented on issue #13238: Should introduce deleting state for ledger deletion

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #13238:
URL: https://github.com/apache/pulsar/issues/13238#issuecomment-1069835123


   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] wuzhanpeng commented on issue #13238: Should introduce deleting state for ledger deletion

Posted by GitBox <gi...@apache.org>.
wuzhanpeng commented on issue #13238:
URL: https://github.com/apache/pulsar/issues/13238#issuecomment-991524211


   I can help with this issue~


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] hangc0276 commented on issue #13238: Should introduce deleting state for ledger deletion

Posted by GitBox <gi...@apache.org>.
hangc0276 commented on issue #13238:
URL: https://github.com/apache/pulsar/issues/13238#issuecomment-991525734


   @wuzhanpeng Thanks for your help, I have assigned this issue to you. If you have any questions or need any help, please ping me at any time.  You'd better write a proposal first.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org