You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/07/05 14:12:47 UTC

[GitHub] [pulsar] Technoboy- opened a new pull request, #16407: [Branch-2.7][Cherry-pick] Fix the reader skips compacted data which original ledger been removed.

Technoboy- opened a new pull request, #16407:
URL: https://github.com/apache/pulsar/pull/16407

   Cherry-pick #12522
   
   ### Motivation
   The compactor update the compaction cursor(mark delete) first and then update the `compactionHorizon` of the compacted topic. During the compaction cursor move forward, the original ledger will be removed if no other durable cursors. At the same time, if the reader is reading data from the original ledger, the reader will skip the data while the original ledger been removed, details to see https://github.com/apache/pulsar/pull/6787. So the reader might skip the compacted data since the `compactionHorizon` have not updated yet.
   
   The approach is:
   
   1. Update the `compactionHorizon` before the compaction cursor move forward,
      so that the reader will not skip the original data before `compactionHorizon` updated.
      If the broker crashes before the new compacted Ledger ID been persistent,
      after the topic been loaded, the compaction can be trigger again and will not loss any data,
      but we will have an orphan ledger cannot be delete in the BookKeeper cluster.
   2. Remove the previous compacted Ledger after the compaction cursor move forward, make sure the new compacted Ledger ID been persistent,Otherwise, we might lost compacted ledger if broker crashes.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] Technoboy- merged pull request #16407: [Branch-2.7][Cherry-pick] Fix the reader skips compacted data which original ledger been removed.

Posted by GitBox <gi...@apache.org>.
Technoboy- merged PR #16407:
URL: https://github.com/apache/pulsar/pull/16407


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org