[GitHub] [bookkeeper] dlg99 commented on pull request #2635: Handle corrupted checkpoints

[GitBox <gi...@apache.org>]

dlg99 commented on pull request #2635:
URL: https://github.com/apache/bookkeeper/pull/2635#issuecomment-790271169


   @sursingh I'd like to clarify my understanding of specific scenario and inner working of the stream store:
   
   Store, i.e. RocksdbKVAsyncStore, writes op to a journal (distributed).
   periodically it checkpoint/writes local rocks db data to the distributed store (BK)
   At some point after checkpoint the store truncates journal.
   Last checkpoint + journal is enough to restore consistent dataset / full data
   
   Now if last checkpoint is corrupt we'll fallback to the previous checkpoint.
   It will pick up whatever journal available (`getDLSNNotLessThanTxId(checkpointedTxId)`) after the checkpoint
   It will not detect if journal data in the range `[checkpointedTxId, firstAvailableInJournalTxId]` is missing.
   The store gets data restored form an older checkpoint + journal that starts after the last (corrupt) checkpoint was created.
   Now the store has lost some data/has inconsistent data.
   
   Is my understanding correct? 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org