[GitHub] ArvinDevel opened a new pull request #927: BP-24: BookieScanner: Enhance Data Integrity

[GitBox <gi...@apache.org>]

ArvinDevel opened a new pull request #927: BP-24: BookieScanner: Enhance Data Integrity
URL: https://github.com/apache/bookkeeper/pull/927
 
 
   Descriptions of the changes in this PR:
   
   Currently Bookie can't deal entry losing gracefully, the AutoRecovery is restricted to the bookie level, which means the AutoRecovery takes effect only after bookie is down. However when a disk fails, either or both the ledger index files and entry log files could potentially become corrupt. BookKeeper needs to provide mechanisms to identify and handle these problems.
   
   In this BP, we introduce Bookie Scanner, which is a background task, to scan index files and entry log files to detect possible corruptions. Since data corruption may happen at any time on any block on any Bookie, it is important to identify these errors in a timely manner. This way, the bookie can remove/compact corrupted entries and re-replicate entries from other replicas, to maintain data integrity and reduce client errors. 
   
   Master Issue: #<master-issue-number>
   
   > ---
   > Be sure to do all of the following to help us incorporate your contribution
   > quickly and easily:
   >
   > If this PR is a BookKeeper Proposal (BP):
   >
   > - [ ] Make sure the PR title is formatted like:
   >     `<BP-#>: Description of bookkeeper proposal`
   >     `e.g. BP-1: 64 bits ledger is support`
   > - [ ] Attach the master issue link in the description of this PR.
   > - [ ] Attach the google doc link if the BP is written in Google Doc.
   >
   > Otherwise:
   > 
   > - [ ] Make sure the PR title is formatted like:
   >     `<Issue # or BOOKKEEPER-#>: Description of pull request`
   >     `e.g. Issue 123: Description ...`
   >     `e.g. BOOKKEEPER-1234: Description ...`
   > - [ ] Make sure tests pass via `mvn clean apache-rat:check install spotbugs:check`.
   > - [ ] Replace `<Issue # or BOOKKEEPER-#>` in the title with the actual Issue/JIRA number.
   > 
   > ---
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services