[jira] [Commented] (BOOKKEEPER-530) data might be lost during compaction.

["Ivan Kelly (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BOOKKEEPER-530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13542439#comment-13542439 ] 

Ivan Kelly commented on BOOKKEEPER-530:
---------------------------------------

I think it has to go through the journal, due to the issues we discussed on BOOKKEEPER-447. To recap, due to the design of LedgerCacheImpl, an index files may be flushed before the corresponding entrylog has been flushed. There is a possibility, that an index file is flushed but the entry logger is not and a crash occurs. The entry is now inaccessible, as the index file points to a bad location.

We can't use the stock Bookie#addEntry though, as this adds to the index before the journal, so the same issue could occur. We need an internal syncAddEntry which does the following sequence.
# add to Journal
# add to entryLogger
# add to ledgerIndex

                
> data might be lost during compaction.
> -------------------------------------
>
>                 Key: BOOKKEEPER-530
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-530
>             Project: Bookkeeper
>          Issue Type: Bug
>          Components: bookkeeper-server
>    Affects Versions: 4.1.0
>            Reporter: Sijie Guo
>             Fix For: 4.2.0
>
>
> {code}
>         try {
>             entryLogger.scanEntryLog(entryLogId, new CompactionScanner(entryLogMeta));
>             // after moving entries to new entry log, remove this old one
>             removeEntryLog(entryLogId);
>         } catch (IOException e) {
>             LOG.info("Premature exception when compacting " + entryLogId, e); 
>         } finally {
>             // clear compacting flag
>             compacting.set(false);
>         }
> {code}
> currently compaction code has a bit problem: as the code described above, old entry log is removed after new entries are added to new entry log, but new entry log might not be flushed. if failures happened after removal but before flush, data would be lost.
> when I implemented compaction feature in BOOKKEEPER-160, I remembered that I took care of letting entry go back to normal addEntry flow to reflect journal and index. But seems that the addEntry doesn't go thru journal, just move entries between entry log files w/o any flush guarantee.
> there are two ideas for this solution:
> simple one is to let compaction going to normal addEntry flow (adding entry to ledger storage and putting it in journal). the other one is GC thread either wait for ledger storage to flush in sync thread in one flush interval or force a ledger storage flush before removing entry log files.
> BTW, it was hard to design a test case by simulating bookie abnormally shut down itself after entry log files are removed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira