[jira] [Commented] (BOOKKEEPER-572) Make the journal a write ahead log

["Ivan Kelly (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BOOKKEEPER-572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622592#comment-13622592 ] 

Ivan Kelly commented on BOOKKEEPER-572:
---------------------------------------

New patch addresses the roll-logic issue. Unfortunately BOOKKEEPER-564's checkpoint changes didn't address the issue, as checkpointing needed to be completely overhauled as the order of ops changed. 

I've gotten rid of LastLogMark and replaced it with JournalMark which is an immutable object. The Journal holds the JournalMark. When the journal flushs, it creates a new journal mark object and flushes it. When a bookie adds an entry to the journal, it will first take the mark from the journal. When the callback occurs, and the bookie adds the entry to the ledger storage, then the mark is passed to the sync thread. Then, when the sync thread runs, it knows to what point of the journal has been stored in the ledger storage and uses it accordingly. The nice thing about this is that the journal knows very little about the mark, and the ledger storage doesn't need to know anything. The work is done by the bookie, and the sync thread, which are the entities interested in marking. 

There's a few more changes to be made, like moving the reading and writing of the mark out of Journal.java, but this is refactoring, so I don't want to do it in this patch.
                
> Make the journal a write ahead log
> ----------------------------------
>
>                 Key: BOOKKEEPER-572
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-572
>             Project: Bookkeeper
>          Issue Type: Bug
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 4.3.0
>
>         Attachments: 0001-BOOKKEEPER-572-Write-to-the-journal-before-writing-t.patch, 0003-BOOKKEEPER-572-Write-to-the-journal-before-writing-t.patch, 0003-BOOKKEEPER-572-Write-to-the-journal-before-writing-t.patch, BookieServer-2013-02-22.snapshot
>
>
> A bookie adds to the LedgerStorage before writing to the journal. This is the fundamental problem behind BOOKKEEPER-447 and blocks a nice solution to BOOKKEEPER-530. By writing to the memory state before the journal, we exposed ourselves to bugs if the bookie crashed before we wrote to the journal. The entry may exist in index, but not in the entrylog, a situation which cannot be distinguished from an I/O error. The comments on BOOKKEEPER-447 goes into more details. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira