[jira] [Comment Edited] (BOOKKEEPER-564) Better checkpoint mechanism

["Sijie Guo (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BOOKKEEPER-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638789#comment-13638789 ] 

Sijie Guo edited comment on BOOKKEEPER-564 at 4/23/13 5:21 AM:
---------------------------------------------------------------

[~ikelly] 

I already commented on your patch in previous comment.

1) you moved LogMark to ledger storage, which makes journal contructor "Journal(conf, logmark)" behavior unclear, which looks like whatever the caller that give a mark, the journal would replay starting from the given mark. but it couldn't, since journal gc its files. It is against what you said Journal is a independent component.

2) you are making ledger storage too ambitious. sync thread (checkpointing) logic should be maintained by Bookie itself, as the sync(checkpointing) logic is part of bookie not ledger storage, it should be common across different ledger storage implementations. And, [~rakeshr] yes, I agreed with you.

> The LogMark storage is managed by LedgerStorage
> Journal doesn't even need the ledger storage dirs anymore.

as I pointed at 1), making LogMark as a part to journal would make Journal clearer on the replaying behavior. Although we used ledger directories in Journal, it doesn't mean that LogMark is part of ledger storage. The directories recording LogMark could be any directory. the only thing why not putting the journal directory is not to affect the performance of journal.
                
      was (Author: hustlmsp):
    [~ikelly] 

I already commented on your patch in previous comment.

1) you moved LogMark to ledger storage, which makes journal contructor "Journal(conf, logmark)" behavior unclear, which looks like whatever the caller that give a mark, the journal would replay starting from the given mark. but it couldn't, since journal gc its files.

2) you are making ledger storage too ambitious. sync thread (checkpointing) logic should be maintained by Bookie itself, as the sync(checkpointing) logic is part of bookie not ledger storage, it should be common across different ledger storage implementations. And, [~rakeshr] yes, I agreed with you.

> The LogMark storage is managed by LedgerStorage
> Journal doesn't even need the ledger storage dirs anymore.

as I pointed at 1), making LogMark as a part to journal would make Journal clearer on the replaying behavior. Although we used ledger directories in Journal, it doesn't mean that LogMark is part of ledger storage. The directories recording LogMark could be any directory. the only thing why not putting the journal directory is not to affect the performance of journal.
                  
> Better checkpoint mechanism
> ---------------------------
>
>                 Key: BOOKKEEPER-564
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-564
>             Project: Bookkeeper
>          Issue Type: Improvement
>          Components: bookkeeper-server
>            Reporter: Sijie Guo
>            Assignee: Sijie Guo
>             Fix For: 4.3.0
>
>         Attachments: 0001-BOOKKEEPER-564-Better-checkpoint-mechanism.patch, 0001-BOOKKEEPER-564-Better-checkpoint-mechanism.patch, 0002-BOOKKEEPER-564-Better-checkpoint-mechanism.patch, BOOKKEEPER-564.patch, BOOKKEEPER-564.patch
>
>
> Currently, SyncThread made a checkpoint too frequently, which affects performance. data is writing to entry logger file might be blocked by syncing same entry logger file, which affect bookie to achieve higher throughput. We could schedule checkpoint only when rotating an entry log file. so new incoming entries would be written to newer entry log file and old entry log file could be synced.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira