[jira] [Commented] (BOOKKEEPER-220) Managed Ledger proposal

["Ivan Kelly (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BOOKKEEPER-220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13289562#comment-13289562 ] 

Ivan Kelly commented on BOOKKEEPER-220:
---------------------------------------

I've just taken a look a the code. I like it a lot. It's very nice, neat and readable. I like the direction it's going. I have a few concerns though.

My biggest concern is that the implementation doesn't take precautions so that split brain does not occur. If a managed ledger is being used as a write ahead log for node A, and node B falsely suspects that A is down, it can start writing to the the write ahead log for A and there's nothing to stop them. This isn't very difficult to prevent, but it needs to be done in your code. You need to use versioned zookeeper writes in the metastore, and ensure that when you create a ledger, all previous ledgers are closed.

Another potential issue in the future is that the async and sync versions of addEntry do not share a code path. This could lead to bugs in the future which get fixed on one and not on the other. It would be better to have a completely async version, and then implement the sync version by calling the async and waiting on a countdown latch. Also, I think it would be nicer to have asyncAddEntry using the async calls and callbacks rather than a worker thread OR if using the worker threads, using the actor pattern. I prefer the former, as it will perform better as more requests can be outstanding at a time, but the latter works too.

Instead of having managed cursor return a list of entries, how about having managed cursor itself present a iterator type interface where you can keep calling next() to get the next entry?

One last small thing, is that it would be better to use a custom exception, rather than java.lang.Exception. 



                
> Managed Ledger proposal
> -----------------------
>
>                 Key: BOOKKEEPER-220
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-220
>             Project: Bookkeeper
>          Issue Type: New Feature
>          Components: bookkeeper-client
>            Reporter: Matteo Merli
>            Assignee: Matteo Merli
>             Fix For: 4.2.0
>
>         Attachments: 0001-BOOKKEEPER-220-Managed-Ledger-proposal.patch, 0001-BOOKKEEPER-220-Managed-Ledger-proposal.patch
>
>
> The ManagedLedger design is based on our need to manage a set of ledgers, with a single writer (at any point in time) and a set on consumers that read entries from it. 
> The ManagedLedger also takes care of periodically closing ledgers to have a "reasonable" sized sets of ledgers that can individually deleted when no more needed.
> I've put on github the interface proposal (along with an early WIP implementation)
> http://github.com/merlimat/managed-ledger

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira