[jira] [Commented] (HBASE-2315) BookKeeper for write-ahead logging

["Flavio Junqueira (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399179#comment-13399179 ] 

Flavio Junqueira commented on HBASE-2315:
-----------------------------------------

Apologies for not being very active here. Yesterday we had a chat with Stack about moving forward with this jira. Here are a few key points I got from the discussion:

* To use the reader and writer interfaces, we need to implement a BookKeeper filesystem because the init method expects such an object;
* The RegionServer maintains a list of existing logs and we need to map them to ledgers. The easiest way is probably to maintain this map in ZooKeeper and manage it through the Filesystem implementation; 
* Currently there is one single FileSystem object in the RegionServer and we would need to have at least two, one being used for BookKeeper;
* Stack suggested that it would be a great idea to to have logs per region with BookKeeper. BookKeeper is designed exactly to serve use cases that have multiple concurrent logs being written to. According to Stack, this feature has the potential of reducing time to recover significantly. Enabling this feature might require some more changes to the wal interface, though, since we would need to separate the edits according to region. (Related to HBASE-4529 ?) 

Let us know if anyone has a comment, and otherwise we will push it forward.
                
> BookKeeper for write-ahead logging
> ----------------------------------
>
>                 Key: HBASE-2315
>                 URL: https://issues.apache.org/jira/browse/HBASE-2315
>             Project: HBase
>          Issue Type: New Feature
>          Components: regionserver
>            Reporter: Flavio Junqueira
>         Attachments: HBASE-2315.patch, bookkeeperOverview.pdf, zookeeper-dev-bookkeeper.jar
>
>
> BookKeeper, a contrib of the ZooKeeper project, is a fault tolerant and high throughput write-ahead logging service. This issue provides an implementation of write-ahead logging for hbase using BookKeeper. Apart from expected throughput improvements, BookKeeper also has stronger durability guarantees compared to the implementation currently used by hbase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira