[jira] Commented: (HBASE-2315) BookKeeper for write-ahead logging

["Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12844564#action_12844564 ] 

Flavio Paiva Junqueira commented on HBASE-2315:
-----------------------------------------------

To compile hbase with this patch, it is necessary to add the bookkeeper jar to the lib folder. I've attached the jar to this jira for convenience, but you can also get it and compile from zookeeper/trunk.

This patch requires a few new configuration properties on hbase-site.xml:

{noformat}
 <property>
        <name>hbase.wal.bk.zkserver</name>
        <value>xxx.yyy.zzz:10281</value> 
</property>

<property>
       <name>hbase.regionserver.hlog.reader.impl</name>
       <value>org.apache.hadoop.hbase.regionserver.wal.BookKeeperLogReader</value>
 </property>

 <property>
       <name>hbase.regionserver.hlog.writer.impl</name>
       <value>org.apache.hadoop.hbase.regionserver.wal.BookKeeperLogWriter</value>
</property>
{noformat}

The zookeeper server (or servers) used with bookkeeper can be the same as the one used with hbase, but the value of the property has to be set regardless of which server you're using. If this is not ok, we may consider changing on the next iteration, assuming there will be one.

It is also important to create the following znodes in the zookeeper instance pointed by hbase.wal.bk.zkserver:

{noformat}
/ledgers
/ledgers/available/
/ledgers/available/<bookie-addr>
{noformat} 

where "/ledgers/available/<bookie-addr>" is a node representing a bookie and the format of <bookie-addr> should be "xxx.yyy.zzz:port". Consequently, there must be one such znode for each available bookie. We have a jira open to make bookie registration automatic, and it should be available soon. 

I have been creating these znodes manually for my tests, but I acknowledge that a script to bootstrap the process would be quite handy.

Finally, I would be happy to get some feedback and suggestions for improvement.

> BookKeeper for write-ahead logging
> ----------------------------------
>
>                 Key: HBASE-2315
>                 URL: https://issues.apache.org/jira/browse/HBASE-2315
>             Project: Hadoop HBase
>          Issue Type: New Feature
>          Components: regionserver
>            Reporter: Flavio Paiva Junqueira
>         Attachments: HBASE-2315.patch, zookeeper-dev-bookkeeper.jar
>
>
> BookKeeper, a contrib of the ZooKeeper project, is a fault tolerant and high throughput write-ahead logging service. This issue provides an implementation of write-ahead logging for hbase using BookKeeper. Apart from expected throughput improvements, BookKeeper also has stronger durability guarantees compared to the implementation currently used by hbase.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.