You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Steven Noels <st...@outerthought.org> on 2010/06/21 17:11:46 UTC
HBase RowLog Library
Hi all,
as promised, here's our take at a component to build queues and WALs that
run on top of HBase.
Available under Apache license from
http://lilycms.org/lily/roadmap/sketchbook/hbaserowlog.html thanks to the
hard work of my colleagues Evert and Bruno.
Rationale:
HBase provides atomic row-level updates. This is one of the important
reasons why we choose HBase <http://hbase.apache.org/> as the underlying
store for our upcoming Lily <http://www.lilycms.org/> open source content
repository, as consistency is very important for users trusting their data
onto our software. Next, we needed some way to rapidly locate HBase-stored
data without key-based access, so we came up with the idea of building
secondary
indexes on top of
HBase<http://lilycms.org/lily/roadmap/sketchbook/hbaseindexes.html>.
Ultimately, we'll also provide a flexible user-level search powered by
SOLR<http://lilycms.org/lily/prerelease/architecture.html>,
which will require us to maintain a frequently (near-real-time) updated SOLR
index alongside our HBase-backed main data store.
All this required us to have a flexible way to facilitate synchronous
actions across multiple HBase tables, and asynchronous queuing of index
updates for SOLR - in a distributed architecture with no real single point
of failure. We've named this the* HBase RowLog library*, and we're releasing
a first version of this RowLog library today.
Enjoy and let us know what you think of it.
Evert is on a well-deserved holiday chasing Pearl Jam concerts throughout
Europe, so you're best of sending remarks here.
Cheers,
Steven
--
Steven Noels
http://outerthought.org/
Open Source Content Applications
Makers of Kauri, Daisy CMS and Lily
Re: HBase RowLog Library
Posted by Steven Noels <st...@outerthought.org>.
On Mon, Jun 21, 2010 at 9:54 PM, Stack <st...@duboce.net> wrote:
> Very nice. I haven't tried it but if the doc is anything to go by,
> its going to work real sweet.
>
> Please be sure to add it to supporting project Steven (I need to
> reorg. this page so the newest stuff appears first).
>
Just done so, thanks for reminding. I see momentum is slowly increasing
towards HBase 1.0, we'll make sure to track trunk development.
Cheers,
Steven.
--
Steven Noels http://outerthought.org/
Open Source Content Applications
Makers of Kauri, Daisy CMS and Lily
Re: HBase RowLog Library
Posted by Stack <st...@duboce.net>.
Very nice. I haven't tried it but if the doc is anything to go by,
its going to work real sweet.
Please be sure to add it to supporting project Steven (I need to
reorg. this page so the newest stuff appears first).
St.Ack
On Mon, Jun 21, 2010 at 8:11 AM, Steven Noels <st...@outerthought.org> wrote:
> Hi all,
>
> as promised, here's our take at a component to build queues and WALs that
> run on top of HBase.
>
> Available under Apache license from
> http://lilycms.org/lily/roadmap/sketchbook/hbaserowlog.html thanks to the
> hard work of my colleagues Evert and Bruno.
>
> Rationale:
>
> HBase provides atomic row-level updates. This is one of the important
> reasons why we choose HBase <http://hbase.apache.org/> as the underlying
> store for our upcoming Lily <http://www.lilycms.org/> open source content
> repository, as consistency is very important for users trusting their data
> onto our software. Next, we needed some way to rapidly locate HBase-stored
> data without key-based access, so we came up with the idea of building
> secondary
> indexes on top of
> HBase<http://lilycms.org/lily/roadmap/sketchbook/hbaseindexes.html>.
> Ultimately, we'll also provide a flexible user-level search powered by
> SOLR<http://lilycms.org/lily/prerelease/architecture.html>,
> which will require us to maintain a frequently (near-real-time) updated SOLR
> index alongside our HBase-backed main data store.
>
> All this required us to have a flexible way to facilitate synchronous
> actions across multiple HBase tables, and asynchronous queuing of index
> updates for SOLR - in a distributed architecture with no real single point
> of failure. We've named this the* HBase RowLog library*, and we're releasing
> a first version of this RowLog library today.
>
> Enjoy and let us know what you think of it.
>
> Evert is on a well-deserved holiday chasing Pearl Jam concerts throughout
> Europe, so you're best of sending remarks here.
> Cheers,
>
> Steven
> --
> Steven Noels
> http://outerthought.org/
> Open Source Content Applications
> Makers of Kauri, Daisy CMS and Lily
>