Hbase as a message store for QPID

[Vijay Devadhar <vd...@salesforce.com>]

Hello QPID users,

Has anyone tried to build or thought about building a Hbase message_store adopter for QPID?
By that I mean, instead of persisting to BDB or Derby DB,  storing into Hbase. Would there be
fundamental reasons (like sub optimal data structures) to not do that?

Thanks
Vijay

RE: Hbase as a message store for QPID

[Vijay Devadhar <vd...@salesforce.com>]

Thank you Robbie and Rob. Good point about lack of transactions in HBase.
That is the kind of info I was looking for.

Rob -- in one of our application stacks, HBase is the only "system of records", by which
I mean it is the only system with DR and HA built in. If we store queue data in
HBase, it will be easier for us to maintain DR and HA for us if we were to store the 
queue data on it as well;

-----Original Message-----
From: Rob Godfrey [mailto:rob.j.godfrey@gmail.com] 
Sent: Monday, March 25, 2013 7:08 AM
To: users@qpid.apache.org
Subject: Re: Hbase as a message store for QPID

I guess my question would be - what is the use case for Hbase?

The way that the Qpid Java Broker uses the store is basically "write only"
during normal operation... it never queries the store to get information unless it has entered a low memory condition and is relying on the store as a kind of poor man's flow-to-disk.  The store is only normally read at startup in order to restore the broker state to how it was before it last shutdown.

As such using something like HBase doesn't immediately provide any clear advantages.

Where I could see something like HBase being useful is if we re-engineered the broker to have a distributed queue model where brokers could cluster and use the store as the shared state for the queue.

Cheers,
Rob

On 25 March 2013 13:46, Robbie Gemmell <ro...@gmail.com> wrote:

> Hi Vijay,
>
> I'm not aware of anyone building an HBase message store. Ultimately 
> anything that satisfies the expected behaviour of the store interface 
> can be used. I think the most interesting issue with use of the 
> distributed NoSQL DBs such as HBase would be the lack of inherant 
> transactions; I guess it can be mimmicked with clever use of data 
> structures, though that will obviously complicate implementation 
> somewhat compared to the existing stores.
>
> Robbie
>
> On 20 March 2013 18:37, Vijay Devadhar <vd...@salesforce.com> wrote:
>
> > Hello QPID users,
> >
> > Has anyone tried to build or thought about building a Hbase 
> > message_store adopter for QPID?
> > By that I mean, instead of persisting to BDB or Derby DB,  storing 
> > into Hbase. Would there be fundamental reasons (like sub optimal 
> > data structures) to not do that?
> >
> > Thanks
> > Vijay
> >
> >
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

Re: Hbase as a message store for QPID

[Rob Godfrey <ro...@gmail.com>]

I guess my question would be - what is the use case for Hbase?

The way that the Qpid Java Broker uses the store is basically "write only"
during normal operation... it never queries the store to get information
unless it has entered a low memory condition and is relying on the store as
a kind of poor man's flow-to-disk.  The store is only normally read at
startup in order to restore the broker state to how it was before it last
shutdown.

As such using something like HBase doesn't immediately provide any clear
advantages.

Where I could see something like HBase being useful is if we re-engineered
the broker to have a distributed queue model where brokers could cluster
and use the store as the shared state for the queue.

Cheers,
Rob

On 25 March 2013 13:46, Robbie Gemmell <ro...@gmail.com> wrote:

> Hi Vijay,
>
> I'm not aware of anyone building an HBase message store. Ultimately
> anything that satisfies the expected behaviour of the store interface can
> be used. I think the most interesting issue with use of the distributed
> NoSQL DBs such as HBase would be the lack of inherant transactions; I guess
> it can be mimmicked with clever use of data structures, though that will
> obviously complicate implementation somewhat compared to the existing
> stores.
>
> Robbie
>
> On 20 March 2013 18:37, Vijay Devadhar <vd...@salesforce.com> wrote:
>
> > Hello QPID users,
> >
> > Has anyone tried to build or thought about building a Hbase message_store
> > adopter for QPID?
> > By that I mean, instead of persisting to BDB or Derby DB,  storing into
> > Hbase. Would there be
> > fundamental reasons (like sub optimal data structures) to not do that?
> >
> > Thanks
> > Vijay
> >
> >
>

Re: Hbase as a message store for QPID

[Robbie Gemmell <ro...@gmail.com>]

Hi Vijay,

I'm not aware of anyone building an HBase message store. Ultimately
anything that satisfies the expected behaviour of the store interface can
be used. I think the most interesting issue with use of the distributed
NoSQL DBs such as HBase would be the lack of inherant transactions; I guess
it can be mimmicked with clever use of data structures, though that will
obviously complicate implementation somewhat compared to the existing
stores.

Robbie

On 20 March 2013 18:37, Vijay Devadhar <vd...@salesforce.com> wrote:

> Hello QPID users,
>
> Has anyone tried to build or thought about building a Hbase message_store
> adopter for QPID?
> By that I mean, instead of persisting to BDB or Derby DB,  storing into
> Hbase. Would there be
> fundamental reasons (like sub optimal data structures) to not do that?
>
> Thanks
> Vijay
>
>