persistence

[Frank Lynch <fr...@iona.com>]

I'd like to start working on a persistence solution for Qpid that will
be compatible with our apache licensed implementation. I've been
thinking about iBatis + Derby for this. I like the flexibility that
iBatis brings, as it allows users to tweak the database table
format/schema if necessary and its easy to slot in an enterprise
strength database (like Oracle, Sybase etc) in leiu of Derby.   
 
Has anyone any opinions or thoughts on this, or should I just start
hacking away at an iBatis based persistence implementation?  
cheers,
--Frank

RE: persistence

[Colin Crist <co...@hermesjms.com>]

A while back I spec'ed out an interface for a MessageStore as a generic JMS
message store to improve on the JMS store I have in Hermes. It never got
past the interface stage and is incomplete but hey.

The idea was that it was a hierarchy of regions containing queues. The
implementaion could then map those regions to different tables or even
databases on the server or if it was not an SQL implementation then memory
mapped files or whatever.

In the following code, you'd have to imagine the JMS message was an
AMQMessage and the destination some kind of internal AMQ queue. You can
ignore the idea of messages having an index, that was to support replay and
is not relevant.

http://hermesjms.cvs.sourceforge.net/hermesjms/MessageStore/source/java/herm
es/store/

I like the idea of regions - in QPID an exchange could have its own region. 

Anyway, I thought it may be something useful to share...

Regards,

Colin.
http://hermesjms.com

-----Original Message-----
From: Alan Conway [mailto:aconway@redhat.com] 
Sent: 08 September 2006 15:28
To: qpid-dev@incubator.apache.org
Subject: Re: persistence

On Fri, 2006-09-08 at 09:56 -0400, Rajith Attapattu wrote:
> I agree with Rafeal, and we need to have a very simple persistant API.
> But then again O/R mappers are in style and end users may prefer 
> things like hibernate, ibatis etc..
> And they maybe convinent to work with.

People will definitely want to change the DB backend, but I predict that no
Qpid user will ever care *how* we get the data into the database, they will
only care:
 - Can I use my favorite database X?
 - Is it fast?

I'm guessing JDBC is all the pluggability we need, so the only reason to use
an O/R mapper is if we think it's a productivity win for us and it doesn't
have significant performance impact. I don't think the Qpid persistence
schemas are going to be very complex so I'm not sure a O/R mapper gives us
much, but I don't know the Java ones well enough to comment further.


'nuff said, back to C++ :)

Re: persistence

[Alan Conway <ac...@redhat.com>]

On Fri, 2006-09-08 at 09:56 -0400, Rajith Attapattu wrote:
> I agree with Rafeal, and we need to have a very simple persistant API.
> But then again O/R mappers are in style and end users may prefer things like
> hibernate, ibatis etc..
> And they maybe convinent to work with.

People will definitely want to change the DB backend, but I predict that
no Qpid user will ever care *how* we get the data into the database,
they will only care:
 - Can I use my favorite database X?
 - Is it fast?

I'm guessing JDBC is all the pluggability we need, so the only reason to
use an O/R mapper is if we think it's a productivity win for us and it
doesn't have significant performance impact. I don't think the Qpid
persistence schemas are going to be very complex so I'm not sure a O/R
mapper gives us much, but I don't know the Java ones well enough to
comment further.


'nuff said, back to C++ :)

Re: persistence

[Rajith Attapattu <ra...@gmail.com>]

I also strongly advocate a pluggble persistant implementation as users may
have a very wide variety of choices for their backend.

I agree with Rafeal, and we need to have a very simple persistant API.
But then again O/R mappers are in style and end users may prefer things like
hibernate, ibatis etc..
And they maybe convinent to work with.

But keeping performance in mind it maybe handy to be close to the data base
as much as possible with raw SQL.
Another layer in btw means more mapping.

How about a proposal on the wiki about the abstraction layer?

Regards,

Rajith

On 9/7/06, Carl Trieloff <cc...@redhat.com> wrote:
>
>
> Frank,
>
> It looks like it would be very simple to create a module that support
> JDBC for the Java side. This
> would mean that anyone can use anything they chose. I would think this
> would be a great first
> step, and then add specific additional modules for other OSS BD's
> projects if desired.
>
> Carl.
>
> Juergen Donnerstag wrote:
> > May I suugest that you make it as pluggable as possible to allow for
> > other implementations (whatever that will be).
> >
> > Juergen
> >
> > On 9/6/06, Frank Lynch <fr...@iona.com> wrote:
> >> I'd like to start working on a persistence solution for Qpid that will
> >> be compatible with our apache licensed implementation. I've been
> >> thinking about iBatis + Derby for this. I like the flexibility that
> >> iBatis brings, as it allows users to tweak the database table
> >> format/schema if necessary and its easy to slot in an enterprise
> >> strength database (like Oracle, Sybase etc) in leiu of Derby.
> >>
> >> Has anyone any opinions or thoughts on this, or should I just start
> >> hacking away at an iBatis based persistence implementation?
> >> cheers,
> >> --Frank
> >>
> >>
>
>

Re: persistence

[Carl Trieloff <cc...@redhat.com>]

Frank,

It looks like it would be very simple to create a module that support 
JDBC for the Java side. This
would mean that anyone can use anything they chose. I would think this 
would be a great first
step, and then add specific additional modules for other OSS BD's 
projects if desired.

Carl.

Juergen Donnerstag wrote:
> May I suugest that you make it as pluggable as possible to allow for
> other implementations (whatever that will be).
>
> Juergen
>
> On 9/6/06, Frank Lynch <fr...@iona.com> wrote:
>> I'd like to start working on a persistence solution for Qpid that will
>> be compatible with our apache licensed implementation. I've been
>> thinking about iBatis + Derby for this. I like the flexibility that
>> iBatis brings, as it allows users to tweak the database table
>> format/schema if necessary and its easy to slot in an enterprise
>> strength database (like Oracle, Sybase etc) in leiu of Derby.
>>
>> Has anyone any opinions or thoughts on this, or should I just start
>> hacking away at an iBatis based persistence implementation?
>> cheers,
>> --Frank
>>
>>

Re: persistence

[Juergen Donnerstag <ju...@gmail.com>]

May I suugest that you make it as pluggable as possible to allow for
other implementations (whatever that will be).

Juergen

On 9/6/06, Frank Lynch <fr...@iona.com> wrote:
> I'd like to start working on a persistence solution for Qpid that will
> be compatible with our apache licensed implementation. I've been
> thinking about iBatis + Derby for this. I like the flexibility that
> iBatis brings, as it allows users to tweak the database table
> format/schema if necessary and its easy to slot in an enterprise
> strength database (like Oracle, Sybase etc) in leiu of Derby.
>
> Has anyone any opinions or thoughts on this, or should I just start
> hacking away at an iBatis based persistence implementation?
> cheers,
> --Frank
>
>

Re: persistence

[Rafael Schloming <ra...@redhat.com>]

We're definitely going to need a generic RDB-backed persistence solution 
at some point, so Derby would be a great place to start. It should only 
take about two tables and a handful of vanilla SQL queries to replicate 
what the bdb back-end is doing. Given this I suspect using an O/R layer 
might be overkill and could prove to be slower than using JDBC directly.

--Rafael

Frank Lynch wrote:
> I'd like to start working on a persistence solution for Qpid that will
> be compatible with our apache licensed implementation. I've been
> thinking about iBatis + Derby for this. I like the flexibility that
> iBatis brings, as it allows users to tweak the database table
> format/schema if necessary and its easy to slot in an enterprise
> strength database (like Oracle, Sybase etc) in leiu of Derby.   
>  
> Has anyone any opinions or thoughts on this, or should I just start
> hacking away at an iBatis based persistence implementation?  
> cheers,
> --Frank
> 

RE: persistence

[Colin Crist <co...@hermesjms.com>]

I'm only an observer on this list but I'd recommend looking at HOWL as a
high speed way to asynchonously spool state into a relational database (or
other store). This is probably easier to do now rather than later given the
recovery implications. 

http://howl.objectweb.org/ 

Also, FWIW (as I say I am only an observer!) it may be better to shy away
from any O/R layer and keep as close to the database as you can. No doubt
the abstractions you'll put in place however will give you a multitude of
options for how to map to the final store, its these interfaces that would
be good to stabalise now.

Regards,

Colin.
http://hermesjms.com

-----Original Message-----
From: Frank Lynch [mailto:frank@iona.com] 
Sent: 06 September 2006 21:30
To: qpid-dev@incubator.apache.org
Subject: persistence

I'd like to start working on a persistence solution for Qpid that will be
compatible with our apache licensed implementation. I've been thinking about
iBatis + Derby for this. I like the flexibility that iBatis brings, as it
allows users to tweak the database table format/schema if necessary and its
easy to slot in an enterprise
strength database (like Oracle, Sybase etc) in leiu of Derby.   
 
Has anyone any opinions or thoughts on this, or should I just start hacking
away at an iBatis based persistence implementation?  
cheers,
--Frank