Jackrabbit DBPersistenceManager

[Wolfgang Gehner <wg...@infonoia.com>]

Dear David, Dear Stefan, 

We are pleased to submit  to you in your role as Jackrabbit committers our DBPersistenceMgr
as potential contribution for Jackrabbit.

It's a checkout of the most recent project including our additions. It is 
available for download at http://www.infonoia.com/files/dbPersistenceMgrProjectInfonoia.zip (15.5MB)

It is hooked into the default workspace.xml, it also links to a readme.txt with db setup instructions.

It is a concept as to how relational database access could be implemented, benefitting from Apache iBatis
that helps map sql to code in a flexible way, and allows to keep sql outside the code. It uses PostgreSQL as database,
but can easily be plugged to virtually any relational db that has a working jdbc driver or a jndi DataSource.

Our suggestions on using the DAO pattern as we understand it for PersistenceManagers found elsewhere in the newsgroup
are an outgrowth of  this implementation; this implementation, however, uses the *current* PersistenceMgr api,
so it does *not* (yet) implement what we suggest in this regard. The db schema part of this is *not* in any way meant to be a recommended ER design, as you will see this can easily be modified, and improved on. All we wanted to do with the DbPersistenceMgr is to show a way we can CRUD to a or any DB via the JCR api. 

So far, this implementation should pass the *api* UnitTests, nothing else.

We would kindly ask you to review the code for inclusion in the Jackrabbit Source along with other persistence managers you have checked in. Please email your feedback directly to me.

Best regards,

Wolfgang Gehner

Infonoia S.A.
rue de Berne 7
1201 Geneva
Tel: +41 22 9000 009
Fax: +41 22 9000 018
wgehner@infonoia.com
http://www.infonoia.com

Sandboxing

[Stefano Mazzocchi <st...@apache.org>]

David Nuescheler wrote:

> another question that i have for the entire 
> jackrabbit-dev-community:
> since it seems like a number of people started to
> implement jackrabbit pms and other 
> extensions that maybe are not finished, should we 
> create a section of the svn tree dedicated to 
> experimental code in general? some sort of sandbox 
> or proposals. experiences from other projects?

one thing I strongly advise against, is the creation of "personal 
branches", these tend to increase the chance of "code ownership" which 
normally leads to friction in the community.

There are two ways of doing this:

  1) internal work area
  2) external work area

as an example, cocoon uses both (the first is called "scratchpad" the 
second is called "whiteboard"). The scratchpad is for little modules 
that hook into the same core, the whiteboard is for proposals that 
change the core.

None of these are "branches" because they are not guaranteed to make it 
into the trunk at any point in time, you can think of it as a mini 
incubator. Other projects call it "sandbox" (jakarta-commons, for example).

Personally, I would suggest to try to force everybody to work on the 
same tree at any given moment in time. If that is not possible, the 
internal work area should be preferred.

The reason for the above is that community building is done by making 
people talk and the more isolation the less people will have to confront 
until it's too late and incompatible decisions were made or lots of 
efforts were spent in development that the community might decide not o 
use (and this pisses people off, normally).

So, work incrementally, small patches and small discussions, one step at 
a time. As thermodynamics teaches, entropy is not increased with 
reversible steps. Try to make every step reversible, so that noise and 
disorder does not increase.

If that is not possible, try hard to minimize the impact and keep the 
communication channels open, even if, and especially, if this means 
making the life harder for the developers. Most of them believe that 
"continuous integration" is a waste of time and that "incremental 
reversible development" is uselessly harder.

They all change their mind sooner of later ;-)

-- 
Stefano.

Re: Jackrabbit DBPersistenceManager

[Wolfgang Gehner <wg...@infonoia.com>]

Hi David,

Thank you for your most welcome comments. I suggest we talk through them and
clarify certain items when we meet in Zurich tomorrow. I will then summarize
our discussion and next steps in this forum.

Best regards,

Wolfgang Gehner
www.infonoia.com

----- Original Message ----- 
From: "David Nuescheler" <da...@gmail.com>
To: <ja...@incubator.apache.org>
Sent: Thursday, November 18, 2004 12:51 PM
Subject: Re: Jackrabbit DBPersistenceManager


> hi wolfgang,
>
> thanks a lot for contributing jdbc pm (persistence manager).
> after taking a first look at the code that you submitted
> personally i would like to have at least the
> following questions answered or issues resolved before
> i personally could vote +1 on the contribution. i am
> convinced that all of those issues/questions can be
> easily fixed/answered:
>
> -------
> first of all there seems an administrational issue
> that as far as i understand with respect to the
> fact that the contribution is a seperable contribution,
> we would need a CLA on file from you. for a patch
> this would not be necessary (please anybody correct me
> if i am wrong)
>
> ---
> what is the benefit of the additional abstraction using
> dao, especially since the generic (abstract) datamodel
> is mandated by the implementation of the pm, shouldn't
> repository and jsr-170 provide such abstraction already?
> personally, i would probably favour a more direct
> access to the db.
>
> ---
> should the empty class "MyHashmap" be removed or renamed?
> as far as i understand it may be useful for future
> extensibility however the classname doesn't inspire a
> clear intention ;) ?
>
> ---
> since you submitted a 15mb package of the entire modified
> svn tree we have problems finding out what and why
> exactly was added / modified (do you think it would be
> possible to just send us a zip of your additions?)
>
> ---
> what is the thought behind creating a dom to pass
> arguments from one java method to another? was this
> just a quick hack or is that a pattern?
>
> ----
> to me it looks like your pm does not support multivalue
> properties, references or binary values (am i wrong?).
> what concerns me about that is that those are important
> functional pieces of a repository.
> stefan hacked up one afternoon quite a while ago
> a prototype for a database driven pm, that was
> able to support the full functional spectrum of a
> repository and being much more lightweight at
> the same time.
> we are currently having internal discussion if we
> should commit that as an example on how we see
> an implementation of a jdbc pm that is very
> lightweight, fast and simple. obviously it requires
> a very abstract generic datamodel in the rdbms
> too...
>
> ----
> generally, i believe that the earlier proposed
> changes to the pm api, with respect to allow a
> pm to operate in a transactional
> fashion, as you suggested, will have quite
> a big impact. so i think it maybe still be
> worth waiting a little bit, until the pm api
> has "settled again" ;)
>
> -------
> this is probably not a complete list of
> questions/comments but it could be a
> starting point.
>
> another question that i have for the entire
> jackrabbit-dev-community:
> since it seems like a number of people started to
> implement jackrabbit pms and other
> extensions that maybe are not finished, should we
> create a section of the svn tree dedicated to
> experimental code in general? some sort of sandbox
> or proposals. experiences from other projects?
>
> additionally, since everybody received wolfgangs
> contribution, i would like to encourage everybody
> to use it and share their experiences.
>
> regards,
> david

Re: Jackrabbit DBPersistenceManager

[Wolfgang Gehner <wg...@infonoia.com>]

Summary of our discussion inline...

----- Original Message ----- 
From: "David Nuescheler" <da...@gmail.com>
To: <ja...@incubator.apache.org>
Sent: Thursday, November 18, 2004 12:51 PM
Subject: Re: Jackrabbit DBPersistenceManager


> hi wolfgang,
>
> thanks a lot for contributing jdbc pm (persistence manager).
> after taking a first look at the code that you submitted
> personally i would like to have at least the
> following questions answered or issues resolved before
> i personally could vote +1 on the contribution. i am
> convinced that all of those issues/questions can be
> easily fixed/answered:
>
> -------
> first of all there seems an administrational issue
> that as far as i understand with respect to the
> fact that the contribution is a seperable contribution,
> we would need a CLA on file from you. for a patch
> this would not be necessary (please anybody correct me
> if i am wrong)

No problem, we can do that; when we do a patch (see below) that may not
be necessary.

>
> ---
> what is the benefit of the additional abstraction using
> dao, especially since the generic (abstract) datamodel
> is mandated by the implementation of the pm, shouldn't
> repository and jsr-170 provide such abstraction already?
> personally, i would probably favour a more direct
> access to the db.

We agreed that in most ways, (Persistent)Node already "is" a
DAO. So we don't propose to add an additional abstraction.

>
> ---
> should the empty class "MyHashmap" be removed or renamed?
> as far as i understand it may be useful for future
> extensibility however the classname doesn't inspire a
> clear intention ;) ?

We agreed to rename to a better name (Row?, MappedItem? we have to think).
We might show the extension options of the use of this class (use any bean
or possibly even PersistentNodeState instead of extensions of HashMap) in a
comment somewhere.

>
> ---
> since you submitted a 15mb package of the entire modified
> svn tree we have problems finding out what and why
> exactly was added / modified (do you think it would be
> possible to just send us a zip of your additions?)

We agreed that we will do a patch as you request.

>
> ---
> what is the thought behind creating a dom to pass
> arguments from one java method to another? was this
> just a quick hack or is that a pattern?

We just created an Element, not a full DOM, and only to reuse your method
signature
which uses an Element. There is no real justification, we can pass the
params as HashMap no sweat.

>
> ----
> to me it looks like your pm does not support multivalue
> properties, references or binary values (am i wrong?).
> what concerns me about that is that those are important
> functional pieces of a repository

We agreed that we would attempt to add these features once you provide the
corresponding unit tests, that way the tests wouldn't have to be written
twice.
You know that blob support can be very database-specific, and datatype use
can vary from application to application (may keep html in longvarchars, but
multi-mb pdf's in datatype blob as supported by each database platform.
IBatis uses
interceptors to allow plugging different blob handlings.

> we are currently having internal discussion if we
> should commit that as an example on how we see
> an implementation of a jdbc pm that is very
> lightweight, fast and simple. obviously it requires
> a very abstract generic datamodel in the rdbms
> too...

DBPersistence layer should be flexible to allow legacy data models, and
tweaking of model (as to blob handling, user rights storage, "flatness" of
node properties storage etc.) to be useful in multiple environments and
applications.

>
> ----
> generally, i believe that the earlier proposed
> changes to the pm api, with respect to allow a
> pm to operate in a transactional
> fashion, as you suggested, will have quite
> a big impact. so i think it maybe still be
> worth waiting a little bit, until the pm api
> has "settled again" ;)

You said you would spend the next week or couple of weeks redesigning the
Persistence manager api etc.
and the result is likely to support transactional multi-row
update/delete/insert among other things. We will be pleased to
provide feedback on that, and then as a next step would do an implementation
that could be committed.

>
> -------
> this is probably not a complete list of
> questions/comments but it could be a
> starting point.
>
> another question that i have for the entire
> jackrabbit-dev-community:
> since it seems like a number of people started to
> implement jackrabbit pms and other
> extensions that maybe are not finished, should we
> create a section of the svn tree dedicated to
> experimental code in general? some sort of sandbox
> or proposals. experiences from other projects?
>
> additionally, since everybody received wolfgangs
> contribution, i would like to encourage everybody
> to use it and share their experiences.
>
> regards,
> david

Best regards,

Wolfgang

Re: Jackrabbit DBPersistenceManager

[David Nuescheler <da...@gmail.com>]

hi wolfgang,

thanks a lot for contributing jdbc pm (persistence manager).
after taking a first look at the code that you submitted
personally i would like to have at least the 
following questions answered or issues resolved before
i personally could vote +1 on the contribution. i am 
convinced that all of those issues/questions can be 
easily fixed/answered:

-------
first of all there seems an administrational issue 
that as far as i understand with respect to the 
fact that the contribution is a seperable contribution,
we would need a CLA on file from you. for a patch
this would not be necessary (please anybody correct me 
if i am wrong)

---
what is the benefit of the additional abstraction using
dao, especially since the generic (abstract) datamodel
is mandated by the implementation of the pm, shouldn't 
repository and jsr-170 provide such abstraction already? 
personally, i would probably favour a more direct 
access to the db.

---
should the empty class "MyHashmap" be removed or renamed?
as far as i understand it may be useful for future
extensibility however the classname doesn't inspire a
clear intention ;) ?

---
since you submitted a 15mb package of the entire modified
svn tree we have problems finding out what and why
exactly was added / modified (do you think it would be
possible to just send us a zip of your additions?)

---
what is the thought behind creating a dom to pass
arguments from one java method to another? was this
just a quick hack or is that a pattern?

----
to me it looks like your pm does not support multivalue 
properties, references or binary values (am i wrong?). 
what concerns me about that is that those are important 
functional pieces of a repository. 
stefan hacked up one afternoon quite a while ago 
a prototype for a database driven pm, that was 
able to support the full functional spectrum of a 
repository and being much more lightweight at 
the same time.
we are currently having internal discussion if we
should commit that as an example on how we see
an implementation of a jdbc pm that is very 
lightweight, fast and simple. obviously it requires 
a very abstract generic datamodel in the rdbms 
too...

----
generally, i believe that the earlier proposed 
changes to the pm api, with respect to allow a 
pm to operate in a transactional 
fashion, as you suggested, will have quite 
a big impact. so i think it maybe still be
worth waiting a little bit, until the pm api 
has "settled again" ;)

-------
this is probably not a complete list of 
questions/comments but it could be a 
starting point.

another question that i have for the entire 
jackrabbit-dev-community:
since it seems like a number of people started to
implement jackrabbit pms and other 
extensions that maybe are not finished, should we 
create a section of the svn tree dedicated to 
experimental code in general? some sort of sandbox 
or proposals. experiences from other projects?

additionally, since everybody received wolfgangs
contribution, i would like to encourage everybody
to use it and share their experiences.

regards,
david