You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by Serge Huber <sh...@jahia.com> on 2005/01/19 16:12:03 UTC
PersistenceManager & childNodeEntries
Hi all,
First of all let me quickly say thank you to all the great work that
went into Jackrabbit and the JSR-170. I realize it must have been highly
political and possibly frustrating, but the result is quite interesting,
and I think that this will help a lot of developers as well as end users.
Anyway, I've been working on evaluating Jackrabbit (as many), and I've
started writing back-end implementations for Apache OJB
(http://db.apache.org/ojb) and Hibernate (http://www.hibernate.org). I
was wondering if somebody already had a go at it, and if not, I'd be
more than willing to contribute the little I've done, despite it being
very hacky.
I've got one problem though : when removing nodes, Jackrabbit calls the
PersistenceManager.destroy(PersistentNodeState) method. What happens
with the childNodeEntries of the parent objects ? It seems that they are
not modified and that Jackrabbit still maintains children that no longer
exist. Is this normal ? Did I miss something ?
Regards,
Serge Huber.
Re: PersistenceManager & childNodeEntries
Posted by Serge Huber <sh...@jahia.com>.
Ooops, seems like the attachement was filtered. It is available here :
http://www.jahia.org/~loom/JackRabbit_Performance_Tests.pdf
Regards,
Serge Huber.
Serge Huber wrote:
>
> Hi Stefan,
>
> Sorry I took so long to come back to you. I wanted to finish some work
> on the persistence managers first, and then come back...Unfortunately
> I ran into some walls with Hibernate, and things slipped... anyway I'm
> sure you know how it is.
>
>> the changes in the PersistenceManager interface are not major.
>> i hope that i am able to commit the redesigned code tomorrow or some
>> time next week.
>>
>>
> Ok I've already updated my code to work with your changes, and all
> seems fine. I actually like the fact that PersistentNodeState and
> PersistentPropertyState are no longer needed. And the changes are a
> LOT faster too (4 times faster on my test machine).
>
>> personally i think that you get the best performance by not using any
>> mappings
>> but a simple yet efficient serialization format. take a look at the
>> serialize/deserialize methods in ObjectPersistenceManager for an
>> example.
>>
>>
> Hmm... At first I agreed with you, but then I started doing some
> benchmark testing with my implementations. I am attaching in PDF a
> quick spreadsheet done with the results I had on a test that does an
> XML import of a medium sized XML file (4500 tags). As I tested under
> Windows, there is a huge performance cost at creating a lot of
> directories and files, so this slows down the ObjectPersistenceManager
> and the XMLPersistenceManager. I think, in that regard, that the two
> database-backed persistence manager I implemented far not too badly in
> my quick tests. I do understand that there could be other
> serialization-based implementations, which would probably far a lot
> better (the InMem persistence manager that does serialization based on
> fill for example is very fast).
>
> I should also talk about why I'm interested in DB-backed persistence
> managers. I think that this makes it a lot simpler to cluster
> Jackrabbit in the future, since the ORM tool can take care of this.
> Filesystem-based clustering is a lot less reliable and sometimes even
> impossible to deploy (some administrators won't allow it).
>
> Finally, I'm now looking at how to contribute this (still early code).
> For the moment I've also worked in the same source code as the
> Jackrabbit project, but I had a good reason for this : I wanted to be
> able to use all the unit tests on my implementations, in order to make
> sure I'm at least providing something that works with the tests. I've
> read a lot on this list about moving contributions into a seperate
> projects - an idea I like too - but would this mean loosing the
> possibility to run the unit tests on the Persistence Manager ? I could
> always copy-paste all the test code in a sub-project but I don't like
> that idea either.
>
> Thank you for all your help,
> Regards,
> Serge Huber.
Re: OJB / Hibernate PersistenceManagers [Was: PersistenceManager & childNodeEntries]
Posted by Serge Huber <se...@jahia.com>.
Hi Stefan,
Seeing that you committed the contribution, I immediately faxed the CLA
to the foundation. So all should be ok now.
Regards,
Serge Huber.
On Feb 14, 2005, at 4:44 PM, Stefan Guggisberg wrote:
> hi serge
>
> On Thu, 10 Feb 2005 12:05:29 +0100, Serge Huber <sh...@jahia.com>
> wrote:
>> Here is the new ZIP with the updated codebase :
>> http://www.jahia.org/~loom/orm-persistence-20050210-1200.zip . This
>> file
>> must be decompressed from the contrib directory as I am using relative
>> paths to run the parent project tests.
>
> thank you for your contribution! i have added your code to the contrib
> directory.
>
> as this is a substantial contribution we would need to have your CLA
> on file.
> if you did not yet send the CLA please do so now. thanks.
>
> cheers
> stefan
>
Re: OJB / Hibernate PersistenceManagers [Was: PersistenceManager & childNodeEntries]
Posted by Stefan Guggisberg <st...@gmail.com>.
hi serge
On Thu, 10 Feb 2005 12:05:29 +0100, Serge Huber <sh...@jahia.com> wrote:
> Here is the new ZIP with the updated codebase :
> http://www.jahia.org/~loom/orm-persistence-20050210-1200.zip . This file
> must be decompressed from the contrib directory as I am using relative
> paths to run the parent project tests.
thank you for your contribution! i have added your code to the contrib
directory.
as this is a substantial contribution we would need to have your CLA on file.
if you did not yet send the CLA please do so now. thanks.
cheers
stefan
Re: OJB / Hibernate PersistenceManagers [Was: PersistenceManager & childNodeEntries]
Posted by Stefan Guggisberg <st...@gmail.com>.
hi serge,
On Thu, 10 Feb 2005 12:05:29 +0100, Serge Huber <sh...@jahia.com> wrote:
>
> Hi Stefan & all,
>
> I've been busy a much longer time than expected on updating the code to
> make it work completely. I think I have managed to remove all the known
> issues as well as improve the code a bit. In the hindsight, I learned a
> valuable lesson : OJB & Hibernate have VERY different ways of managing
> collections, especially how they handle removal within collections, and
> unfortunately I wasn't able to share all my POJO objects between the two
> implementations. The culprit here is OJB, which requires a specific
> interface or a specific collection implementation (which I chose) to be
> "notified" of collection element removal. Anyway, it should work now but
> I failed in my initial goal of using one set of ORM objects for both
> implementations.
>
> >- there's a broken dependency in the project.xml. maven couldn't
> > find mysql-connector-java-3.1.6-bin.jar. i changed the dependency
> > to version 3.0.10-stable-bin and the build succeeded.
> >
> >
> Sorry about that. I use an internal repository here and I had missed
> that. It is corrected now.
>
> >- the junit test cases are located in the package org.jahia.jcr.test,
> > i'd prefer to put hem in org.apache.jackrabbit.test.*
> >
> >
> Again an oversight. It is now corrected.
>
> >- the source code is missing the correct apache license headers
> >
> >
> I think I corrected all the files, but I might have missed one or two,
> thank you for letting me know if you see anything. I also added the
> license header to all the configuration files that would allow it.
>
> >- the indentation of the java source is inconsistent
> >
> >
> This was due to a bug in my IDE (JBuilder 2005). It is now corrected.
>
> >- NodeReferences are not yet fully supported (destroy, exists)
> >
> >
> I have fully implemented these, but unfortunately it doesn't seem like
> Jackrabbit has test cases for these ?
not yet, but we will add some soon.
>
> >- BLOBs are not yet supported
> >
> >
> This is the great news in my new version, BLOBs are now supported ! I
> have only tested with HSQLDB and MySQL though. Also HSQLDB is a bit
> slower than MySQL as it doesn't use *real* blobs, but binary arrays to
> store the binary content (it's actually even crazier than that. If you
> look at the HSQLDB script files you see that the whole binary is
> contained in an SQL INSERT statement). But anyway I have implemented a
> test case that inserts a 1MB BLOB into the repository and it seems to
> work (I have also tested with 15MB BLOBs, but for reasons of test speed
> I have reduced the size to something smaller).
>
> >if you would like to contribute the orm-persistence package
> >(which i would very appreciate :) please fix the above issues.
> >we would also need a CLA on file from you.
> >
> >
> Ok I'll send in the CLA asap.
great, please let me know when you've done so.
>
> Here is the new ZIP with the updated codebase :
> http://www.jahia.org/~loom/orm-persistence-20050210-1200.zip . This file
> must be decompressed from the contrib directory as I am using relative
> paths to run the parent project tests.
thanks, i will have a look at it soon, probably some time next week.
>
> Last but not least, I must say it's not always easy to stay up to date
> with all the commits going on these days, especially since half of the
> time it seems that things are broken (I have had a lot of test failures,
> especially in versioning). Usually things get corrected the same day so
> I didn't report it, but it might help to know where in the process of
> updating to 16.2 you guys are :)
sorry, we're still in the process of updating the implementation ...
there are a couple of semantic changes in the api that
are unfortunately non-trivial to implement and that require
complete rewrites of certain aereas (e.g. import, workspace
operations etc).
cheers
stefan
>
> cheers,
> Serge...
>
Re: OJB / Hibernate PersistenceManagers [Was: PersistenceManager
& childNodeEntries]
Posted by Serge Huber <sh...@jahia.com>.
Hi Stefan & all,
I've been busy a much longer time than expected on updating the code to
make it work completely. I think I have managed to remove all the known
issues as well as improve the code a bit. In the hindsight, I learned a
valuable lesson : OJB & Hibernate have VERY different ways of managing
collections, especially how they handle removal within collections, and
unfortunately I wasn't able to share all my POJO objects between the two
implementations. The culprit here is OJB, which requires a specific
interface or a specific collection implementation (which I chose) to be
"notified" of collection element removal. Anyway, it should work now but
I failed in my initial goal of using one set of ORM objects for both
implementations.
>- there's a broken dependency in the project.xml. maven couldn't
> find mysql-connector-java-3.1.6-bin.jar. i changed the dependency
> to version 3.0.10-stable-bin and the build succeeded.
>
>
Sorry about that. I use an internal repository here and I had missed
that. It is corrected now.
>- the junit test cases are located in the package org.jahia.jcr.test,
> i'd prefer to put hem in org.apache.jackrabbit.test.*
>
>
Again an oversight. It is now corrected.
>- the source code is missing the correct apache license headers
>
>
I think I corrected all the files, but I might have missed one or two,
thank you for letting me know if you see anything. I also added the
license header to all the configuration files that would allow it.
>- the indentation of the java source is inconsistent
>
>
This was due to a bug in my IDE (JBuilder 2005). It is now corrected.
>- NodeReferences are not yet fully supported (destroy, exists)
>
>
I have fully implemented these, but unfortunately it doesn't seem like
Jackrabbit has test cases for these ?
>- BLOBs are not yet supported
>
>
This is the great news in my new version, BLOBs are now supported ! I
have only tested with HSQLDB and MySQL though. Also HSQLDB is a bit
slower than MySQL as it doesn't use *real* blobs, but binary arrays to
store the binary content (it's actually even crazier than that. If you
look at the HSQLDB script files you see that the whole binary is
contained in an SQL INSERT statement). But anyway I have implemented a
test case that inserts a 1MB BLOB into the repository and it seems to
work (I have also tested with 15MB BLOBs, but for reasons of test speed
I have reduced the size to something smaller).
>if you would like to contribute the orm-persistence package
>(which i would very appreciate :) please fix the above issues.
>we would also need a CLA on file from you.
>
>
Ok I'll send in the CLA asap.
Here is the new ZIP with the updated codebase :
http://www.jahia.org/~loom/orm-persistence-20050210-1200.zip . This file
must be decompressed from the contrib directory as I am using relative
paths to run the parent project tests.
Last but not least, I must say it's not always easy to stay up to date
with all the commits going on these days, especially since half of the
time it seems that things are broken (I have had a lot of test failures,
especially in versioning). Usually things get corrected the same day so
I didn't report it, but it might help to know where in the process of
updating to 16.2 you guys are :)
cheers,
Serge...
OJB / Hibernate PersistenceManagers [Was: PersistenceManager & childNodeEntries]
Posted by Stefan Guggisberg <st...@gmail.com>.
hi serge,
> I've finally managed to put a first *very early* version of my code up,
> it's available here :
> http://www.jahia.org/~loom/orm-persistence-20050131-1630.zip
great! i downloaded the archive, read the readme.txt and was
able to build it & run the test. looks good so far!
a couple of minor issues though:
- there's a broken dependency in the project.xml. maven couldn't
find mysql-connector-java-3.1.6-bin.jar. i changed the dependency
to version 3.0.10-stable-bin and the build succeeded.
- the junit test cases are located in the package org.jahia.jcr.test,
i'd prefer to put hem in org.apache.jackrabbit.test.*
- the source code is missing the correct apache license headers
- the indentation of the java source is inconsistent
- NodeReferences are not yet fully supported (destroy, exists)
- BLOBs are not yet supported
if you would like to contribute the orm-persistence package
(which i would very appreciate :) please fix the above issues.
we would also need a CLA on file from you.
thanks & keep up the good work!
cheers
stefan
Re: PersistenceManager & childNodeEntries
Posted by Stefan Guggisberg <st...@gmail.com>.
hi serge,
On Mon, 31 Jan 2005 16:44:53 +0100, Serge Huber <sh...@jahia.com> wrote:
> Stefan Guggisberg wrote:
>
> >yes, i did some fine tuning and it seems to be quite effective :)
> >
> >
> Indeed. But I'm now seeing some sluggliness in the Search indexing, but
> this is quite normal, indexing takes time.
>
> >i assume you got me somehow wrong. i was not saying that file-based
> >persistence managers (ObjectPersistenceManager &
> >XMLPersistenceManager) are faster than db-based persistence managers.
> >
> >what i wanted to say is that if you're using a db-based persistence manager
> >then using a very *simple* serilalization format gives you a much better
> >performance rather than mapping the objcts to complex data models/db schemas.
> >
> >i did tests where i stored serialized ItemState objects as raw byte arrays
> >in relational and b-tree based databases and the results were very promising.
> >take a look at the InMemPersistenceManager where i am doing exactly that,
> >storing serialized ItemState objects as byte arrays. the serialization is
> >handled by the static methods serialize/deserialize in ObjectPersistenceManager.
> >
> >
> Actually at first I did misunderstand what you meant, but when looking
> at the InMemPersistenceManager implementation I understood what you
> meant. Anyway, in the PDF I sent last week, the results seem to agree
> with you :)
>
> >but this is just my personal opinion, feel free to do it your way.
> >
> >a word on the file-based persistence managers
> >(ObjectPersistenceManager & XMLPersistenceManager) which are storing
> >the data in serialized and xml format
> >in a file system abstraction (FileSystem interface):
> >i am totally aware of the poor scalability/performance when using
> >LocalFileSystem
> >on a windows box. blame the windows file system whose performance
> >detoriates very quickly if the number of files & folders increase. run the same
> >tests on a linux box (using ext3 or reiser fs) and the differences in
> >performance
> >will be dramatic. or alternatively use the CQFileSystem (day's unix
> >fs-in-a-file)
> >on a windows box. the default workspace configuration in jackrabbit already
> >uses cqfs (see src/conf/repository.xml).
> >
> >
> Ok. I also assumed it would be faster on Linux, but I didn't have time
> to test it. As for cqfs, what's the license on that ? Where is the
> source code ?
there's a LICENSE.txt in the jar file (cqfs-3.5.6.jar). it's not
open-source but
it's free for non-commercial use. cqfs is an optional dependency in
the project.xml.
if you did run "maven jar" you should already have it on your machine.
search the list for "cqfs" or "CQFileSystem", there were a couple of posts
regarding cqfs.
>
> >btw: there was a long discussion regarding performance & scalability,
> >the subject was "the 1mn test: first jackrabbit performance and
> >scalability tests".
> >
> >
> Ok thank you, I will look it up if it's available in the archives. I'm
> actually not entirely focused on performance, but this was the first
> thing I wanted to do to get a feel of how well Jackrabbit fared, as I am
> looking at it for some projects, and I must it's a lot more mature than
> I expected it to be !
great to hear that, thanks.
>
> >i assume that this is just a configuration/maven issue. i'm sure you can
> >setup a separate project (e.g. a custom persistence manager) that
> >has a dependency on jackrabbit and that can run the jackrabbit unit tests.
> >i am not a maven expert though ;)
> >
> >
> You were right :) Just after I sent my mail I managed to do just that,
> but I do require the source code of Jackrabbit as a parent project.
> Basically all I did was :
>
> <unitTestSourceDirectory>../../src/test</unitTestSourceDirectory>
>
> >thank you very much for your valuable input. if you're ready to contribute
> >your custom persistence manager implementations we'd certainly be
> >interested. be prepared though for occassional changes in the
> >PersistenceManager interface.
> >
> >
> Don't worry about changes, as long as they actually improve things I'm
> all for them.
>
> I've finally managed to put a first *very early* version of my code up,
> it's available here :
> http://www.jahia.org/~loom/orm-persistence-20050131-1630.zip
>
> Basically what I've done is that I have assumed this would go into :
>
> contrib/orm-persistence
>
> So basically uncompress the zip in the contrib directory and it will be
> in the correct location. I've also included a README.txt file which is
> important because database-based persistence managers need a little more
> setup. I also had to document existing limitations and known issues...
>
> As for the code, most of it was put together very quickly so I'm sure
> there are ways to improve it :) I just want to get initial feedback to
> improve things. Let me know what you think.
thanks, i will have a look at it asap.
cheers
stefan
>
> cheers,
> Serge...
>
Re: PersistenceManager & childNodeEntries
Posted by Serge Huber <sh...@jahia.com>.
Stefan Guggisberg wrote:
>yes, i did some fine tuning and it seems to be quite effective :)
>
>
Indeed. But I'm now seeing some sluggliness in the Search indexing, but
this is quite normal, indexing takes time.
>i assume you got me somehow wrong. i was not saying that file-based
>persistence managers (ObjectPersistenceManager &
>XMLPersistenceManager) are faster than db-based persistence managers.
>
>what i wanted to say is that if you're using a db-based persistence manager
>then using a very *simple* serilalization format gives you a much better
>performance rather than mapping the objcts to complex data models/db schemas.
>
>i did tests where i stored serialized ItemState objects as raw byte arrays
>in relational and b-tree based databases and the results were very promising.
>take a look at the InMemPersistenceManager where i am doing exactly that,
>storing serialized ItemState objects as byte arrays. the serialization is
>handled by the static methods serialize/deserialize in ObjectPersistenceManager.
>
>
Actually at first I did misunderstand what you meant, but when looking
at the InMemPersistenceManager implementation I understood what you
meant. Anyway, in the PDF I sent last week, the results seem to agree
with you :)
>but this is just my personal opinion, feel free to do it your way.
>
>a word on the file-based persistence managers
>(ObjectPersistenceManager & XMLPersistenceManager) which are storing
>the data in serialized and xml format
>in a file system abstraction (FileSystem interface):
>i am totally aware of the poor scalability/performance when using
>LocalFileSystem
>on a windows box. blame the windows file system whose performance
>detoriates very quickly if the number of files & folders increase. run the same
>tests on a linux box (using ext3 or reiser fs) and the differences in
>performance
>will be dramatic. or alternatively use the CQFileSystem (day's unix
>fs-in-a-file)
>on a windows box. the default workspace configuration in jackrabbit already
>uses cqfs (see src/conf/repository.xml).
>
>
Ok. I also assumed it would be faster on Linux, but I didn't have time
to test it. As for cqfs, what's the license on that ? Where is the
source code ?
>btw: there was a long discussion regarding performance & scalability,
>the subject was "the 1mn test: first jackrabbit performance and
>scalability tests".
>
>
Ok thank you, I will look it up if it's available in the archives. I'm
actually not entirely focused on performance, but this was the first
thing I wanted to do to get a feel of how well Jackrabbit fared, as I am
looking at it for some projects, and I must it's a lot more mature than
I expected it to be !
>i assume that this is just a configuration/maven issue. i'm sure you can
>setup a separate project (e.g. a custom persistence manager) that
>has a dependency on jackrabbit and that can run the jackrabbit unit tests.
>i am not a maven expert though ;)
>
>
You were right :) Just after I sent my mail I managed to do just that,
but I do require the source code of Jackrabbit as a parent project.
Basically all I did was :
<unitTestSourceDirectory>../../src/test</unitTestSourceDirectory>
>thank you very much for your valuable input. if you're ready to contribute
>your custom persistence manager implementations we'd certainly be
>interested. be prepared though for occassional changes in the
>PersistenceManager interface.
>
>
Don't worry about changes, as long as they actually improve things I'm
all for them.
I've finally managed to put a first *very early* version of my code up,
it's available here :
http://www.jahia.org/~loom/orm-persistence-20050131-1630.zip
Basically what I've done is that I have assumed this would go into :
contrib/orm-persistence
So basically uncompress the zip in the contrib directory and it will be
in the correct location. I've also included a README.txt file which is
important because database-based persistence managers need a little more
setup. I also had to document existing limitations and known issues...
As for the code, most of it was put together very quickly so I'm sure
there are ways to improve it :) I just want to get initial feedback to
improve things. Let me know what you think.
cheers,
Serge...
Re: PersistenceManager & childNodeEntries
Posted by Stefan Guggisberg <st...@gmail.com>.
hi serge,
first of all sorry for the late response.
On Thu, 27 Jan 2005 15:09:41 +0100, Serge Huber <sh...@jahia.com> wrote:
>
> Hi Stefan,
>
> Sorry I took so long to come back to you. I wanted to finish some work
> on the persistence managers first, and then come back...Unfortunately I
> ran into some walls with Hibernate, and things slipped... anyway I'm
> sure you know how it is.
>
> >the changes in the PersistenceManager interface are not major.
> >i hope that i am able to commit the redesigned code tomorrow or
> >some time next week.
> >
> >
> Ok I've already updated my code to work with your changes, and all seems
> fine. I actually like the fact that PersistentNodeState and
> PersistentPropertyState are no longer needed. And the changes are a LOT
> faster too (4 times faster on my test machine).
yes, i did some fine tuning and it seems to be quite effective :)
>
> >personally i think that you get the best performance by not using any mappings
> >but a simple yet efficient serialization format. take a look at the
> >serialize/deserialize methods in ObjectPersistenceManager for an
> >example.
> >
> >
> Hmm... At first I agreed with you, but then I started doing some
> benchmark testing with my implementations. I am attaching in PDF a quick
> spreadsheet done with the results I had on a test that does an XML
> import of a medium sized XML file (4500 tags). As I tested under
> Windows, there is a huge performance cost at creating a lot of
> directories and files, so this slows down the ObjectPersistenceManager
> and the XMLPersistenceManager. I think, in that regard, that the two
> database-backed persistence manager I implemented far not too badly in
> my quick tests. I do understand that there could be other
> serialization-based implementations, which would probably far a lot
> better (the InMem persistence manager that does serialization based on
> fill for example is very fast).
i assume you got me somehow wrong. i was not saying that file-based
persistence managers (ObjectPersistenceManager &
XMLPersistenceManager) are faster than db-based persistence managers.
what i wanted to say is that if you're using a db-based persistence manager
then using a very *simple* serilalization format gives you a much better
performance rather than mapping the objcts to complex data models/db schemas.
i did tests where i stored serialized ItemState objects as raw byte arrays
in relational and b-tree based databases and the results were very promising.
take a look at the InMemPersistenceManager where i am doing exactly that,
storing serialized ItemState objects as byte arrays. the serialization is
handled by the static methods serialize/deserialize in ObjectPersistenceManager.
but this is just my personal opinion, feel free to do it your way.
a word on the file-based persistence managers
(ObjectPersistenceManager & XMLPersistenceManager) which are storing
the data in serialized and xml format
in a file system abstraction (FileSystem interface):
i am totally aware of the poor scalability/performance when using
LocalFileSystem
on a windows box. blame the windows file system whose performance
detoriates very quickly if the number of files & folders increase. run the same
tests on a linux box (using ext3 or reiser fs) and the differences in
performance
will be dramatic. or alternatively use the CQFileSystem (day's unix
fs-in-a-file)
on a windows box. the default workspace configuration in jackrabbit already
uses cqfs (see src/conf/repository.xml).
btw: there was a long discussion regarding performance & scalability,
the subject was "the 1mn test: first jackrabbit performance and
scalability tests".
>
> I should also talk about why I'm interested in DB-backed persistence
> managers. I think that this makes it a lot simpler to cluster Jackrabbit
> in the future, since the ORM tool can take care of this.
> Filesystem-based clustering is a lot less reliable and sometimes even
> impossible to deploy (some administrators won't allow it).
>
> Finally, I'm now looking at how to contribute this (still early code).
> For the moment I've also worked in the same source code as the
> Jackrabbit project, but I had a good reason for this : I wanted to be
> able to use all the unit tests on my implementations, in order to make
> sure I'm at least providing something that works with the tests. I've
> read a lot on this list about moving contributions into a seperate
> projects - an idea I like too - but would this mean loosing the
> possibility to run the unit tests on the Persistence Manager ? I could
> always copy-paste all the test code in a sub-project but I don't like
> that idea either.
i assume that this is just a configuration/maven issue. i'm sure you can
setup a separate project (e.g. a custom persistence manager) that
has a dependency on jackrabbit and that can run the jackrabbit unit tests.
i am not a maven expert though ;)
thank you very much for your valuable input. if you're ready to contribute
your custom persistence manager implementations we'd certainly be
interested. be prepared though for occassional changes in the
PersistenceManager interface.
cheers
stefan
>
> Thank you for all your help,
> Regards,
> Serge Huber.
>
>
Re: PersistenceManager & childNodeEntries
Posted by Serge Huber <sh...@jahia.com>.
Hi Stefan,
Sorry I took so long to come back to you. I wanted to finish some work
on the persistence managers first, and then come back...Unfortunately I
ran into some walls with Hibernate, and things slipped... anyway I'm
sure you know how it is.
>the changes in the PersistenceManager interface are not major.
>i hope that i am able to commit the redesigned code tomorrow or
>some time next week.
>
>
Ok I've already updated my code to work with your changes, and all seems
fine. I actually like the fact that PersistentNodeState and
PersistentPropertyState are no longer needed. And the changes are a LOT
faster too (4 times faster on my test machine).
>personally i think that you get the best performance by not using any mappings
>but a simple yet efficient serialization format. take a look at the
>serialize/deserialize methods in ObjectPersistenceManager for an
>example.
>
>
Hmm... At first I agreed with you, but then I started doing some
benchmark testing with my implementations. I am attaching in PDF a quick
spreadsheet done with the results I had on a test that does an XML
import of a medium sized XML file (4500 tags). As I tested under
Windows, there is a huge performance cost at creating a lot of
directories and files, so this slows down the ObjectPersistenceManager
and the XMLPersistenceManager. I think, in that regard, that the two
database-backed persistence manager I implemented far not too badly in
my quick tests. I do understand that there could be other
serialization-based implementations, which would probably far a lot
better (the InMem persistence manager that does serialization based on
fill for example is very fast).
I should also talk about why I'm interested in DB-backed persistence
managers. I think that this makes it a lot simpler to cluster Jackrabbit
in the future, since the ORM tool can take care of this.
Filesystem-based clustering is a lot less reliable and sometimes even
impossible to deploy (some administrators won't allow it).
Finally, I'm now looking at how to contribute this (still early code).
For the moment I've also worked in the same source code as the
Jackrabbit project, but I had a good reason for this : I wanted to be
able to use all the unit tests on my implementations, in order to make
sure I'm at least providing something that works with the tests. I've
read a lot on this list about moving contributions into a seperate
projects - an idea I like too - but would this mean loosing the
possibility to run the unit tests on the Persistence Manager ? I could
always copy-paste all the test code in a sub-project but I don't like
that idea either.
Thank you for all your help,
Regards,
Serge Huber.
Re: PersistenceManager & childNodeEntries
Posted by Stefan Guggisberg <st...@gmail.com>.
On Thu, 20 Jan 2005 11:23:29 +0100, Serge Huber <sh...@jahia.com> wrote:
> Hi Stefan,
>
> >contributions are always very welcome! currentlly a major redesign
> >of the transaction support code and persistence manager inerfaces
> >is in the works. i know i've been telling this for quite some time now
> >but it turned out to be more complicated than we expected :(
> >i suggest you wait until i have committed the redesigned
> >implementation.
> >
> >
> Ok. I've got mostly working code here though, so I might put it up
> somewhere for some people to try. Basically what I've done is that I've
> been trying to run all the tests against both OJB and Hibernate
> implementations, and apart from a problem in search (on Hibernate), it
> seems to work. Note that I haven't implemented BLOB support yet and a
> few things are still missing, but I was mostly trying to evaluate
> performance against a DB, so in that regard it is already quite complete.
>
> As for the redesign, is it possible to have an idea of what is going to
> change ? Anyway the code I did for the ORM mapping seems quite generic,
> and shouldn't be too long to change (the hard part was getting it done
> the first time).
the changes in the PersistenceManager interface are not major.
i hope that i am able to commit the redesigned code tomorrow or
some time next week.
>
> >we also have to find a way of how to integrate/manage contributions.
> >i imagine a separate location in jackrabbit's svn repository where the
> >contributed source code can be dropped and special maven projects/targets
> >to build it. we should make sure that contributed code (e.g. persistence
> >managers) can not break the main jackrabbit build (e.g. when persistence
> >manager interfaces have changed again).
> >
> >
> Ok, but then again maybe it would also be good to know the problems that
> some implementations face when changing the PersistenceManager. For
> example for the moment I copy all the Persistent* data into ORM objects,
> because the Persistent* object structure was not very compatible with
> ORM mapping (complicated types such as QName, no empty constructors,
> etc...).
personally i think that you get the best performance by not using any mappings
but a simple yet efficient serialization format. take a look at the
serialize/deserialize methods in ObjectPersistenceManager for an
example.
cheers
stefan
>
> >if you remove a property or a node, the parent node will be modified also
> >as the child entry is removed from its child entries list.
> >
> >what you are describing shouldn't happen. can you provide sample
> >code that reproduces the problem?
> >
> >
> Actually I just found the problem, it was in my code :)
>
> cheers,
> Serge...
>
Re: PersistenceManager & childNodeEntries
Posted by Serge Huber <sh...@jahia.com>.
Hi Stefan,
>contributions are always very welcome! currentlly a major redesign
>of the transaction support code and persistence manager inerfaces
>is in the works. i know i've been telling this for quite some time now
>but it turned out to be more complicated than we expected :(
>i suggest you wait until i have committed the redesigned
>implementation.
>
>
Ok. I've got mostly working code here though, so I might put it up
somewhere for some people to try. Basically what I've done is that I've
been trying to run all the tests against both OJB and Hibernate
implementations, and apart from a problem in search (on Hibernate), it
seems to work. Note that I haven't implemented BLOB support yet and a
few things are still missing, but I was mostly trying to evaluate
performance against a DB, so in that regard it is already quite complete.
As for the redesign, is it possible to have an idea of what is going to
change ? Anyway the code I did for the ORM mapping seems quite generic,
and shouldn't be too long to change (the hard part was getting it done
the first time).
>we also have to find a way of how to integrate/manage contributions.
>i imagine a separate location in jackrabbit's svn repository where the
>contributed source code can be dropped and special maven projects/targets
>to build it. we should make sure that contributed code (e.g. persistence
>managers) can not break the main jackrabbit build (e.g. when persistence
>manager interfaces have changed again).
>
>
Ok, but then again maybe it would also be good to know the problems that
some implementations face when changing the PersistenceManager. For
example for the moment I copy all the Persistent* data into ORM objects,
because the Persistent* object structure was not very compatible with
ORM mapping (complicated types such as QName, no empty constructors,
etc...).
>if you remove a property or a node, the parent node will be modified also
>as the child entry is removed from its child entries list.
>
>what you are describing shouldn't happen. can you provide sample
>code that reproduces the problem?
>
>
Actually I just found the problem, it was in my code :)
cheers,
Serge...
Re: PersistenceManager & childNodeEntries
Posted by Stefan Guggisberg <st...@gmail.com>.
hi serge
On Wed, 19 Jan 2005 16:12:03 +0100, Serge Huber <sh...@jahia.com> wrote:
>
> Hi all,
>
> First of all let me quickly say thank you to all the great work that
> went into Jackrabbit and the JSR-170. I realize it must have been highly
> political and possibly frustrating, but the result is quite interesting,
> and I think that this will help a lot of developers as well as end users.
thanks, that's good to hear.
>
> Anyway, I've been working on evaluating Jackrabbit (as many), and I've
> started writing back-end implementations for Apache OJB
> (http://db.apache.org/ojb) and Hibernate (http://www.hibernate.org). I
> was wondering if somebody already had a go at it, and if not, I'd be
> more than willing to contribute the little I've done, despite it being
> very hacky.
contributions are always very welcome! currentlly a major redesign
of the transaction support code and persistence manager inerfaces
is in the works. i know i've been telling this for quite some time now
but it turned out to be more complicated than we expected :(
i suggest you wait until i have committed the redesigned
implementation.
we also have to find a way of how to integrate/manage contributions.
i imagine a separate location in jackrabbit's svn repository where the
contributed source code can be dropped and special maven projects/targets
to build it. we should make sure that contributed code (e.g. persistence
managers) can not break the main jackrabbit build (e.g. when persistence
manager interfaces have changed again).
> I've got one problem though : when removing nodes, Jackrabbit calls the
> PersistenceManager.destroy(PersistentNodeState) method. What happens
> with the childNodeEntries of the parent objects ? It seems that they are
> not modified and that Jackrabbit still maintains children that no longer
> exist. Is this normal ? Did I miss something ?
if you remove a property or a node, the parent node will be modified also
as the child entry is removed from its child entries list.
what you are describing shouldn't happen. can you provide sample
code that reproduces the problem?
cheers
stefan
>
> Regards,
> Serge Huber.
>