You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jackrabbit.apache.org by Nicolas <nt...@gmail.com> on 2006/10/25 15:07:02 UTC
PersistenceManagers Benchmark
Hi,
I am wondering if someone benchmarked each Persistence Manager with
Jackrabbit?
I would like to understand better which PM is the best for which use. For
instance I would have thought file based PM have an overall better
performance than RDBMS ones. But it seems I am wrong. Am I the only one
curious about this?
If nobody did it, I can write one and put it back to the wiki if anyone's
interested.
BR,
Nico
my blog! http://www.deviant-abstraction.net !!
AW: PersistenceManagers Benchmark
Posted by KÖLL Claus <C....@TIROL.GV.AT>.
Are there any news about the Benchmark ?
I have looked on the google site but there is no activity ?
i hope this initiative is still alive :-)
-----Ursprüngliche Nachricht-----
Von: Alexandru Popescu [mailto:the.mindstorm.mailinglist@gmail.com]
Gesendet: Montag, 30. Oktober 2006 11:04
An: users@jackrabbit.apache.org
Betreff: Re: PersistenceManagers Benchmark
On 10/30/06, Artem Melentyev <me...@yandex.ru> wrote:
> eXo JCR (based on jackrabbit) has some performance tests by JMeter:
>
> http://docs.exoplatform.org/exo-documents/exo-jcr.site/jmeter.html
> svn://svn.forge.objectweb.org/svnroot/exoplatform/v2.x/exo-jcr/applications/perftest
>
> Maybe it will help you.
>
Thanks a lot Artem. I will definitely take a look at those.
./alex
--
.w( the_mindstorm )p.
> Alexandru Popescu wrote:
> > I have created the project under Google Code:
> > http://code.google.com/p/jcrperf/.
>
> --
> Artem Melentyev, UralSU, CS401
>
>
Re: PersistenceManagers Benchmark
Posted by Alexandru Popescu <th...@gmail.com>.
On 10/30/06, Artem Melentyev <me...@yandex.ru> wrote:
> eXo JCR (based on jackrabbit) has some performance tests by JMeter:
>
> http://docs.exoplatform.org/exo-documents/exo-jcr.site/jmeter.html
> svn://svn.forge.objectweb.org/svnroot/exoplatform/v2.x/exo-jcr/applications/perftest
>
> Maybe it will help you.
>
Thanks a lot Artem. I will definitely take a look at those.
./alex
--
.w( the_mindstorm )p.
> Alexandru Popescu wrote:
> > I have created the project under Google Code:
> > http://code.google.com/p/jcrperf/.
>
> --
> Artem Melentyev, UralSU, CS401
>
>
Re: PersistenceManagers Benchmark
Posted by Artem Melentyev <me...@yandex.ru>.
eXo JCR (based on jackrabbit) has some performance tests by JMeter:
http://docs.exoplatform.org/exo-documents/exo-jcr.site/jmeter.html
svn://svn.forge.objectweb.org/svnroot/exoplatform/v2.x/exo-jcr/applications/perftest
Maybe it will help you.
Alexandru Popescu wrote:
> I have created the project under Google Code:
> http://code.google.com/p/jcrperf/.
--
Artem Melentyev, UralSU, CS401
Re: PersistenceManagers Benchmark
Posted by Alexandru Popescu <th...@gmail.com>.
I have created the project under Google Code: http://code.google.com/p/jcrperf/.
Please send me offline the account information so that I can add you
to the project.
./alex
--
.w( the_mindstorm )p.
On 10/28/06, Nicolas <nt...@gmail.com> wrote:
> Good points.
>
> Who is creating the project on java.net? Under which name?
>
> Nicolas
> my blog! http://www.deviant-abstraction.net !!
>
>
Re: PersistenceManagers Benchmark
Posted by Nicolas <nt...@gmail.com>.
Good points.
Who is creating the project on java.net? Under which name?
Nicolas
my blog! http://www.deviant-abstraction.net !!
Re: PersistenceManagers Benchmark
Posted by Alexandru Popescu <th...@gmail.com>.
On 10/28/06, Ottinger, Joseph <jo...@techtarget.com> wrote:
> Sure, although I don't think that the presence of more than one binary
> type is really important - after all, what you're really doing is
> storing data from an inputstream into the repository (and retrieving it)
> - the actual data and datatype aren't really important.
>
Weirdly enough (and hopefully my memory is not bad again), I read that
reading node-types in a concurrent app has some impact.
Guys, I would really appreciate if we can concentrate our efforts...
and I really think there will be better chances to have something
working. Let's start a new thread if we need some more ideas and I am
strongly suggesting to create a public project for this.
./alex
--
.w( the_mindstorm )p.
> But I don't know about the other tests, no - URLs?
>
> -----Original Message-----
> From: Nicolas [mailto:ntoper@gmail.com]
> Sent: Saturday, October 28, 2006 8:45 AM
> To: users@jackrabbit.apache.org
> Subject: Re: PersistenceManagers Benchmark
>
> Could we put pictures and video too in the test set please?
>
> I am of course interested in participating in it. Do you know about
> Andrew test (it's a test designed for AFS) and the LADDIS one? They
> might give us some ideas.
>
> Nico
>
RE: PersistenceManagers Benchmark
Posted by "Ottinger, Joseph" <jo...@techtarget.com>.
Sure, although I don't think that the presence of more than one binary
type is really important - after all, what you're really doing is
storing data from an inputstream into the repository (and retrieving it)
- the actual data and datatype aren't really important.
But I don't know about the other tests, no - URLs?
-----Original Message-----
From: Nicolas [mailto:ntoper@gmail.com]
Sent: Saturday, October 28, 2006 8:45 AM
To: users@jackrabbit.apache.org
Subject: Re: PersistenceManagers Benchmark
Could we put pictures and video too in the test set please?
I am of course interested in participating in it. Do you know about
Andrew test (it's a test designed for AFS) and the LADDIS one? They
might give us some ideas.
Nico
Re: PersistenceManagers Benchmark
Posted by Nicolas <nt...@gmail.com>.
Could we put pictures and video too in the test set please?
I am of course interested in participating in it. Do you know about Andrew
test (it's a test designed for AFS) and the LADDIS one? They might give us
some ideas.
Nico
RE: PersistenceManagers Benchmark
Posted by "Ottinger, Joseph" <jo...@techtarget.com>.
Incidentally, I'm still in the process of working on the JCR benchmark.
I have a set of elements stored, in a set of documents (text), a set of
audio files (streams), and a large number of, well, numbers. Still some
synchronization problems, as I'm having to work out how locks work in
JCR (hey, I just started writing the thing today, and I think I know
where the issue is.)
Writing is only the first step, of course. Reading, querying are next.
Initial results on stores, though, show the filesystem as being
*slightly* faster than the RDMS, with BDB integration still on its way.
Stores are likely to be far more rare than reads and queries, so this
data isn't worth a lot yet.
Alexandru and I will probably have a project set up on googlecode or
java.net soon so everyone can participate.
-----Original Message-----
From: Nicolas [mailto:ntoper@gmail.com]
Sent: Friday, October 27, 2006 5:57 PM
To: users@jackrabbit.apache.org
Subject: Re: PersistenceManagers Benchmark
Hi Marcin,
I think they were discussing about the importXML method (available in
Workspace and Session classes). If my understanding of Jackrabbit is
correct, those methods allow specifically to import some XML in a
workspace;
Nico
On 10/27/06, Marcin Nowak <ma...@comarch.com> wrote:
>
> As far as I know atomic parsing of nodes from XML is not available a
> this moment in JackRabit, am I right?
>
> BR,
> Marcin Nowak
>
> -----Original Message-----
> From: robert burrell donkin [mailto:robertburrelldonkin@gmail.com]
> Sent: Friday, October 27, 2006 1:41 PM
> To: users@jackrabbit.apache.org
> Subject: Re: PersistenceManagers Benchmark
>
> On 10/27/06, Marcin Nowak <ma...@comarch.com> wrote:
> > Hi,
> >
> > Regarding stress tests, I need to know where are the limitations of
> > importing XML's and where is a huge memory usage coming from.
>
> transactional importing of xml data is always going to be memory
> intensive
>
> the memory used in binding could be reduced by (say) custom SAX
> handling or pull parsing the input stream so long as each node were
> saved atomically
>
> - robert
>
>
> --
> Empowering business
> http://webinars.comarch.com
>
--
a+
Nico
my blog! http://www.deviant-abstraction.net !!
Re: PersistenceManagers Benchmark
Posted by Nicolas <nt...@gmail.com>.
Hi Marcin,
I think they were discussing about the importXML method (available in
Workspace and Session classes). If my understanding of Jackrabbit is
correct, those methods allow specifically to import some XML in a workspace;
Nico
On 10/27/06, Marcin Nowak <ma...@comarch.com> wrote:
>
> As far as I know atomic parsing of nodes from XML is not available a this
> moment in JackRabit, am I right?
>
> BR,
> Marcin Nowak
>
> -----Original Message-----
> From: robert burrell donkin [mailto:robertburrelldonkin@gmail.com]
> Sent: Friday, October 27, 2006 1:41 PM
> To: users@jackrabbit.apache.org
> Subject: Re: PersistenceManagers Benchmark
>
> On 10/27/06, Marcin Nowak <ma...@comarch.com> wrote:
> > Hi,
> >
> > Regarding stress tests, I need to know where are the limitations of
> > importing XML's and where is a huge memory usage coming from.
>
> transactional importing of xml data is always going to be memory intensive
>
> the memory used in binding could be reduced by (say) custom SAX
> handling or pull parsing the input stream so long as each node were
> saved atomically
>
> - robert
>
>
> --
> Empowering business
> http://webinars.comarch.com
>
--
a+
Nico
my blog! http://www.deviant-abstraction.net !!
RE: PersistenceManagers Benchmark
Posted by Marcin Nowak <ma...@comarch.com>.
As far as I know atomic parsing of nodes from XML is not available a this
moment in JackRabit, am I right?
BR,
Marcin Nowak
-----Original Message-----
From: robert burrell donkin [mailto:robertburrelldonkin@gmail.com]
Sent: Friday, October 27, 2006 1:41 PM
To: users@jackrabbit.apache.org
Subject: Re: PersistenceManagers Benchmark
On 10/27/06, Marcin Nowak <ma...@comarch.com> wrote:
> Hi,
>
> Regarding stress tests, I need to know where are the limitations of
> importing XML's and where is a huge memory usage coming from.
transactional importing of xml data is always going to be memory intensive
the memory used in binding could be reduced by (say) custom SAX
handling or pull parsing the input stream so long as each node were
saved atomically
- robert
--
Empowering business
http://webinars.comarch.com
Re: PersistenceManagers Benchmark
Posted by robert burrell donkin <ro...@gmail.com>.
On 10/27/06, Marcin Nowak <ma...@comarch.com> wrote:
> Hi,
>
> Regarding stress tests, I need to know where are the limitations of
> importing XML's and where is a huge memory usage coming from.
transactional importing of xml data is always going to be memory intensive
the memory used in binding could be reduced by (say) custom SAX
handling or pull parsing the input stream so long as each node were
saved atomically
- robert
RE: PersistenceManagers Benchmark
Posted by Marcin Nowak <ma...@comarch.com>.
Hi,
Regarding stress tests, I need to know where are the limitations of
importing XML's and where is a huge memory usage coming from.
BR,
Marcin Nowak
-----Original Message-----
From: Alexandru Popescu [mailto:the.mindstorm.mailinglist@gmail.com]
Sent: Thursday, October 26, 2006 1:35 AM
To: users@jackrabbit.apache.org
Subject: Re: PersistenceManagers Benchmark
On 10/26/06, Nicolas <nt...@gmail.com> wrote:
> Hi Alexandru,
>
> Test NG is indeed an excellent idea. I will have a close look at it.
>
... and the help on using it is guaranteed ;-)
> The next step is I think to define the test suite.
>
> Currently we have found to check for:
>
> 1/ concurrencies
> 2/ stress test
> 3/ JCR test suite
>
> Any other ideas?
>
I am not sure what do you mean by the above. I think the suite should
contain:
- insert tests
- retrieve tests
- fetch tests
- query tests
>From this we can easily create load + stress tests. I think we can
start looking at the current available tests (maybe Jackrabbit
committers will help us identify the aboves) and then move them little
by little.
Here is an example (considering that in TestNG you can easily pass
parameters to tests and so on):
initial test (existing test - I assume JUnit test):
public void testCreateNodeTest() {
// node creation test
}
>From this one we can extract a common method (that will be invoked by
load or stress test):
public void createNodeTest(int numberOfNodes) {
}
Then, load test:
@Test
@Parameters({"createNumberOfNodes"})
public createNodesLoadTest(int numberOfNodes) {
// tracking code
createNodeTest(numberOfNodes);
}
and the stress test:
@Test(invocationCount=50, threadPoolSize=25)
@Parameters({"createNumberOfNodes"})
public createNodesLoadTest(int numberOfNodes) {
// tracking code
createNodeTest(numberOfNodes)
}
If we want this refined then we can start refactoring the measurement
code into @Before/@After methods.
Does this make any sense to you?
./alex
--
.w( the_mindstorm )p.
TestNG co-founder
EclipseTestNG Creator
> BR,
> Nico
> my blog! http://www.deviant-abstraction.net !!
>
>
--
Empowering business
http://webinars.comarch.com
Re: PersistenceManagers Benchmark
Posted by Alexandru Popescu <th...@gmail.com>.
On 10/26/06, Nicolas <nt...@gmail.com> wrote:
> Hi Alexandru,
>
> Test NG is indeed an excellent idea. I will have a close look at it.
>
... and the help on using it is guaranteed ;-)
> The next step is I think to define the test suite.
>
> Currently we have found to check for:
>
> 1/ concurrencies
> 2/ stress test
> 3/ JCR test suite
>
> Any other ideas?
>
I am not sure what do you mean by the above. I think the suite should contain:
- insert tests
- retrieve tests
- fetch tests
- query tests
>From this we can easily create load + stress tests. I think we can
start looking at the current available tests (maybe Jackrabbit
committers will help us identify the aboves) and then move them little
by little.
Here is an example (considering that in TestNG you can easily pass
parameters to tests and so on):
initial test (existing test - I assume JUnit test):
public void testCreateNodeTest() {
// node creation test
}
>From this one we can extract a common method (that will be invoked by
load or stress test):
public void createNodeTest(int numberOfNodes) {
}
Then, load test:
@Test
@Parameters({"createNumberOfNodes"})
public createNodesLoadTest(int numberOfNodes) {
// tracking code
createNodeTest(numberOfNodes);
}
and the stress test:
@Test(invocationCount=50, threadPoolSize=25)
@Parameters({"createNumberOfNodes"})
public createNodesLoadTest(int numberOfNodes) {
// tracking code
createNodeTest(numberOfNodes)
}
If we want this refined then we can start refactoring the measurement
code into @Before/@After methods.
Does this make any sense to you?
./alex
--
.w( the_mindstorm )p.
TestNG co-founder
EclipseTestNG Creator
> BR,
> Nico
> my blog! http://www.deviant-abstraction.net !!
>
>
Re: PersistenceManagers Benchmark
Posted by Nicolas <nt...@gmail.com>.
Hi Alexandru,
Test NG is indeed an excellent idea. I will have a close look at it.
The next step is I think to define the test suite.
Currently we have found to check for:
1/ concurrencies
2/ stress test
3/ JCR test suite
Any other ideas?
BR,
Nico
my blog! http://www.deviant-abstraction.net !!
Re: PersistenceManagers Benchmark
Posted by Alexandru Popescu <th...@gmail.com>.
On 10/25/06, Nicolas <nt...@gmail.com> wrote:
> Hi Joseph,
>
> Excellent ideas.
>
> I could use the JCR test suite and add some stress testing (ie 10 000 nodes,
> importing XML). This way I could also compare different JCR implementation
> according to different needs + JR different PM (I am currently more
> interested in the Persistence Manager but this test suite can be easily
> exteneded later).
>
> What kind of stress test should be needed?
>
If you are gonna use TestNG then concurrency stress testing can be a breath ;-).
For example:
making the following simple test a concurrency one:
@Test
public void nodeCreation() {
// create 1000 nodes
}
@Test(invocationCount=10, threadPoolSize=10)
public void nodeCreation() {
// create 1000 nodes
}
I think I already have something for simple queries.
If you have a good start, maybe we can create a Google Code hosted
project to maintain/share these.
./alex
--
.w( the_mindstorm )p.
> Should I put this result on the wiki or on a different web site? Same
> question for the source code of the test?
>
> Nicolas
>
> On 10/25/06, Ottinger, Joseph <jo...@techtarget.com> wrote:
> >
> > Well, what we should do is determine what operations should be tested -
> > then we can create a benchmark fairly easily. Storing a node is easy
> > enough; querying the node is easy enough; storing properties, too.
> >
> > Any other suggestions for common operations?
> >
> > And is anyone else considering commons-jcr for creating common
> > operations for JCR?
> >
> > -----Original Message-----
> > From: Alexandru Popescu [mailto:the.mindstorm.mailinglist@gmail.com]
> > Sent: Wednesday, October 25, 2006 12:53 PM
> > To: users@jackrabbit.apache.org
> > Subject: Re: PersistenceManagers Benchmark
> >
> > #: Nicolas changed the world a bit at a time by saying (astral date:
> > 10/25/2006 4:07 PM) :#
> > > Hi,
> > >
> > > I am wondering if someone benchmarked each Persistence Manager with
> > > Jackrabbit?
> > >
> > > I would like to understand better which PM is the best for which use.
> > > For instance I would have thought file based PM have an overall better
> >
> > > performance than RDBMS ones. But it seems I am wrong. Am I the only
> > > one curious about this?
> > >
> > > If nobody did it, I can write one and put it back to the wiki if
> > > anyone's interested.
> > >
> > > BR,
> > > Nico
> > > my blog! http://www.deviant-abstraction.net !!
> > >
> >
> > Hi Nico!
> >
> > I would be very interested in seeing some results for this.
> > Unfortunately I don't have any time now to invest into this (nor a clear
> > idea how to do it). It would be great if you can create something,
> > because afterwards I am quite sure people will start contributing to it.
> >
> > ./alex
> > --
> > .w( the_mindstorm )p.
> >
>
>
>
> --
> a+
> Nico
> my blog! http://www.deviant-abstraction.net !!
>
>
Re: PersistenceManagers Benchmark
Posted by Nicolas <nt...@gmail.com>.
Hi Joseph,
Excellent ideas.
I could use the JCR test suite and add some stress testing (ie 10 000 nodes,
importing XML). This way I could also compare different JCR implementation
according to different needs + JR different PM (I am currently more
interested in the Persistence Manager but this test suite can be easily
exteneded later).
What kind of stress test should be needed?
Should I put this result on the wiki or on a different web site? Same
question for the source code of the test?
Nicolas
On 10/25/06, Ottinger, Joseph <jo...@techtarget.com> wrote:
>
> Well, what we should do is determine what operations should be tested -
> then we can create a benchmark fairly easily. Storing a node is easy
> enough; querying the node is easy enough; storing properties, too.
>
> Any other suggestions for common operations?
>
> And is anyone else considering commons-jcr for creating common
> operations for JCR?
>
> -----Original Message-----
> From: Alexandru Popescu [mailto:the.mindstorm.mailinglist@gmail.com]
> Sent: Wednesday, October 25, 2006 12:53 PM
> To: users@jackrabbit.apache.org
> Subject: Re: PersistenceManagers Benchmark
>
> #: Nicolas changed the world a bit at a time by saying (astral date:
> 10/25/2006 4:07 PM) :#
> > Hi,
> >
> > I am wondering if someone benchmarked each Persistence Manager with
> > Jackrabbit?
> >
> > I would like to understand better which PM is the best for which use.
> > For instance I would have thought file based PM have an overall better
>
> > performance than RDBMS ones. But it seems I am wrong. Am I the only
> > one curious about this?
> >
> > If nobody did it, I can write one and put it back to the wiki if
> > anyone's interested.
> >
> > BR,
> > Nico
> > my blog! http://www.deviant-abstraction.net !!
> >
>
> Hi Nico!
>
> I would be very interested in seeing some results for this.
> Unfortunately I don't have any time now to invest into this (nor a clear
> idea how to do it). It would be great if you can create something,
> because afterwards I am quite sure people will start contributing to it.
>
> ./alex
> --
> .w( the_mindstorm )p.
>
--
a+
Nico
my blog! http://www.deviant-abstraction.net !!
RE: PersistenceManagers Benchmark
Posted by "Ottinger, Joseph" <jo...@techtarget.com>.
Well, what we should do is determine what operations should be tested -
then we can create a benchmark fairly easily. Storing a node is easy
enough; querying the node is easy enough; storing properties, too.
Any other suggestions for common operations?
And is anyone else considering commons-jcr for creating common
operations for JCR?
-----Original Message-----
From: Alexandru Popescu [mailto:the.mindstorm.mailinglist@gmail.com]
Sent: Wednesday, October 25, 2006 12:53 PM
To: users@jackrabbit.apache.org
Subject: Re: PersistenceManagers Benchmark
#: Nicolas changed the world a bit at a time by saying (astral date:
10/25/2006 4:07 PM) :#
> Hi,
>
> I am wondering if someone benchmarked each Persistence Manager with
> Jackrabbit?
>
> I would like to understand better which PM is the best for which use.
> For instance I would have thought file based PM have an overall better
> performance than RDBMS ones. But it seems I am wrong. Am I the only
> one curious about this?
>
> If nobody did it, I can write one and put it back to the wiki if
> anyone's interested.
>
> BR,
> Nico
> my blog! http://www.deviant-abstraction.net !!
>
Hi Nico!
I would be very interested in seeing some results for this.
Unfortunately I don't have any time now to invest into this (nor a clear
idea how to do it). It would be great if you can create something,
because afterwards I am quite sure people will start contributing to it.
./alex
--
.w( the_mindstorm )p.
Re: PersistenceManagers Benchmark
Posted by Alexandru Popescu <th...@gmail.com>.
#: Nicolas changed the world a bit at a time by saying (astral date: 10/25/2006 4:07 PM) :#
> Hi,
>
> I am wondering if someone benchmarked each Persistence Manager with
> Jackrabbit?
>
> I would like to understand better which PM is the best for which use. For
> instance I would have thought file based PM have an overall better
> performance than RDBMS ones. But it seems I am wrong. Am I the only one
> curious about this?
>
> If nobody did it, I can write one and put it back to the wiki if anyone's
> interested.
>
> BR,
> Nico
> my blog! http://www.deviant-abstraction.net !!
>
Hi Nico!
I would be very interested in seeing some results for this. Unfortunately I don't have any time now
to invest into this (nor a clear idea how to do it). It would be great if you can create something,
because afterwards I am quite sure people will start contributing to it.
./alex
--
.w( the_mindstorm )p.