You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@openjpa.apache.org by gkorland <gk...@gmail.com> on 2010/11/01 11:14:53 UTC
JPAB results
Did anyone review the JPAB (http://www.jpab.org/) results?
It seems like all the other JPA guys are running faster and even better...
--
View this message in context: http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5693298.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: JPAB results
Posted by Ilan Kirsh <ki...@objectdb.com>.
Thank you Milosz. Since InnoDB is transaction-safe and it becomes the new
default storage engine of MySQL I switched to the last MySQL version and now
all the JPA implementations use MySQL with InnoDB.
I published the new results at: http://temp.jpab.org/All/All/All.html
temp.jpab.org .
The new results are based on using OpenJPA and DataNucleus after tuning, but
still default configuration for Hibernate and EclipseLink.
At the moment, OpenJPA is slightly faster than other JPA implementations
with MySQL and PostgreSQL but slightly slower than Hibernate with HSQLDB &
H2 and EclipseLink with Derby in embedded mode.
--
View this message in context: http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5766716.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: JPAB results
Posted by Miłosz Tylenda <mt...@o2.pl>.
Hi Ilan,
> About MySQL - I think that Hibernate and EclipseLink use MyISAM, where
> OpenJPA and DataNucleus use InnoDB. Is it possible to tell OpenJPA to use
> MyISAM as well? It might be faster.
That can be done by specifying "tableType" property of MySQLDictionary:
<property name="openjpa.jdbc.DBDictionary" value="batchLimit=100,tableType=myisam"/>
This might not work with dictionary auto-detection. If so, use this:
<property name="openjpa.jdbc.DBDictionary" value="mysql(batchLimit=100,tableType=myisam)"/>
Cheers,
Milosz
> Here is an example of a persistence.xml file that was used by R3 (for
> PostgreSQL).
>
>
> <?xml version="1.0" encoding="UTF-8"?>
> <persistence xmlns="http://java.sun.com/xml/ns/persistence"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
> http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd" version="2.0">
> <persistence-unit name="OpenJPA_R3-PostgreSQL-server">
>
> <provider>org.apache.openjpa.persistence.PersistenceProviderImpl</provider>
> <class>org.jpab.basic.Person</class>
> <class>org.jpab.col.CollectionPerson</class>
> <class>org.jpab.ext.PersonBase</class>
> <class>org.jpab.ext.PersonExt</class>
> <class>org.jpab.ext.PersonExtExt</class>
> <class>org.jpab.index.IndexedPerson</class>
> <class>org.jpab.node.Node</class>
> <properties>
> <property name="openjpa.ConnectionProperties"
> value="DriverClassName=org.postgresql.Driver,
> Url=jdbc:postgresql://localhost:5432/jpab7493330740, MaxActive=100,
> MaxWait=10000, TestOnBorrow=true, Username=postgres, Password=admin,
> poolPreparedStatements=true"/>
> <property name="openjpa.ConnectionRetainMode" value="always"/>
> <property name="openjpa.ConnectionDriverName"
> value="org.apache.commons.dbcp.BasicDataSource"/>
> <property name="openjpa.jdbc.DBDictionary" value="batchLimit=100"/>
> <property name="openjpa.DataCache" value="true"/>
> <property name="openjpa.Log" value="DefaultLevel=ERROR"/>
> <property name="openjpa.jdbc.SynchronizeMappings"
> value="buildSchema"/>
> </properties>
> </persistence-unit>
> </persistence>
>
Re: JPAB results
Posted by Ilan Kirsh <ki...@objectdb.com>.
Thanks for all of you for the productive comments and tuning tips.
I ran the benchmark again using the new configuration and I updated the
results on:
http://temp.jpab.org/All/All/All.html
The new run is marked as OpenJPA_R3.
There are less failures now (actually only one exception that is thrown a
few times when using client-server mode), but probably you will still be
interested in exploring the exceptions that are thrown when using the
default configuration.
Regarding performance, R3 is slightly better than R2, but the difference now
is not as large as the difference between R2 and the initial run.
Apparently, R2 already overcame the main performance issues, so the
additional tuning had less impact.
About MySQL - I think that Hibernate and EclipseLink use MyISAM, where
OpenJPA and DataNucleus use InnoDB. Is it possible to tell OpenJPA to use
MyISAM as well? It might be faster.
Here is an example of a persistence.xml file that was used by R3 (for
PostgreSQL).
<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd" version="2.0">
<persistence-unit name="OpenJPA_R3-PostgreSQL-server">
<provider>org.apache.openjpa.persistence.PersistenceProviderImpl</provider>
<class>org.jpab.basic.Person</class>
<class>org.jpab.col.CollectionPerson</class>
<class>org.jpab.ext.PersonBase</class>
<class>org.jpab.ext.PersonExt</class>
<class>org.jpab.ext.PersonExtExt</class>
<class>org.jpab.index.IndexedPerson</class>
<class>org.jpab.node.Node</class>
<properties>
<property name="openjpa.ConnectionProperties"
value="DriverClassName=org.postgresql.Driver,
Url=jdbc:postgresql://localhost:5432/jpab7493330740, MaxActive=100,
MaxWait=10000, TestOnBorrow=true, Username=postgres, Password=admin,
poolPreparedStatements=true"/>
<property name="openjpa.ConnectionRetainMode" value="always"/>
<property name="openjpa.ConnectionDriverName"
value="org.apache.commons.dbcp.BasicDataSource"/>
<property name="openjpa.jdbc.DBDictionary" value="batchLimit=100"/>
<property name="openjpa.DataCache" value="true"/>
<property name="openjpa.Log" value="DefaultLevel=ERROR"/>
<property name="openjpa.jdbc.SynchronizeMappings"
value="buildSchema"/>
</properties>
</persistence-unit>
</persistence>
I still don't see how results might be affected by schema generation. Time
measurement starts after about 30 seconds of intensive warm up of the tested
EntityManager, which includes using all the relevant user defined classes. I
don't think that additional schema generation activity should take place
after such warm up - and if it does it might be an issue that should be
solved and fixed on the implementation side. Nevertheless, if I am wrong - I
will be able to change the tests in way that every test will use 2 different
persistence units - one for warm up and second for test (persistence units
are generated automatically by the benchmark framework anyway).
In addition, although I agree that schema generation should be excluded from
time measurement I don't think that manual generation of the schema should
replace automatic schema generation by the tested tools, unless someone can
reveal significant differences in the schema that different ORM tools
generate for these tests.
I encourage you to explore the benchmark further. If you find a different
configuration that works better for OpenJPA I am willing to try it.
I would like to update the main http://www.jpab.org website with better
OpenJPA results - but it has to be done with results of a publicly available
OpenJPA version, rather than with a development snapshot. So I might have to
wait for the release of OpenJPA 2.1
Thank you for your cooperation.
Best regards,
Ilan Kirsh
--
View this message in context: http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5739063.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: JPAB results
Posted by Pinaki Poddar <pp...@apache.org>.
Hi Ilan,
Three usual suspects for performance with OpenJPA are
1. build-time enhancement
2. database connection pooling
3. L2 cache i.e. DataCache usage
Apart from (1), your configuration is using the other two. But keeping the
warm up out of timing measurements, you are in effect taking care of
build-time enhancement as well. So I will consider the configuration as fair
comparison with other vendors.
Your comprehensive and exhaustive work on this area is highly appreciated.
However, your measurements quite drastically differ from some my internal
and unpublished observations. And I have not been able to dive into
analyzing the difference due to other commitments.
-----
Pinaki
--
View this message in context: http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5735837.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: JPAB results
Posted by Matthew Adams <ma...@matthewadams.me>.
Since most production systems don't really rely upon generated schemas, it
seems reasonable to hand craft a schema for each supported database that
uses reasonable assumptions comparable to those that would be made in a
production system (primary/surrogate keys, unique keys, foreign keys, proper
column types and sizes, etc). Each db schema would be part of the source of
the test, and each test starts with a virgin database with its given
schema. It doesn't seem to difficult given that the domain model objects
have been hand crafted.
That way, the benchmark wouldn't have to worry about schema creation or
sychronization or the timings of those operations and standard JPA OR
mapping metadata could be used.
-matthew
On Thu, Nov 11, 2010 at 8:49 AM, Michael Dick <mi...@gmail.com>wrote:
> The schema generation tools might operate in unpredictable ways -
> potentially skewing the results.
>
> Currently the test harness assumes that the schema tools will execute and
> be
> done executing by the time you've created the first EntityManager. This
> isn't guaranteed, some tools may not execute until the first time you
> access
> an entity. In that case your test is, unknowingly, skewed by the shema
> tools.
>
> I haven't looked at the tests to see whether this affects the OpenJPA
> results, I'll try to take a look later today if I can find some cycles. I
> do
> know that when SyncrhonizeMappings is found OpenJPA obtains a lot of
> connections. I put in a quick fix that reduced the use of connections, but
> I
> wouldn't say it's complete by any means.
>
> My recommendation would be to do the table generation separately. OpenJPA
> provides a standalone
> tool<
> http://openjpa.apache.org/builds/latest/docs/manual/manual.html#ref_guide_mapping_mappingtool
> >to
> generate tables, maybe the other providers have similar tools. That
> would
> save you the time of creating a separate persistence unit, at least.
>
> One other thing. It looks like the test expects each EntityManager to use a
> single connection until the EntityManager is closed. If that's the case you
> might want to add this property
> <property name="openjpa.ConnectionRetainMode" value="always"/>
>
> This setting is suitable when the application tightly controls the
> EntityManager, and wants to minimize connection pool use. It's not suitable
> in all scenarios (e.g. container managed EntityManagers), but if you want
> to
> force a single connection to be used this is the way to go.
>
> Hope this helps,
> -mike
>
>
>
> On Wed, Nov 10, 2010 at 8:16 PM, Ilan Kirsh <ki...@objectdb.com> wrote:
>
> >
> > Hi Mike,
> >
> > Automatic schema generation is used by all the tested JPA implementations
> -
> > in order to simplify the benchmark. I agree, however, that it should be
> > excluded from time measurement because it is not a real factor in
> > production. I will try to explain what I did and I will be glad to get
> your
> > opinion.
> >
> > The time of every test run is divided into 2 phases. The first phase is
> > always excluded from time measurement. Only the performance in the second
> > phase is taken into account. This way, initialization operations
> including
> > class loading, enhancement by the Java Agent enhancer, JIT compilation,
> JVM
> > tuning (inlining and other optimizations) and schema generation - are all
> > excluded from time measurement.
> >
> > Connecting to the database (invocation of createEntityManager) is also
> > completed before starting to measure time to avoid the influence of
> > connection pools.
> >
> > Given the above, do you still think that the SynchronizeMappings property
> > might affect the results. If so, how?
> >
> > If necessary I may generate 2 persistence units for every test (per
> > ORM/DBMS) - one for initialization (including schema generation) and the
> > other for the time measurement. But I want to keep the benchmark simple
> as
> > possible, so I will do that only if it is really necessary.
> >
> > Best regards,
> > Ilan
> > --
> > View this message in context:
> > http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5727348.html
> > Sent from the OpenJPA Users mailing list archive at Nabble.com.
> >
>
--
mailto:matthew@matthewadams.me
skype:matthewadams12
yahoo:matthewadams
aol:matthewadams12
google-talk:matthewadams12@gmail.com<go...@gmail.com>
msn:matthew@matthewadams.me <ms...@matthewadams.me>
http://matthewadams.me
http://www.linkedin.com/in/matthewadams
Re: JPAB results
Posted by Michael Dick <mi...@gmail.com>.
The schema generation tools might operate in unpredictable ways -
potentially skewing the results.
Currently the test harness assumes that the schema tools will execute and be
done executing by the time you've created the first EntityManager. This
isn't guaranteed, some tools may not execute until the first time you access
an entity. In that case your test is, unknowingly, skewed by the shema
tools.
I haven't looked at the tests to see whether this affects the OpenJPA
results, I'll try to take a look later today if I can find some cycles. I do
know that when SyncrhonizeMappings is found OpenJPA obtains a lot of
connections. I put in a quick fix that reduced the use of connections, but I
wouldn't say it's complete by any means.
My recommendation would be to do the table generation separately. OpenJPA
provides a standalone
tool<http://openjpa.apache.org/builds/latest/docs/manual/manual.html#ref_guide_mapping_mappingtool>to
generate tables, maybe the other providers have similar tools. That
would
save you the time of creating a separate persistence unit, at least.
One other thing. It looks like the test expects each EntityManager to use a
single connection until the EntityManager is closed. If that's the case you
might want to add this property
<property name="openjpa.ConnectionRetainMode" value="always"/>
This setting is suitable when the application tightly controls the
EntityManager, and wants to minimize connection pool use. It's not suitable
in all scenarios (e.g. container managed EntityManagers), but if you want to
force a single connection to be used this is the way to go.
Hope this helps,
-mike
On Wed, Nov 10, 2010 at 8:16 PM, Ilan Kirsh <ki...@objectdb.com> wrote:
>
> Hi Mike,
>
> Automatic schema generation is used by all the tested JPA implementations -
> in order to simplify the benchmark. I agree, however, that it should be
> excluded from time measurement because it is not a real factor in
> production. I will try to explain what I did and I will be glad to get your
> opinion.
>
> The time of every test run is divided into 2 phases. The first phase is
> always excluded from time measurement. Only the performance in the second
> phase is taken into account. This way, initialization operations including
> class loading, enhancement by the Java Agent enhancer, JIT compilation, JVM
> tuning (inlining and other optimizations) and schema generation - are all
> excluded from time measurement.
>
> Connecting to the database (invocation of createEntityManager) is also
> completed before starting to measure time to avoid the influence of
> connection pools.
>
> Given the above, do you still think that the SynchronizeMappings property
> might affect the results. If so, how?
>
> If necessary I may generate 2 persistence units for every test (per
> ORM/DBMS) - one for initialization (including schema generation) and the
> other for the time measurement. But I want to keep the benchmark simple as
> possible, so I will do that only if it is really necessary.
>
> Best regards,
> Ilan
> --
> View this message in context:
> http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5727348.html
> Sent from the OpenJPA Users mailing list archive at Nabble.com.
>
Re: JPAB results
Posted by Ilan Kirsh <ki...@objectdb.com>.
Hi Mike,
Automatic schema generation is used by all the tested JPA implementations -
in order to simplify the benchmark. I agree, however, that it should be
excluded from time measurement because it is not a real factor in
production. I will try to explain what I did and I will be glad to get your
opinion.
The time of every test run is divided into 2 phases. The first phase is
always excluded from time measurement. Only the performance in the second
phase is taken into account. This way, initialization operations including
class loading, enhancement by the Java Agent enhancer, JIT compilation, JVM
tuning (inlining and other optimizations) and schema generation - are all
excluded from time measurement.
Connecting to the database (invocation of createEntityManager) is also
completed before starting to measure time to avoid the influence of
connection pools.
Given the above, do you still think that the SynchronizeMappings property
might affect the results. If so, how?
If necessary I may generate 2 persistence units for every test (per
ORM/DBMS) - one for initialization (including schema generation) and the
other for the time measurement. But I want to keep the benchmark simple as
possible, so I will do that only if it is really necessary.
Best regards,
Ilan
--
View this message in context: http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5727348.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: JPAB results
Posted by Michael Dick <mi...@gmail.com>.
Is it possible to create the tables manually before collecting performance
data? If so you could remove
<property name="openjpa.jdbc.SynchronizeMappings" value="buildSchema"/>
from persistence.xml.
You'd need to do this for the other JPA providers as well, of course, but
removing the SyncMappings eliminates the table generation code which might
not be optimized for benchmarks. One would rarely use such a property in
production at any rate.
-mike
On Wed, Nov 10, 2010 at 2:04 AM, Ilan Kirsh <ki...@objectdb.com> wrote:
>
> Thank you Georgi,
>
> If there are no more comments and suggestions I will use the following
> persistence.xml for the next run:
>
>
> <?xml version="1.0" encoding="UTF-8"?>
> <persistence xmlns="http://java.sun.com/xml/ns/persistence"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
> http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd" version="2.0">
> <persistence-unit name="OpenJPA2-MySQL-server">
>
> <provider>org.apache.openjpa.persistence.PersistenceProviderImpl</provider>
> <class>org.jpab.basic.Person</class>
> <class>org.jpab.col.CollectionPerson</class>
> <class>org.jpab.ext.PersonBase</class>
> <class>org.jpab.ext.PersonExt</class>
> <class>org.jpab.ext.PersonExtExt</class>
> <class>org.jpab.index.IndexedPerson</class>
> <class>org.jpab.node.Node</class>
> <properties>
> <property name="openjpa.ConnectionProperties"
> value="DriverClassName=com.mysql.jdbc.Driver,
> Url=jdbc:mysql://localhost:3306/jpab4897789836, MaxActive=100,
> MaxWait=10000, TestOnBorrow=true, Username=admin, Password=admin,
> poolPreparedStatements=true"/>
> <property name="openjpa.ConnectionDriverName"
> value="org.apache.commons.dbcp.BasicDataSource"/>
> <property name="openjpa.jdbc.DBDictionary" value="batchLimit=100"/>
> <property name="openjpa.DataCache" value="true"/>
> <property name="openjpa.Log" value="DefaultLevel=ERROR"/>
> <property name="openjpa.jdbc.SynchronizeMappings"
> value="buildSchema"/>
> </properties>
> </persistence-unit>
> </persistence>
>
> By the way, just a heads up - the benchmark site is being featured on
> JavaLobby:
>
> http://java.dzone.com/articles/new-jpa-performance-benchmark
>
> I have mentioned at the end that initial OpenJPA tuning work has produced
> better results.
>
> Best regards,
> Ilan
>
> --
> View this message in context:
> http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5723794.html
> Sent from the OpenJPA Users mailing list archive at Nabble.com.
>
Re: JPAB results
Posted by Rick Curtis <cu...@gmail.com>.
Ilan -
Looks like I sent the previous email a little prematurely, please try
setting this DBCP property also.
<property name="openjpa.ConnectionProperties"
value="poolPreparedStatements=true"/>
Thanks,
Rick
On Thu, Nov 11, 2010 at 4:07 PM, Rick Curtis <cu...@gmail.com> wrote:
> Ilan -
>
> I did some perf testing / profiling today and came across another property
> that should help out quite a lot.
>
> property-3=<property name="openjpa.ConnectionRetainMode" value="always"/>
>
> The net of this property is that each EntityManager will keep a connection
> open to the DB as long as it is alive. Note to anyone else reading this
> post: This property shouldn't be used in a container managed environment as
> EntityManagers can bee pooled and in turn connections will never be closed.
>
> Let me know how it goes. Thanks for all of the hard work!
>
> Rick
>
Re: JPAB results
Posted by Rick Curtis <cu...@gmail.com>.
Ilan -
I did some perf testing / profiling today and came across another property
that should help out quite a lot.
property-3=<property name="openjpa.ConnectionRetainMode" value="always"/>
The net of this property is that each EntityManager will keep a connection
open to the DB as long as it is alive. Note to anyone else reading this
post: This property shouldn't be used in a container managed environment as
EntityManagers can bee pooled and in turn connections will never be closed.
Let me know how it goes. Thanks for all of the hard work!
Rick
Re: JPAB results
Posted by Ilan Kirsh <ki...@objectdb.com>.
Thank you Georgi,
If there are no more comments and suggestions I will use the following
persistence.xml for the next run:
<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd" version="2.0">
<persistence-unit name="OpenJPA2-MySQL-server">
<provider>org.apache.openjpa.persistence.PersistenceProviderImpl</provider>
<class>org.jpab.basic.Person</class>
<class>org.jpab.col.CollectionPerson</class>
<class>org.jpab.ext.PersonBase</class>
<class>org.jpab.ext.PersonExt</class>
<class>org.jpab.ext.PersonExtExt</class>
<class>org.jpab.index.IndexedPerson</class>
<class>org.jpab.node.Node</class>
<properties>
<property name="openjpa.ConnectionProperties"
value="DriverClassName=com.mysql.jdbc.Driver,
Url=jdbc:mysql://localhost:3306/jpab4897789836, MaxActive=100,
MaxWait=10000, TestOnBorrow=true, Username=admin, Password=admin,
poolPreparedStatements=true"/>
<property name="openjpa.ConnectionDriverName"
value="org.apache.commons.dbcp.BasicDataSource"/>
<property name="openjpa.jdbc.DBDictionary" value="batchLimit=100"/>
<property name="openjpa.DataCache" value="true"/>
<property name="openjpa.Log" value="DefaultLevel=ERROR"/>
<property name="openjpa.jdbc.SynchronizeMappings"
value="buildSchema"/>
</properties>
</persistence-unit>
</persistence>
By the way, just a heads up - the benchmark site is being featured on
JavaLobby:
http://java.dzone.com/articles/new-jpa-performance-benchmark
I have mentioned at the end that initial OpenJPA tuning work has produced
better results.
Best regards,
Ilan
--
View this message in context: http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5723794.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: JPAB results
Posted by Georgi Naplatanov <go...@oles.biz>.
Hello Ilan.
You can consider to switch on statement pooling. Just add to property
"openjpa.ConnectionProperties" "poolPreparedStatements=true".
Best regards
Georgi
Ilan Kirsh wrote:
> Hi Pinaki,
>
> Thank you for the reference.
>
> Could you please verify the following persistence.xml file (which seems to
> run fine) before I run the entire benchmark again?
>
> <?xml version="1.0" encoding="UTF-8"?>
> <persistence xmlns="http://java.sun.com/xml/ns/persistence"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
> http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd" version="2.0">
> <persistence-unit name="OpenJPA2-MySQL-server">
>
> <provider>org.apache.openjpa.persistence.PersistenceProviderImpl</provider>
> <class>org.jpab.basic.Person</class>
> <class>org.jpab.col.CollectionPerson</class>
> <class>org.jpab.ext.PersonBase</class>
> <class>org.jpab.ext.PersonExt</class>
> <class>org.jpab.ext.PersonExtExt</class>
> <class>org.jpab.index.IndexedPerson</class>
> <class>org.jpab.node.Node</class>
> <properties>
> <property name="openjpa.ConnectionProperties"
> value="DriverClassName=com.mysql.jdbc.Driver,
> Url=jdbc:mysql://localhost:3306/jpab4897789836, MaxActive=100,
> MaxWait=10000, TestOnBorrow=true, Username=admin, Password=admin"/>
> <property name="openjpa.ConnectionDriverName"
> value="org.apache.commons.dbcp.BasicDataSource"/>
> <property name="openjpa.jdbc.DBDictionary" value="batchLimit=100"/>
> <property name="openjpa.DataCache" value="true"/>
> <property name="openjpa.Log" value="DefaultLevel=ERROR"/>
> <property name="openjpa.jdbc.SynchronizeMappings"
> value="buildSchema"/>
> </properties>
> </persistence-unit>
> </persistence>
>
>
> Regarding your previous post:
>
>
> Pinaki Poddar wrote:
>> "A huge performance gap has been detected when using simple basic entities
>> with small transaction/retrieval size. Comparing the normalized speed of
>> OpenJPA with MySQL database server (0.15) to the normalized speed of
>> Hibernate with MySQL database server (8.5) reveals that in that case,
>> Hibernate with MySQL server is 56.7 times faster than OpenJPA with MySQL
>> server. "
>>
>> All the OpenJPA users out there must be real dumb and must immediately
>> switch to something that is 56.7 times faster. Or not. When a benchmark
>> developer reports such drastic results -- they would be better off to
>> double-check their observations -- given that both the vendors had been in
>> operation for quite a while and users are *not* dumb. Actually such
>> drastic observations degrade one's confidence in these reported numbers
>> despite of them being presented with panache.
>>
>
> Please notice that 56.7 refers to an extreme edge case that has been
> detected automatically (this is a machine generated analysis) and not to the
> average performance gap.
>
> But if you look now at the new results:
> http://temp.jpab.org/OpenJPA_R2/MySQL/server/Hibernate/MySQL/server.html
> you can see that the maximum gap is now only 6 (the 56.7 has gone).
>
> Since I spent a lot of time in the last years on performance and
> optimizations, I know that a factor of 56 is not unusual. Look for example
> at:
> http://temp.jpab.org/OpenJPA/PostgreSQL/server/OpenJPA_R2/PostgreSQL/server.html#All
> You can see a performance gap of x200 - when comparing OpenJPA with
> PostreSQL to itself. One run with the OpenJPA 2.0 and out of the box
> configuration and the other run with OpenJPA 2.1 and some tuning.
>
> Jerry's observation should be taken in this context:
>
>
> No1UNo wrote:
>> This test suite was clearly designed around the strengths of ObjectDB.
>> The factor of six between the 'ObjectDB server' and 'Hibernate with
>> PostgreSQL server' performance is just not credible otherwise.
>>
>
> Since ObjectDB requires much less processing (no JPA-JDBC-SQL-Database
> layers and most of the time complete objects are processed as atomic byte[]
> elements, rather than as collections of fields) - a factor of 6 looks very
> reasonable. You can see that even the same product might produce a much
> larger factor of 200.
>
> Best regards,
> Ilan
Re: JPAB results
Posted by Ilan Kirsh <ki...@objectdb.com>.
Hi Pinaki,
Thank you for the reference.
Could you please verify the following persistence.xml file (which seems to
run fine) before I run the entire benchmark again?
<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd" version="2.0">
<persistence-unit name="OpenJPA2-MySQL-server">
<provider>org.apache.openjpa.persistence.PersistenceProviderImpl</provider>
<class>org.jpab.basic.Person</class>
<class>org.jpab.col.CollectionPerson</class>
<class>org.jpab.ext.PersonBase</class>
<class>org.jpab.ext.PersonExt</class>
<class>org.jpab.ext.PersonExtExt</class>
<class>org.jpab.index.IndexedPerson</class>
<class>org.jpab.node.Node</class>
<properties>
<property name="openjpa.ConnectionProperties"
value="DriverClassName=com.mysql.jdbc.Driver,
Url=jdbc:mysql://localhost:3306/jpab4897789836, MaxActive=100,
MaxWait=10000, TestOnBorrow=true, Username=admin, Password=admin"/>
<property name="openjpa.ConnectionDriverName"
value="org.apache.commons.dbcp.BasicDataSource"/>
<property name="openjpa.jdbc.DBDictionary" value="batchLimit=100"/>
<property name="openjpa.DataCache" value="true"/>
<property name="openjpa.Log" value="DefaultLevel=ERROR"/>
<property name="openjpa.jdbc.SynchronizeMappings"
value="buildSchema"/>
</properties>
</persistence-unit>
</persistence>
Regarding your previous post:
Pinaki Poddar wrote:
>
> "A huge performance gap has been detected when using simple basic entities
> with small transaction/retrieval size. Comparing the normalized speed of
> OpenJPA with MySQL database server (0.15) to the normalized speed of
> Hibernate with MySQL database server (8.5) reveals that in that case,
> Hibernate with MySQL server is 56.7 times faster than OpenJPA with MySQL
> server. "
>
> All the OpenJPA users out there must be real dumb and must immediately
> switch to something that is 56.7 times faster. Or not. When a benchmark
> developer reports such drastic results -- they would be better off to
> double-check their observations -- given that both the vendors had been in
> operation for quite a while and users are *not* dumb. Actually such
> drastic observations degrade one's confidence in these reported numbers
> despite of them being presented with panache.
>
Please notice that 56.7 refers to an extreme edge case that has been
detected automatically (this is a machine generated analysis) and not to the
average performance gap.
But if you look now at the new results:
http://temp.jpab.org/OpenJPA_R2/MySQL/server/Hibernate/MySQL/server.html
you can see that the maximum gap is now only 6 (the 56.7 has gone).
Since I spent a lot of time in the last years on performance and
optimizations, I know that a factor of 56 is not unusual. Look for example
at:
http://temp.jpab.org/OpenJPA/PostgreSQL/server/OpenJPA_R2/PostgreSQL/server.html#All
You can see a performance gap of x200 - when comparing OpenJPA with
PostreSQL to itself. One run with the OpenJPA 2.0 and out of the box
configuration and the other run with OpenJPA 2.1 and some tuning.
Jerry's observation should be taken in this context:
No1UNo wrote:
>
> This test suite was clearly designed around the strengths of ObjectDB.
> The factor of six between the 'ObjectDB server' and 'Hibernate with
> PostgreSQL server' performance is just not credible otherwise.
>
Since ObjectDB requires much less processing (no JPA-JDBC-SQL-Database
layers and most of the time complete objects are processed as atomic byte[]
elements, rather than as collections of fields) - a factor of 6 looks very
reasonable. You can see that even the same product might produce a much
larger factor of 200.
Best regards,
Ilan
--
View this message in context: http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5718395.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: JPAB results
Posted by Pinaki Poddar <pp...@apache.org>.
Ilan,
> which seems to have pooling on. If not, how can I enable this?
http://openjpa.apache.org/faq.html#FAQ-HowdoIenableconnectionpoolinginOpenJPA%253F
-----
Pinaki
--
View this message in context: http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5716932.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: JPAB results
Posted by Ilan Kirsh <ki...@objectdb.com>.
Thank you Jeremy for this important info. Many thanks also to yourself and
others on this forum for the helpful and constructive comments.
I ran the benchmark again following Jeremy's instructions:
Jeremy Bauer wrote:
>
> https://repository.apache.org/snapshots/org/apache/openjpa/apache-openjpa/2.1.0-SNAPSHOT/apache-openjpa-2.1.0-20101104.065025-93-binary.zip
>
> This snapshot has connection pooling included and enabled by default. If
> you'd like, you can enable basic, un-tuned, statement batching and data
> caching with these properties.
>
> property-3=<property name="openjpa.DataCache" value="true"/>
> property-4=<property name="openjpa.jdbc.DBDictionary"
> value="batchLimit=100"/>
>
I put the results on http://temp.jpab.org. The new results are presented as
OpenJPA_R2.
There are less failures now. All the tests pass when using H2 or HSQLDB in
embedded mode. There are still some other issues. Again, any
comments/criticisms are much appreciated.
As an aside, I've noticed that testing an ORM is much more involved than
testing an object database such as ObjectDB. There are many
provider/database combinations, and each database has its own issues. Would
you be interested in using JPAB (which is built for testing permutations) as
a framework for running your own tests easily against various DBMSs?
Regarding the database connection pool:
Georgi Naplatanov wrote:
>
> Do not forget to enable statement pooling in database connection pool
> configuration, if it is not enabled by default.
>
Thanks for the comment Georgi. I'm using the snapshot indicated by Jeremy
which seems to have pooling on. If not, how can I enable this? How can I
tell?
Best regards,
Ilan
--
View this message in context: http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5715056.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: JPAB results
Posted by Georgi Naplatanov <go...@oles.biz>.
Hi Ilan.
Do not forget to enable statement pooling in database connection pool
configuration, if it is not enabled by default.
Best regards
Georgi
Jeremy Bauer wrote:
> Hi Ilan,
>
> Thanks for your post, additional details about JPAB, and your willingness to
> try different OpenJPA configurations. While the performance gap is
> concerning, myself and others are especially concerned with the functional
> problems you are seeing. I downloaded the benchmark jar and was able to
> reproduce the errors you've documented. Running the multi-threaded tests
> with OpenJPA 2.0.0 on a multi-core system consistently produced duplicate
> key exceptions. I found that this issue is fixed in our 2.0.2-SNAPSHOT and
> also in 2.1.0-SNAPSHOT/trunk. However, the 2.0.2 snapshot runs into another
> threading issue, which is caused by OpenJPA's QuerySQLCache. By disabling
> the cache with the property below, I was able to get a clean run with
> 2.0.2-SNAPSHOT. Current trunk level code has some fixes in this area and
> worked without it.
>
> property-3=<property name="openjpa.jdbc.QuerySQLCache" value="false"/>
>
> I am doing a bit of experimentation with the benchmark in the areas of
> connection pooling, data caching, and statement batching. While OpenJPA has
> many performance tuning switches and dials, these three typically have the
> most impact. Some of the other providers have these performance features
> enabled by default, so enabling them in OpenJPA helps level the field.
>
> If you have time and wouldn't mind giving OpenJPA another go, you can get
> the latest 2.1.0-SNAPSHOT from here:
> https://repository.apache.org/snapshots/org/apache/openjpa/apache-openjpa/2.1.0-SNAPSHOT/
>
> as of this note the latest snapshot is
> https://repository.apache.org/snapshots/org/apache/openjpa/apache-openjpa/2.1.0-SNAPSHOT/apache-openjpa-2.1.0-20101104.065025-93-binary.zip
>
> This snapshot has connection pooling included and enabled by default. If
> you'd like, you can enable basic, un-tuned, statement batching and data
> caching with these properties.
>
> property-3=<property name="openjpa.DataCache" value="true"/>
> property-4=<property name="openjpa.jdbc.DBDictionary"
> value="batchLimit=100"/>
>
> I and I'm sure others others would be interested in the result. If I come
> up with a better set of basic tuning parameters, I'll pass them along.
> Others, please feel free to do the same.
>
> -Jeremy
>
> On Wed, Nov 3, 2010 at 11:24 AM, Ilan Kirsh <ki...@objectdb.com> wrote:
>
>> Hi all,
>>
>> I am the author of the http://www.jpab.org JPAB benchmark .
>>
>> As Rick wrote - this is indeed an out of the box benchmark. Default
>> configuration was used for all the tested products.
>>
>> I read with interest what you said about connection pooling. I am keen to
>> try this but I wonder whether it will really make a difference because the
>> tests do not use short term database connections, and anyway, the time of
>> connecting to the database is excluded from the time measurement.
>>
>> But probably there are other things that can be done to improve
>> performance.
>> I'd like to run the tests again with optimized OpenJPA configuration but
>> for
>> this I will need some help from OpenJPA experts.
>>
>> Regarding test failures - I have used standard JPA for the tests. I am
>> obviously keen to fix up any problems which are my errors, though, so I'd
>> be
>> very happy for details I have got incorrect to be pointed out and fixed.
>>
>> However, please notice that the same tests that failed with OpenJPA passes
>> with other combinations such as Hibernate/MySQL. Therefore, this could
>> really be a problem with the OpenJPA processing, as Kevin wrote. I think
>> that these tests indicate at least some issues that might have to be fixed
>> in OpenJPA.
>>
>> Finally, I'd like to address any concerns that the tests are specifically
>> designed to make http://www.objectdb.com ObjectDB look best. The tests
>> are
>> very simple and include only standard JPA operations on simple object
>> models. It is open source - does anyone see anything that is biased towards
>> ObjectDB performance? Actually, the performance gap could be even larger
>> if
>> the object model would be more complex. It is true, however, as explained
>> on
>> http://www.jpab.org/Benchmark_FAQ.html, that when the bottleneck is disk
>> activity or network overhead - performance gaps are expected to be much
>> lower.
>>
>> Best regards,
>> Ilan Kirsh
>> --
>> View this message in context:
>> http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5702044.html
>> Sent from the OpenJPA Users mailing list archive at Nabble.com.
>>
>
Re: JPAB results
Posted by Jeremy Bauer <te...@gmail.com>.
Hi Ilan,
Thanks for your post, additional details about JPAB, and your willingness to
try different OpenJPA configurations. While the performance gap is
concerning, myself and others are especially concerned with the functional
problems you are seeing. I downloaded the benchmark jar and was able to
reproduce the errors you've documented. Running the multi-threaded tests
with OpenJPA 2.0.0 on a multi-core system consistently produced duplicate
key exceptions. I found that this issue is fixed in our 2.0.2-SNAPSHOT and
also in 2.1.0-SNAPSHOT/trunk. However, the 2.0.2 snapshot runs into another
threading issue, which is caused by OpenJPA's QuerySQLCache. By disabling
the cache with the property below, I was able to get a clean run with
2.0.2-SNAPSHOT. Current trunk level code has some fixes in this area and
worked without it.
property-3=<property name="openjpa.jdbc.QuerySQLCache" value="false"/>
I am doing a bit of experimentation with the benchmark in the areas of
connection pooling, data caching, and statement batching. While OpenJPA has
many performance tuning switches and dials, these three typically have the
most impact. Some of the other providers have these performance features
enabled by default, so enabling them in OpenJPA helps level the field.
If you have time and wouldn't mind giving OpenJPA another go, you can get
the latest 2.1.0-SNAPSHOT from here:
https://repository.apache.org/snapshots/org/apache/openjpa/apache-openjpa/2.1.0-SNAPSHOT/
as of this note the latest snapshot is
https://repository.apache.org/snapshots/org/apache/openjpa/apache-openjpa/2.1.0-SNAPSHOT/apache-openjpa-2.1.0-20101104.065025-93-binary.zip
This snapshot has connection pooling included and enabled by default. If
you'd like, you can enable basic, un-tuned, statement batching and data
caching with these properties.
property-3=<property name="openjpa.DataCache" value="true"/>
property-4=<property name="openjpa.jdbc.DBDictionary"
value="batchLimit=100"/>
I and I'm sure others others would be interested in the result. If I come
up with a better set of basic tuning parameters, I'll pass them along.
Others, please feel free to do the same.
-Jeremy
On Wed, Nov 3, 2010 at 11:24 AM, Ilan Kirsh <ki...@objectdb.com> wrote:
>
> Hi all,
>
> I am the author of the http://www.jpab.org JPAB benchmark .
>
> As Rick wrote - this is indeed an out of the box benchmark. Default
> configuration was used for all the tested products.
>
> I read with interest what you said about connection pooling. I am keen to
> try this but I wonder whether it will really make a difference because the
> tests do not use short term database connections, and anyway, the time of
> connecting to the database is excluded from the time measurement.
>
> But probably there are other things that can be done to improve
> performance.
> I'd like to run the tests again with optimized OpenJPA configuration but
> for
> this I will need some help from OpenJPA experts.
>
> Regarding test failures - I have used standard JPA for the tests. I am
> obviously keen to fix up any problems which are my errors, though, so I'd
> be
> very happy for details I have got incorrect to be pointed out and fixed.
>
> However, please notice that the same tests that failed with OpenJPA passes
> with other combinations such as Hibernate/MySQL. Therefore, this could
> really be a problem with the OpenJPA processing, as Kevin wrote. I think
> that these tests indicate at least some issues that might have to be fixed
> in OpenJPA.
>
> Finally, I'd like to address any concerns that the tests are specifically
> designed to make http://www.objectdb.com ObjectDB look best. The tests
> are
> very simple and include only standard JPA operations on simple object
> models. It is open source - does anyone see anything that is biased towards
> ObjectDB performance? Actually, the performance gap could be even larger
> if
> the object model would be more complex. It is true, however, as explained
> on
> http://www.jpab.org/Benchmark_FAQ.html, that when the bottleneck is disk
> activity or network overhead - performance gaps are expected to be much
> lower.
>
> Best regards,
> Ilan Kirsh
> --
> View this message in context:
> http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5702044.html
> Sent from the OpenJPA Users mailing list archive at Nabble.com.
>
Re: JPAB results
Posted by Ilan Kirsh <ki...@objectdb.com>.
Hi all,
I am the author of the http://www.jpab.org JPAB benchmark .
As Rick wrote - this is indeed an out of the box benchmark. Default
configuration was used for all the tested products.
I read with interest what you said about connection pooling. I am keen to
try this but I wonder whether it will really make a difference because the
tests do not use short term database connections, and anyway, the time of
connecting to the database is excluded from the time measurement.
But probably there are other things that can be done to improve performance.
I'd like to run the tests again with optimized OpenJPA configuration but for
this I will need some help from OpenJPA experts.
Regarding test failures - I have used standard JPA for the tests. I am
obviously keen to fix up any problems which are my errors, though, so I'd be
very happy for details I have got incorrect to be pointed out and fixed.
However, please notice that the same tests that failed with OpenJPA passes
with other combinations such as Hibernate/MySQL. Therefore, this could
really be a problem with the OpenJPA processing, as Kevin wrote. I think
that these tests indicate at least some issues that might have to be fixed
in OpenJPA.
Finally, I'd like to address any concerns that the tests are specifically
designed to make http://www.objectdb.com ObjectDB look best. The tests are
very simple and include only standard JPA operations on simple object
models. It is open source - does anyone see anything that is biased towards
ObjectDB performance? Actually, the performance gap could be even larger if
the object model would be more complex. It is true, however, as explained on
http://www.jpab.org/Benchmark_FAQ.html, that when the bottleneck is disk
activity or network overhead - performance gaps are expected to be much
lower.
Best regards,
Ilan Kirsh
--
View this message in context: http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5702044.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: JPAB results
Posted by Michael Dick <mi...@gmail.com>.
Connection pooling is particularly important if the SynchronizeMappings
option is used, as I found in
OPENJPA-1847<https://issues.apache.org/jira/browse/OPENJPA-1847>.
The benchmark appears to use this property (I took a quick glance at the
openjpa related properties file).
Actually I'm very skeptical about any benchmark that creates the tables on
the fly. Such a configuration is useful in a test / dev environment but I
don't think many DBAs would allow it in production.
-mike
On Tue, Nov 2, 2010 at 9:15 AM, Rick Curtis <cu...@gmail.com> wrote:
> I also took a quick look and it appears that this is another out of the box
> test. Caching would probably bridge the gap between us and Hibernate...
> also
> I'm not sure they are running with the auto connection pooling stuff Donald
> has been working on.
>
> Thanks,
> Rick
>
> On Tue, Nov 2, 2010 at 8:49 AM, Kevin Sutter <kw...@gmail.com> wrote:
>
> > Although this "benchmark" is obviously biased, it's still good to be
> > reminded of potential pitfalls and/or inconsistencies. For example, I
> took
> > a quick look at one set of results and it seems to indicate a repeating
> > issue with obtaining a primary key sequence from the OpenJPA sequence
> > table. This could be a problem with the application, or it could be a
> > problem with the OpenJPA processing. Digging in a bit on these type of
> > issues would probably be worthwhile.
> >
> > But, spending a lot of time attempting to match the results of ObjectDB
> is
> > probably not that interesting...
> >
> > Kevin
> >
> > On Tue, Nov 2, 2010 at 7:58 AM, Georgi Naplatanov <go...@oles.biz>
> wrote:
> >
> > > Hi gkorland.
> > >
> > > Do not trust on this tests. It is better to make your own test and then
> > > decide which implementation is good for you.
> > >
> > > My testing indicates that OpenJPA (with or without cache) is faster
> than
> > > Hibernate on mostly write and less read operations.
> > >
> > > On read operation probably all JPA implementation are very close due
> the
> > > cache.
> > >
> > > Best regards
> > > Georgi
> > >
> > >
> > > gkorland wrote:
> > >
> > >> Did anyone review the JPAB (http://www.jpab.org/) results? It seems
> > like
> > >> all the other JPA guys are running faster and even better...
> > >>
> > >
> > >
> >
>
Re: JPAB results
Posted by Pinaki Poddar <pp...@apache.org>.
The "benchmark" quotes [1]
"A huge performance gap has been detected when using simple basic entities
with small transaction/retrieval size. Comparing the normalized speed of
OpenJPA with MySQL database server (0.15) to the normalized speed of
Hibernate with MySQL database server (8.5) reveals that in that case,
Hibernate with MySQL server is 56.7 times faster than OpenJPA with MySQL
server. "
All the OpenJPA users out there must be real dumb and must immediately
switch to something that is 56.7 times faster. Or not. When a benchmark
developer reports such drastic results -- they would be better off to
double-check their observations -- given that both the vendors had been in
operation for quite a while and users are *not* dumb. Actually such drastic
observations degrade one's confidence in these reported numbers despite of
them being presented with panache.
A cursory look tells that the OpenJPA tests are often failing on getting a
database connection. Looks like there is no connection pooling for OpenJPA.
If anyone care to run the tests (I refuse to call it a benchmark) to alley
that "56.7 times slower fear", will benefit from using a connection pool [2]
(please note that an auto-pooling of database connections are on the cards)
[1] http://www.jpab.org/Hibernate/MySQL/server/OpenJPA/MySQL/server.html
[2]
http://openjpa.apache.org/faq.html#FAQ-HowdoIenableconnectionpoolinginOpenJPA%253F
-----
Pinaki
--
View this message in context: http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5697918.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: JPAB results
Posted by Jeremy Bauer <te...@gmail.com>.
It looks like benchmark is configured to use openjpa-all-2.0.0 so it
wouldn't be using auto-enablement of DBCP. The auto-enablement wasn't
available until recently in openjpa-all-2.1.0-SNAPSHOT. I scanned some of
the benchmark config files and they are using direct-to-DB jdbc URL's, so
nor does it look like pooling is explicitly enabled. Like so many other
JSE-based benchmarks, I think we'd see more favorable OpenJPA results with
connection pooling.
-Jeremy
On Tue, Nov 2, 2010 at 9:15 AM, Rick Curtis <cu...@gmail.com> wrote:
> I also took a quick look and it appears that this is another out of the box
> test. Caching would probably bridge the gap between us and Hibernate...
> also
> I'm not sure they are running with the auto connection pooling stuff Donald
> has been working on.
>
> Thanks,
> Rick
>
> On Tue, Nov 2, 2010 at 8:49 AM, Kevin Sutter <kw...@gmail.com> wrote:
>
> > Although this "benchmark" is obviously biased, it's still good to be
> > reminded of potential pitfalls and/or inconsistencies. For example, I
> took
> > a quick look at one set of results and it seems to indicate a repeating
> > issue with obtaining a primary key sequence from the OpenJPA sequence
> > table. This could be a problem with the application, or it could be a
> > problem with the OpenJPA processing. Digging in a bit on these type of
> > issues would probably be worthwhile.
> >
> > But, spending a lot of time attempting to match the results of ObjectDB
> is
> > probably not that interesting...
> >
> > Kevin
> >
> > On Tue, Nov 2, 2010 at 7:58 AM, Georgi Naplatanov <go...@oles.biz>
> wrote:
> >
> > > Hi gkorland.
> > >
> > > Do not trust on this tests. It is better to make your own test and then
> > > decide which implementation is good for you.
> > >
> > > My testing indicates that OpenJPA (with or without cache) is faster
> than
> > > Hibernate on mostly write and less read operations.
> > >
> > > On read operation probably all JPA implementation are very close due
> the
> > > cache.
> > >
> > > Best regards
> > > Georgi
> > >
> > >
> > > gkorland wrote:
> > >
> > >> Did anyone review the JPAB (http://www.jpab.org/) results? It seems
> > like
> > >> all the other JPA guys are running faster and even better...
> > >>
> > >
> > >
> >
>
Re: JPAB results
Posted by Rick Curtis <cu...@gmail.com>.
I also took a quick look and it appears that this is another out of the box
test. Caching would probably bridge the gap between us and Hibernate... also
I'm not sure they are running with the auto connection pooling stuff Donald
has been working on.
Thanks,
Rick
On Tue, Nov 2, 2010 at 8:49 AM, Kevin Sutter <kw...@gmail.com> wrote:
> Although this "benchmark" is obviously biased, it's still good to be
> reminded of potential pitfalls and/or inconsistencies. For example, I took
> a quick look at one set of results and it seems to indicate a repeating
> issue with obtaining a primary key sequence from the OpenJPA sequence
> table. This could be a problem with the application, or it could be a
> problem with the OpenJPA processing. Digging in a bit on these type of
> issues would probably be worthwhile.
>
> But, spending a lot of time attempting to match the results of ObjectDB is
> probably not that interesting...
>
> Kevin
>
> On Tue, Nov 2, 2010 at 7:58 AM, Georgi Naplatanov <go...@oles.biz> wrote:
>
> > Hi gkorland.
> >
> > Do not trust on this tests. It is better to make your own test and then
> > decide which implementation is good for you.
> >
> > My testing indicates that OpenJPA (with or without cache) is faster than
> > Hibernate on mostly write and less read operations.
> >
> > On read operation probably all JPA implementation are very close due the
> > cache.
> >
> > Best regards
> > Georgi
> >
> >
> > gkorland wrote:
> >
> >> Did anyone review the JPAB (http://www.jpab.org/) results? It seems
> like
> >> all the other JPA guys are running faster and even better...
> >>
> >
> >
>
Re: JPAB results
Posted by No1UNo <je...@jerrycarter.org>.
Here's my quick take after a cursory glance through the results.
This test suite was clearly designed around the strengths of ObjectDB. The factor of six between the 'ObjectDB server' and 'Hibernate with PostgreSQL server' performance is just not credible otherwise.
Once you get past that, there is an even more troubling claim that no combination of OpenJPA with any database passed all 58 tests. It is not clear whether this is a bug in their tests or in OpenJPA, but that would seem to be worth examining. There are two tests which are reported as failing across all databases:
* Multithreading Test - Query Operations - Few Entities (5/100)
* Multithreading Test - Query Operations - Many Entities (5000)
It might be worth while to look at one or both of these first.
-=- Jerry
On Nov 2, 2010, at 9:50 AM, Kevin Sutter [via OpenJPA] wrote:
> Although this "benchmark" is obviously biased, it's still good to be
> reminded of potential pitfalls and/or inconsistencies. For example, I took
> a quick look at one set of results and it seems to indicate a repeating
> issue with obtaining a primary key sequence from the OpenJPA sequence
> table. This could be a problem with the application, or it could be a
> problem with the OpenJPA processing. Digging in a bit on these type of
> issues would probably be worthwhile.
>
> But, spending a lot of time attempting to match the results of ObjectDB is
> probably not that interesting...
>
> Kevin
>
> On Tue, Nov 2, 2010 at 7:58 AM, Georgi Naplatanov <[hidden email]> wrote:
>
> > Hi gkorland.
> >
> > Do not trust on this tests. It is better to make your own test and then
> > decide which implementation is good for you.
> >
> > My testing indicates that OpenJPA (with or without cache) is faster than
> > Hibernate on mostly write and less read operations.
> >
> > On read operation probably all JPA implementation are very close due the
> > cache.
> >
> > Best regards
> > Georgi
> >
> >
> > gkorland wrote:
> >
> >> Did anyone review the JPAB (http://www.jpab.org/) results? It seems like
> >> all the other JPA guys are running faster and even better...
> >>
> >
> >
>
>
> View message @ http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5697322.html
> To start a new topic under OpenJPA Users, email ml-node+208411-1703014788-244680@n2.nabble.com
> To unsubscribe from OpenJPA Users, click here.
>
--
View this message in context: http://openjpa.208410.n2.nabble.com/JPAB-results-tp5693298p5697524.html
Sent from the OpenJPA Users mailing list archive at Nabble.com.
Re: JPAB results
Posted by Kevin Sutter <kw...@gmail.com>.
Although this "benchmark" is obviously biased, it's still good to be
reminded of potential pitfalls and/or inconsistencies. For example, I took
a quick look at one set of results and it seems to indicate a repeating
issue with obtaining a primary key sequence from the OpenJPA sequence
table. This could be a problem with the application, or it could be a
problem with the OpenJPA processing. Digging in a bit on these type of
issues would probably be worthwhile.
But, spending a lot of time attempting to match the results of ObjectDB is
probably not that interesting...
Kevin
On Tue, Nov 2, 2010 at 7:58 AM, Georgi Naplatanov <go...@oles.biz> wrote:
> Hi gkorland.
>
> Do not trust on this tests. It is better to make your own test and then
> decide which implementation is good for you.
>
> My testing indicates that OpenJPA (with or without cache) is faster than
> Hibernate on mostly write and less read operations.
>
> On read operation probably all JPA implementation are very close due the
> cache.
>
> Best regards
> Georgi
>
>
> gkorland wrote:
>
>> Did anyone review the JPAB (http://www.jpab.org/) results? It seems like
>> all the other JPA guys are running faster and even better...
>>
>
>
Re: JPAB results
Posted by Georgi Naplatanov <go...@oles.biz>.
Hi gkorland.
Do not trust on this tests. It is better to make your own test and then
decide which implementation is good for you.
My testing indicates that OpenJPA (with or without cache) is faster than
Hibernate on mostly write and less read operations.
On read operation probably all JPA implementation are very close due the
cache.
Best regards
Georgi
gkorland wrote:
> Did anyone review the JPAB (http://www.jpab.org/) results?
> It seems like all the other JPA guys are running faster and even better...