You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openjpa.apache.org by Kevin Sutter <kw...@gmail.com> on 2006/10/26 04:24:44 UTC
Possible performance concerns?
Hi,
We're doing some primitive performance comparisons and the results could be
better... I'm wondering whether there are some configuration properties
that need to be tweaked to get more suitable performance. I've enabled the
DataCache and the QueryCache via the following properties, but they didn't
help much:
<property name="openjpa.DataCache" value="true"/>
<property name="openjpa.RemoteCommitProvider" value="sjvm"/>
<property name="openjpa.QueryCache" value="CacheSize=1000,
SoftReferenceSize=100"/>
Looking at some of the thread dumps in sample executions, there seems to be
a common theme with this reflective classloading. (The sample is a simple
jpql finder invocation that uses getSingleResult().) Is this normal
processing? Or, is there some configuration option(s) that would help with
this? Thanks for any ideas.
(BTW, this callstack is from a 0.9.0 driver, but we have also reproduced
this with a 0.9.5 driver as well.)
4XESTACKTRACE at
java/lang/ClassLoader.loadClass(ClassLoader.java:561(Compiled
Code))
4XESTACKTRACE at java/lang/Class.forNameImpl(Native Method)
4XESTACKTRACE at java/lang/Class.forName(Class.java:164(Compiled
Code))
4XESTACKTRACE at serp/util/Strings.toClass(Strings.java:160(Compiled
Code))
4XESTACKTRACE at serp/util/Strings.toClass(Strings.java:140(Compiled
Code))
4XESTACKTRACE at org/apache/openjpa/kernel/QueryImpl.toClass(
QueryImpl.java:1552(Compiled Code))
4XESTACKTRACE at org/apache/openjpa/kernel/QueryImpl.classForName(
QueryImpl.java:1499(Compiled Code))
4XESTACKTRACE at
org/apache/openjpa/kernel/ExpressionStoreQuery$1.classForName(
ExpressionStoreQuery.java:104(Compiled Code))
4XESTACKTRACE at
org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.getPathOrConstant(
JPQLExpressionBuilder.java:1247(Compiled Code))
4XESTACKTRACE at
org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.eval(
JPQLExpressionBuilder.java:861(Compiled Code))
4XESTACKTRACE at
org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.getValue(
JPQLExpressionBuilder.java:1356(Compiled Code))
4XESTACKTRACE at
org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.getValue(
JPQLExpressionBuilder.java:1342(Compiled Code))
4XESTACKTRACE at
org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.eval(
JPQLExpressionBuilder.java:702(Compiled Code))
4XESTACKTRACE at
org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.getExpression(
JPQLExpressionBuilder.java:1333(Compiled Code))
4XESTACKTRACE at
org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.eval(
JPQLExpressionBuilder.java:709(Compiled Code))
4XESTACKTRACE at
org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.evalWhereClause(
JPQLExpressionBuilder.java:484(Compiled Code))
4XESTACKTRACE at
org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.getQueryExpressions(
JPQLExpressionBuilder.java:261(Compiled Code))
4XESTACKTRACE at org/apache/openjpa/kernel/jpql/JPQLParser.eval(
JPQLParser.java:58(Compiled Code))
4XESTACKTRACE at
org/apache/openjpa/kernel/ExpressionStoreQuery$DataStoreExecutor.<init>(
ExpressionStoreQuery.java:648(Compiled Code))
4XESTACKTRACE at
org/apache/openjpa/kernel/ExpressionStoreQuery.newDataStoreExecutor(
ExpressionStoreQuery.java:162(Compiled Code))
4XESTACKTRACE at
org/apache/openjpa/kernel/QueryImpl.createExecutor(QueryImpl.java:675(Compiled
Code))
4XESTACKTRACE at
org/apache/openjpa/kernel/QueryImpl.compileForDataStore(QueryImpl.java:633(Compiled
Code))
4XESTACKTRACE at
org/apache/openjpa/kernel/QueryImpl.compileForExecutor(QueryImpl.java:609(Compiled
Code))
4XESTACKTRACE at org/apache/openjpa/kernel/QueryImpl.getOperation(
QueryImpl.java:1421(Compiled Code))
4XESTACKTRACE at
org/apache/openjpa/kernel/DelegatingQuery.getOperation(DelegatingQuery.java:120(Compiled
Code))
4XESTACKTRACE at org/apache/openjpa/persistence/QueryImpl.execute(
QueryImpl.java:205(Compiled Code))
4XESTACKTRACE at
org/apache/openjpa/persistence/QueryImpl.getSingleResult(QueryImpl.java:271(Compiled
Code))
Re: Possible performance concerns?
Posted by Abe White <aw...@bea.com>.
> I did a little playing around, in my non-exhaustive testing it
> seemed to
> call classForName each time you accessed a field on an Entity.
>
> "Select p.firstName from Person p where p.id = 1" called
> QueryImpl.classForName twice.
QueryImpl.classForName uses the MethodDataRepository method I
mentioned before doing anything else, so counting calls to that
method is not the same as couting Class.forName calls.
_______________________________________________________________________
Notice: This email message, together with any attachments, may contain
information of BEA Systems, Inc., its subsidiaries and affiliated
entities, that may be confidential, proprietary, copyrighted and/or
legally privileged, and is intended solely for the use of the individual
or entity named in this message. If you are not the intended recipient,
and have received this message in error, please immediately return this
by email and then delete it.
Re: Possible performance concerns?
Posted by Michael Dick <mi...@gmail.com>.
On 10/27/06, Abe White <aw...@bea.com> wrote:
>
> > Fair enough. But, unless Kodo is replacing the JPQL parsing, I
> > don't see
> > how you can get around this problem. I suppose it's possible that
> > other
> > performance improvements could negate the performance hit
> > associated with
> > the reflective class loading. But, it would seem that if Kodo 4.1
> > is using
> > OpenJPA, then these same type of issues would exist there.
>
> Kodo caches compiled queries.
Is there a plugin point where we could add a compiled query cache? It seems
like something that would be nice to have in OpenJPA, even if it isn't as
robust as the one in Kodo.
Also, regarding a name->type cache, we already have that in OpenJPA
> with the MetaDataRepository.getMetaData(String alias, ...) method. I
> don't know anything about our JPQL parsing, but I'm sure it's already
> using that method. Maybe it tries to use both that method and then
> reflective loading if no alias exists? Does it try to load anything
> other than the "from" clause members as classes?
I did a little playing around, in my non-exhaustive testing it seemed to
call classForName each time you accessed a field on an Entity.
"Select p.firstName from Person p where p.id = 1" called
QueryImpl.classForName twice.
"Select p.firstName from Person p" called classForName once.
I really haven't looked into this enough to know whether I'm just being
dense, is there a way to cache the class so we don't have to look it up each
time?
_______________________
>
________________________________________________
> Notice: This email message, together with any attachments, may contain
> information of BEA Systems, Inc., its subsidiaries and affiliated
> entities, that may be confidential, proprietary, copyrighted and/or
> legally privileged, and is intended solely for the use of the individual
> or entity named in this message. If you are not the intended recipient,
> and have received this message in error, please immediately return this
> by email and then delete it.
>
--
-Michael Dick
Re: Possible performance concerns?
Posted by Abe White <aw...@bea.com>.
> Fair enough. But, unless Kodo is replacing the JPQL parsing, I
> don't see
> how you can get around this problem. I suppose it's possible that
> other
> performance improvements could negate the performance hit
> associated with
> the reflective class loading. But, it would seem that if Kodo 4.1
> is using
> OpenJPA, then these same type of issues would exist there.
Kodo caches compiled queries.
Also, regarding a name->type cache, we already have that in OpenJPA
with the MetaDataRepository.getMetaData(String alias, ...) method. I
don't know anything about our JPQL parsing, but I'm sure it's already
using that method. Maybe it tries to use both that method and then
reflective loading if no alias exists? Does it try to load anything
other than the "from" clause members as classes?
_______________________________________________________________________
Notice: This email message, together with any attachments, may contain
information of BEA Systems, Inc., its subsidiaries and affiliated
entities, that may be confidential, proprietary, copyrighted and/or
legally privileged, and is intended solely for the use of the individual
or entity named in this message. If you are not the intended recipient,
and have received this message in error, please immediately return this
by email and then delete it.
Re: Possible performance concerns?
Posted by Kevin Sutter <kw...@gmail.com>.
On 10/27/06, Abe White <aw...@bea.com> wrote:
> Kodo has additional performance and caching features, so there may be
> hotspots in OpenJPA that don't exist in Kodo.
Fair enough. But, unless Kodo is replacing the JPQL parsing, I don't see
how you can get around this problem. I suppose it's possible that other
performance improvements could negate the performance hit associated with
the reflective class loading. But, it would seem that if Kodo 4.1 is using
OpenJPA, then these same type of issues would exist there.
I'm hoping our perf
> team will do some standalone OpenJPA testing at some point, but I
> don't know when that'll be.
Maybe this is the key part of your message -- no real performance testing
has been performed yet. So, maybe we're just now experiencing and
discovering these performance concerns at the same time.
Thanks,
Kevin
Re: Possible performance concerns?
Posted by Abe White <aw...@bea.com>.
> This was one of the thoughts we had as well. But, I wanted to
> check base
> with you guys first to see if maybe we just didn't have it tuned
> correctly.
> Have you pushed any performance runs with Kodo 4.1 yet?
Kodo has additional performance and caching features, so there may be
hotspots in OpenJPA that don't exist in Kodo. I'm hoping our perf
team will do some standalone OpenJPA testing at some point, but I
don't know when that'll be.
_______________________________________________________________________
Notice: This email message, together with any attachments, may contain
information of BEA Systems, Inc., its subsidiaries and affiliated
entities, that may be confidential, proprietary, copyrighted and/or
legally privileged, and is intended solely for the use of the individual
or entity named in this message. If you are not the intended recipient,
and have received this message in error, please immediately return this
by email and then delete it.
Re: Possible performance concerns?
Posted by Kevin Sutter <kw...@gmail.com>.
Marc,
Thanks for the reply. Embedded comments below...
On 10/26/06, Marc Prud'hommeaux <mp...@apache.org> wrote:
In some JPQL parsing, we need to check to see if a token is a class
> name or a variable or something else, and we only check that by
> trying a Class.forName(). What kind of performance hit are you seeing
> from this? Are these happening from the same query string, or
> different query strings?
This is from the same query string. In this primitive test, we're just
driving the same query over and over again. That's why we were surprised to
see the same Class.forName() invocation. My original thought was that maybe
the QueryCache would help with this, but that looks to be a cache for the
returned data.
In general, class loading slowness is one of those things that is
> frequently very fast in a stand-alone environment, but can slow down
> dramatically in a managed environment (due to the additional
> complexities of class loading that containers often need to have). It
> would be interesting if you could try the same set of queries in both
> a stand-alone and managed environment and compare the results.
Right now, all of the performance testing is in a stand-alone environment.
The managed environment isn't setup just yet.
It could be the sort of thing where we just need to cache name->class
> mappings (including lookup failures) in the JPQLExpressionBuilder and
> see if that helps at least with repetitive name lookups.
This was one of the thoughts we had as well. But, I wanted to check base
with you guys first to see if maybe we just didn't have it tuned correctly.
Have you pushed any performance runs with Kodo 4.1 yet?
Thanks,
Kevin
On Oct 25, 2006, at 7:24 PM, Kevin Sutter wrote:
>
> > Hi,
> > We're doing some primitive performance comparisons and the results
> > could be
> > better... I'm wondering whether there are some configuration
> > properties
> > that need to be tweaked to get more suitable performance. I've
> > enabled the
> > DataCache and the QueryCache via the following properties, but they
> > didn't
> > help much:
> >
> > <property name="openjpa.DataCache" value="true"/>
> > <property name="openjpa.RemoteCommitProvider"
> > value="sjvm"/>
> > <property name="openjpa.QueryCache" value="CacheSize=1000,
> > SoftReferenceSize=100"/>
> >
> > Looking at some of the thread dumps in sample executions, there
> > seems to be
> > a common theme with this reflective classloading. (The sample is a
> > simple
> > jpql finder invocation that uses getSingleResult().) Is this normal
> > processing? Or, is there some configuration option(s) that would
> > help with
> > this? Thanks for any ideas.
> >
> > (BTW, this callstack is from a 0.9.0 driver, but we have also
> > reproduced
> > this with a 0.9.5 driver as well.)
> >
> > 4XESTACKTRACE at
> > java/lang/ClassLoader.loadClass(ClassLoader.java:561(Compiled
> > Code))
> > 4XESTACKTRACE at java/lang/Class.forNameImpl(Native Method)
> > 4XESTACKTRACE at java/lang/Class.forName(Class.java:164
> > (Compiled
> > Code))
> > 4XESTACKTRACE at serp/util/Strings.toClass(Strings.java:160
> > (Compiled
> > Code))
> > 4XESTACKTRACE at serp/util/Strings.toClass(Strings.java:140
> > (Compiled
> > Code))
> > 4XESTACKTRACE at org/apache/openjpa/kernel/QueryImpl.toClass(
> > QueryImpl.java:1552(Compiled Code))
> > 4XESTACKTRACE at org/apache/openjpa/kernel/
> > QueryImpl.classForName(
> > QueryImpl.java:1499(Compiled Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/kernel/ExpressionStoreQuery$1.classForName(
> > ExpressionStoreQuery.java:104(Compiled Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/kernel/jpql/
> > JPQLExpressionBuilder.getPathOrConstant(
> > JPQLExpressionBuilder.java:1247(Compiled Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.eval(
> > JPQLExpressionBuilder.java:861(Compiled Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.getValue(
> > JPQLExpressionBuilder.java:1356(Compiled Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.getValue(
> > JPQLExpressionBuilder.java:1342(Compiled Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.eval(
> > JPQLExpressionBuilder.java:702(Compiled Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.getExpression(
> > JPQLExpressionBuilder.java:1333(Compiled Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.eval(
> > JPQLExpressionBuilder.java:709(Compiled Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.evalWhereClause(
> > JPQLExpressionBuilder.java:484(Compiled Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/kernel/jpql/
> > JPQLExpressionBuilder.getQueryExpressions(
> > JPQLExpressionBuilder.java:261(Compiled Code))
> > 4XESTACKTRACE at org/apache/openjpa/kernel/jpql/
> > JPQLParser.eval(
> > JPQLParser.java:58(Compiled Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/kernel/ExpressionStoreQuery
> > $DataStoreExecutor.<init>(
> > ExpressionStoreQuery.java:648(Compiled Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/kernel/ExpressionStoreQuery.newDataStoreExecutor(
> > ExpressionStoreQuery.java:162(Compiled Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/kernel/QueryImpl.createExecutor(QueryImpl.java:
> > 675(Compiled
> > Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/kernel/QueryImpl.compileForDataStore
> > (QueryImpl.java:633(Compiled
> > Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/kernel/QueryImpl.compileForExecutor
> > (QueryImpl.java:609(Compiled
> > Code))
> > 4XESTACKTRACE at org/apache/openjpa/kernel/
> > QueryImpl.getOperation(
> > QueryImpl.java:1421(Compiled Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/kernel/DelegatingQuery.getOperation
> > (DelegatingQuery.java:120(Compiled
> > Code))
> > 4XESTACKTRACE at org/apache/openjpa/persistence/
> > QueryImpl.execute(
> > QueryImpl.java:205(Compiled Code))
> > 4XESTACKTRACE at
> > org/apache/openjpa/persistence/QueryImpl.getSingleResult
> > (QueryImpl.java:271(Compiled
> > Code))
>
>
Re: Possible performance concerns?
Posted by Marc Prud'hommeaux <mp...@apache.org>.
Kevin-
In some JPQL parsing, we need to check to see if a token is a class
name or a variable or something else, and we only check that by
trying a Class.forName(). What kind of performance hit are you seeing
from this? Are these happening from the same query string, or
different query strings?
In general, class loading slowness is one of those things that is
frequently very fast in a stand-alone environment, but can slow down
dramatically in a managed environment (due to the additional
complexities of class loading that containers often need to have). It
would be interesting if you could try the same set of queries in both
a stand-alone and managed environment and compare the results.
It could be the sort of thing where we just need to cache name->class
mappings (including lookup failures) in the JPQLExpressionBuilder and
see if that helps at least with repetitive name lookups.
On Oct 25, 2006, at 7:24 PM, Kevin Sutter wrote:
> Hi,
> We're doing some primitive performance comparisons and the results
> could be
> better... I'm wondering whether there are some configuration
> properties
> that need to be tweaked to get more suitable performance. I've
> enabled the
> DataCache and the QueryCache via the following properties, but they
> didn't
> help much:
>
> <property name="openjpa.DataCache" value="true"/>
> <property name="openjpa.RemoteCommitProvider"
> value="sjvm"/>
> <property name="openjpa.QueryCache" value="CacheSize=1000,
> SoftReferenceSize=100"/>
>
> Looking at some of the thread dumps in sample executions, there
> seems to be
> a common theme with this reflective classloading. (The sample is a
> simple
> jpql finder invocation that uses getSingleResult().) Is this normal
> processing? Or, is there some configuration option(s) that would
> help with
> this? Thanks for any ideas.
>
> (BTW, this callstack is from a 0.9.0 driver, but we have also
> reproduced
> this with a 0.9.5 driver as well.)
>
> 4XESTACKTRACE at
> java/lang/ClassLoader.loadClass(ClassLoader.java:561(Compiled
> Code))
> 4XESTACKTRACE at java/lang/Class.forNameImpl(Native Method)
> 4XESTACKTRACE at java/lang/Class.forName(Class.java:164
> (Compiled
> Code))
> 4XESTACKTRACE at serp/util/Strings.toClass(Strings.java:160
> (Compiled
> Code))
> 4XESTACKTRACE at serp/util/Strings.toClass(Strings.java:140
> (Compiled
> Code))
> 4XESTACKTRACE at org/apache/openjpa/kernel/QueryImpl.toClass(
> QueryImpl.java:1552(Compiled Code))
> 4XESTACKTRACE at org/apache/openjpa/kernel/
> QueryImpl.classForName(
> QueryImpl.java:1499(Compiled Code))
> 4XESTACKTRACE at
> org/apache/openjpa/kernel/ExpressionStoreQuery$1.classForName(
> ExpressionStoreQuery.java:104(Compiled Code))
> 4XESTACKTRACE at
> org/apache/openjpa/kernel/jpql/
> JPQLExpressionBuilder.getPathOrConstant(
> JPQLExpressionBuilder.java:1247(Compiled Code))
> 4XESTACKTRACE at
> org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.eval(
> JPQLExpressionBuilder.java:861(Compiled Code))
> 4XESTACKTRACE at
> org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.getValue(
> JPQLExpressionBuilder.java:1356(Compiled Code))
> 4XESTACKTRACE at
> org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.getValue(
> JPQLExpressionBuilder.java:1342(Compiled Code))
> 4XESTACKTRACE at
> org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.eval(
> JPQLExpressionBuilder.java:702(Compiled Code))
> 4XESTACKTRACE at
> org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.getExpression(
> JPQLExpressionBuilder.java:1333(Compiled Code))
> 4XESTACKTRACE at
> org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.eval(
> JPQLExpressionBuilder.java:709(Compiled Code))
> 4XESTACKTRACE at
> org/apache/openjpa/kernel/jpql/JPQLExpressionBuilder.evalWhereClause(
> JPQLExpressionBuilder.java:484(Compiled Code))
> 4XESTACKTRACE at
> org/apache/openjpa/kernel/jpql/
> JPQLExpressionBuilder.getQueryExpressions(
> JPQLExpressionBuilder.java:261(Compiled Code))
> 4XESTACKTRACE at org/apache/openjpa/kernel/jpql/
> JPQLParser.eval(
> JPQLParser.java:58(Compiled Code))
> 4XESTACKTRACE at
> org/apache/openjpa/kernel/ExpressionStoreQuery
> $DataStoreExecutor.<init>(
> ExpressionStoreQuery.java:648(Compiled Code))
> 4XESTACKTRACE at
> org/apache/openjpa/kernel/ExpressionStoreQuery.newDataStoreExecutor(
> ExpressionStoreQuery.java:162(Compiled Code))
> 4XESTACKTRACE at
> org/apache/openjpa/kernel/QueryImpl.createExecutor(QueryImpl.java:
> 675(Compiled
> Code))
> 4XESTACKTRACE at
> org/apache/openjpa/kernel/QueryImpl.compileForDataStore
> (QueryImpl.java:633(Compiled
> Code))
> 4XESTACKTRACE at
> org/apache/openjpa/kernel/QueryImpl.compileForExecutor
> (QueryImpl.java:609(Compiled
> Code))
> 4XESTACKTRACE at org/apache/openjpa/kernel/
> QueryImpl.getOperation(
> QueryImpl.java:1421(Compiled Code))
> 4XESTACKTRACE at
> org/apache/openjpa/kernel/DelegatingQuery.getOperation
> (DelegatingQuery.java:120(Compiled
> Code))
> 4XESTACKTRACE at org/apache/openjpa/persistence/
> QueryImpl.execute(
> QueryImpl.java:205(Compiled Code))
> 4XESTACKTRACE at
> org/apache/openjpa/persistence/QueryImpl.getSingleResult
> (QueryImpl.java:271(Compiled
> Code))