You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openjpa.apache.org by Kevin Sutter <kw...@gmail.com> on 2007/10/11 17:59:55 UTC
Question: Application of QueryCompilationCache for find() and lazy relationships?
Hi,
For those of you that are more verse with the QueryCompilationCache, I'm
wondering how difficult it would be to utilize this type of cache for the
em.find() operations as well as traversal of lazy relationships. We're
finding that these operations are more prevalent in customer's applications
than the use of createQuery() and createNamedQuery() operations and it's
affecting our performance (especially as compared to our competitors). So,
I am wondering whether anybody has put any thought in this space before I do
a deeper dive. Any suggestions or thoughts?
Thanks,
Kevin
Re: Question: Application of QueryCompilationCache for find() and lazy relationships?
Posted by Kevin Sutter <kw...@gmail.com>.
On 10/17/07, Don Brady <db...@comintel.com> wrote:
>
> Just to add a user note if not inappropriate, my company would really
> like to see this.
>
> We are finding that our OpenJPA application runs slower than a pure JDBC
> application and it looks as if the re-generation of the dynamic SQL is a
> major factor....
Right. It's going to be tough to perform as well as pure JDBC, but it's a
nice goal... :-) The flexibility of dynamic SQL and the abstraction from
the individual database idiosyncrasies makes OpenJPA a viable alternative.
I think we can improve the area of SQL generation with the proper caching.
This is just a tricky area of the code...
By the way, on another note, I have never seen anything as robust as
> OpenJPA is proving to be - we have not run into a single bug or memory
> leak since we deployed our application to Production on WebSphere 6.1
> the better part of a year ago. I expected at least the occasional
> issue, but no problems at all have transpired!
Thank you for the testimonial! I know from our interactions on both this
forum and the WebSphere forum that you have had your share of questions...
:-) But, it's great to hear that your experience with both OpenJPA and
WebSphere has been so positive.
Thanks,
Kevin
Don
>
>
> Patrick Linskey wrote:
> > Hi,
> >
> > Yes, SQL caching will be a big win. Sadly, as you point out, there is
> > no one clear spot to do the caching, and a number of things must be
> > incorporated into the key for the cache. That said, it's definitely
> > doable.
> >
> > -Patrick
> >
> > On 10/16/07, Kevin Sutter <kw...@gmail.com> wrote:
> >> Patrick,
> >> As I continued to look at this area, it really wasn't the lack of the
> Query
> >> Compilation Cache that was causing the performance concerns (as I
> believe
> >> you had already figured out). But, rather, it's the constant
> re-generation
> >> of the dynamic SQL that is causing the performance concerns. Even when
> >> dealing with named queries, the SQL is always generated from scratch
> when
> >> performing the execute().
> >>
> >> I've started to look at possibly caching the generated SQL or at least
> >> aspects of the SQL string, but so far, I have not found the "magic
> spots"
> >> for any necessary caches. The generation of the SQL seems to be
> tightly
> >> bound with the processing of the query itself (setting parameters,
> setting
> >> hints, setting max results, etc). Anyway, I am still digging into this
> >> during my spare time to see if I can help out with this performance
> >> concern...
> >>
> >> Thanks,
> >> Kevin
> >>
> >> On 10/11/07, Patrick Linskey <pl...@gmail.com> wrote:
> >>> The query compilation cache doesn't cache data that would be directly
> >>> useful for those calls. However, I'm all for optimizing those
> >>> pathways, and the query compilation cache could be a place to store
> >>> whatever intermediate caches we are able to assemble.
> >>>
> >>> -Patrick
> >>>
> >>> On 10/11/07, Kevin Sutter <kw...@gmail.com> wrote:
> >>>> Hi,
> >>>> For those of you that are more verse with the QueryCompilationCache,
> I'm
> >>>> wondering how difficult it would be to utilize this type of cache for
> >>> the
> >>>> em.find() operations as well as traversal of lazy
> relationships. We're
> >>>> finding that these operations are more prevalent in customer's
> >>> applications
> >>>> than the use of createQuery() and createNamedQuery() operations and
> it's
> >>>> affecting our performance (especially as compared to our
> >>> competitors). So,
> >>>> I am wondering whether anybody has put any thought in this space
> before
> >>> I do
> >>>> a deeper dive. Any suggestions or thoughts?
> >>>>
> >>>> Thanks,
> >>>> Kevin
> >>>>
> >>>
> >>> --
> >>> Patrick Linskey
> >>> 202 669 5907
> >>>
> >
> >
>
>
Re: Question: Application of QueryCompilationCache for find() and lazy relationships?
Posted by Patrick Linskey <pl...@gmail.com>.
Do either of you guys have any microbenchmarks that highlight this by
any chance? I've got an idea that might work.
-Patrick
On 10/16/07, Don Brady <db...@comintel.com> wrote:
> Just to add a user note if not inappropriate, my company would really
> like to see this.
>
> We are finding that our OpenJPA application runs slower than a pure JDBC
> application and it looks as if the re-generation of the dynamic SQL is a
> major factor....
>
> By the way, on another note, I have never seen anything as robust as
> OpenJPA is proving to be - we have not run into a single bug or memory
> leak since we deployed our application to Production on WebSphere 6.1
> the better part of a year ago. I expected at least the occasional
> issue, but no problems at all have transpired!
>
> Don
>
>
> Patrick Linskey wrote:
> > Hi,
> >
> > Yes, SQL caching will be a big win. Sadly, as you point out, there is
> > no one clear spot to do the caching, and a number of things must be
> > incorporated into the key for the cache. That said, it's definitely
> > doable.
> >
> > -Patrick
> >
> > On 10/16/07, Kevin Sutter <kw...@gmail.com> wrote:
> >> Patrick,
> >> As I continued to look at this area, it really wasn't the lack of the Query
> >> Compilation Cache that was causing the performance concerns (as I believe
> >> you had already figured out). But, rather, it's the constant re-generation
> >> of the dynamic SQL that is causing the performance concerns. Even when
> >> dealing with named queries, the SQL is always generated from scratch when
> >> performing the execute().
> >>
> >> I've started to look at possibly caching the generated SQL or at least
> >> aspects of the SQL string, but so far, I have not found the "magic spots"
> >> for any necessary caches. The generation of the SQL seems to be tightly
> >> bound with the processing of the query itself (setting parameters, setting
> >> hints, setting max results, etc). Anyway, I am still digging into this
> >> during my spare time to see if I can help out with this performance
> >> concern...
> >>
> >> Thanks,
> >> Kevin
> >>
> >> On 10/11/07, Patrick Linskey <pl...@gmail.com> wrote:
> >>> The query compilation cache doesn't cache data that would be directly
> >>> useful for those calls. However, I'm all for optimizing those
> >>> pathways, and the query compilation cache could be a place to store
> >>> whatever intermediate caches we are able to assemble.
> >>>
> >>> -Patrick
> >>>
> >>> On 10/11/07, Kevin Sutter <kw...@gmail.com> wrote:
> >>>> Hi,
> >>>> For those of you that are more verse with the QueryCompilationCache, I'm
> >>>> wondering how difficult it would be to utilize this type of cache for
> >>> the
> >>>> em.find() operations as well as traversal of lazy relationships. We're
> >>>> finding that these operations are more prevalent in customer's
> >>> applications
> >>>> than the use of createQuery() and createNamedQuery() operations and it's
> >>>> affecting our performance (especially as compared to our
> >>> competitors). So,
> >>>> I am wondering whether anybody has put any thought in this space before
> >>> I do
> >>>> a deeper dive. Any suggestions or thoughts?
> >>>>
> >>>> Thanks,
> >>>> Kevin
> >>>>
> >>>
> >>> --
> >>> Patrick Linskey
> >>> 202 669 5907
> >>>
> >
> >
>
>
--
Patrick Linskey
202 669 5907
Re: Question: Application of QueryCompilationCache for find() and
lazy relationships?
Posted by Don Brady <db...@comintel.com>.
Just to add a user note if not inappropriate, my company would really
like to see this.
We are finding that our OpenJPA application runs slower than a pure JDBC
application and it looks as if the re-generation of the dynamic SQL is a
major factor....
By the way, on another note, I have never seen anything as robust as
OpenJPA is proving to be - we have not run into a single bug or memory
leak since we deployed our application to Production on WebSphere 6.1
the better part of a year ago. I expected at least the occasional
issue, but no problems at all have transpired!
Don
Patrick Linskey wrote:
> Hi,
>
> Yes, SQL caching will be a big win. Sadly, as you point out, there is
> no one clear spot to do the caching, and a number of things must be
> incorporated into the key for the cache. That said, it's definitely
> doable.
>
> -Patrick
>
> On 10/16/07, Kevin Sutter <kw...@gmail.com> wrote:
>> Patrick,
>> As I continued to look at this area, it really wasn't the lack of the Query
>> Compilation Cache that was causing the performance concerns (as I believe
>> you had already figured out). But, rather, it's the constant re-generation
>> of the dynamic SQL that is causing the performance concerns. Even when
>> dealing with named queries, the SQL is always generated from scratch when
>> performing the execute().
>>
>> I've started to look at possibly caching the generated SQL or at least
>> aspects of the SQL string, but so far, I have not found the "magic spots"
>> for any necessary caches. The generation of the SQL seems to be tightly
>> bound with the processing of the query itself (setting parameters, setting
>> hints, setting max results, etc). Anyway, I am still digging into this
>> during my spare time to see if I can help out with this performance
>> concern...
>>
>> Thanks,
>> Kevin
>>
>> On 10/11/07, Patrick Linskey <pl...@gmail.com> wrote:
>>> The query compilation cache doesn't cache data that would be directly
>>> useful for those calls. However, I'm all for optimizing those
>>> pathways, and the query compilation cache could be a place to store
>>> whatever intermediate caches we are able to assemble.
>>>
>>> -Patrick
>>>
>>> On 10/11/07, Kevin Sutter <kw...@gmail.com> wrote:
>>>> Hi,
>>>> For those of you that are more verse with the QueryCompilationCache, I'm
>>>> wondering how difficult it would be to utilize this type of cache for
>>> the
>>>> em.find() operations as well as traversal of lazy relationships. We're
>>>> finding that these operations are more prevalent in customer's
>>> applications
>>>> than the use of createQuery() and createNamedQuery() operations and it's
>>>> affecting our performance (especially as compared to our
>>> competitors). So,
>>>> I am wondering whether anybody has put any thought in this space before
>>> I do
>>>> a deeper dive. Any suggestions or thoughts?
>>>>
>>>> Thanks,
>>>> Kevin
>>>>
>>>
>>> --
>>> Patrick Linskey
>>> 202 669 5907
>>>
>
>
Re: Question: Application of QueryCompilationCache for find() and lazy relationships?
Posted by Patrick Linskey <pl...@gmail.com>.
Hi,
Yes, SQL caching will be a big win. Sadly, as you point out, there is
no one clear spot to do the caching, and a number of things must be
incorporated into the key for the cache. That said, it's definitely
doable.
-Patrick
On 10/16/07, Kevin Sutter <kw...@gmail.com> wrote:
> Patrick,
> As I continued to look at this area, it really wasn't the lack of the Query
> Compilation Cache that was causing the performance concerns (as I believe
> you had already figured out). But, rather, it's the constant re-generation
> of the dynamic SQL that is causing the performance concerns. Even when
> dealing with named queries, the SQL is always generated from scratch when
> performing the execute().
>
> I've started to look at possibly caching the generated SQL or at least
> aspects of the SQL string, but so far, I have not found the "magic spots"
> for any necessary caches. The generation of the SQL seems to be tightly
> bound with the processing of the query itself (setting parameters, setting
> hints, setting max results, etc). Anyway, I am still digging into this
> during my spare time to see if I can help out with this performance
> concern...
>
> Thanks,
> Kevin
>
> On 10/11/07, Patrick Linskey <pl...@gmail.com> wrote:
> >
> > The query compilation cache doesn't cache data that would be directly
> > useful for those calls. However, I'm all for optimizing those
> > pathways, and the query compilation cache could be a place to store
> > whatever intermediate caches we are able to assemble.
> >
> > -Patrick
> >
> > On 10/11/07, Kevin Sutter <kw...@gmail.com> wrote:
> > > Hi,
> > > For those of you that are more verse with the QueryCompilationCache, I'm
> > > wondering how difficult it would be to utilize this type of cache for
> > the
> > > em.find() operations as well as traversal of lazy relationships. We're
> > > finding that these operations are more prevalent in customer's
> > applications
> > > than the use of createQuery() and createNamedQuery() operations and it's
> > > affecting our performance (especially as compared to our
> > competitors). So,
> > > I am wondering whether anybody has put any thought in this space before
> > I do
> > > a deeper dive. Any suggestions or thoughts?
> > >
> > > Thanks,
> > > Kevin
> > >
> >
> >
> > --
> > Patrick Linskey
> > 202 669 5907
> >
>
--
Patrick Linskey
202 669 5907
Re: Question: Application of QueryCompilationCache for find() and lazy relationships?
Posted by Kevin Sutter <kw...@gmail.com>.
Patrick,
As I continued to look at this area, it really wasn't the lack of the Query
Compilation Cache that was causing the performance concerns (as I believe
you had already figured out). But, rather, it's the constant re-generation
of the dynamic SQL that is causing the performance concerns. Even when
dealing with named queries, the SQL is always generated from scratch when
performing the execute().
I've started to look at possibly caching the generated SQL or at least
aspects of the SQL string, but so far, I have not found the "magic spots"
for any necessary caches. The generation of the SQL seems to be tightly
bound with the processing of the query itself (setting parameters, setting
hints, setting max results, etc). Anyway, I am still digging into this
during my spare time to see if I can help out with this performance
concern...
Thanks,
Kevin
On 10/11/07, Patrick Linskey <pl...@gmail.com> wrote:
>
> The query compilation cache doesn't cache data that would be directly
> useful for those calls. However, I'm all for optimizing those
> pathways, and the query compilation cache could be a place to store
> whatever intermediate caches we are able to assemble.
>
> -Patrick
>
> On 10/11/07, Kevin Sutter <kw...@gmail.com> wrote:
> > Hi,
> > For those of you that are more verse with the QueryCompilationCache, I'm
> > wondering how difficult it would be to utilize this type of cache for
> the
> > em.find() operations as well as traversal of lazy relationships. We're
> > finding that these operations are more prevalent in customer's
> applications
> > than the use of createQuery() and createNamedQuery() operations and it's
> > affecting our performance (especially as compared to our
> competitors). So,
> > I am wondering whether anybody has put any thought in this space before
> I do
> > a deeper dive. Any suggestions or thoughts?
> >
> > Thanks,
> > Kevin
> >
>
>
> --
> Patrick Linskey
> 202 669 5907
>
Re: Question: Application of QueryCompilationCache for find() and lazy relationships?
Posted by Patrick Linskey <pl...@gmail.com>.
The query compilation cache doesn't cache data that would be directly
useful for those calls. However, I'm all for optimizing those
pathways, and the query compilation cache could be a place to store
whatever intermediate caches we are able to assemble.
-Patrick
On 10/11/07, Kevin Sutter <kw...@gmail.com> wrote:
> Hi,
> For those of you that are more verse with the QueryCompilationCache, I'm
> wondering how difficult it would be to utilize this type of cache for the
> em.find() operations as well as traversal of lazy relationships. We're
> finding that these operations are more prevalent in customer's applications
> than the use of createQuery() and createNamedQuery() operations and it's
> affecting our performance (especially as compared to our competitors). So,
> I am wondering whether anybody has put any thought in this space before I do
> a deeper dive. Any suggestions or thoughts?
>
> Thanks,
> Kevin
>
--
Patrick Linskey
202 669 5907