You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cayenne.apache.org by Marcin Skladaniec <ma...@ish.com.au> on 2007/11/02 01:50:40 UTC
localObject vs prefetches: would there be a benefit ?
Hi
We are improving the performance of our client application (ROP). Not
surprisingly we have found that adding prefetches did improve the
speed significantly.
I have a question though about which is about efficiency of
localObject. We are fetching a list of objects (query with prefetches)
to a specific one non-editable context. If the object is to be edited
it has to be copied to another context which allows committing
changes. After the object is copied all the related object are
accessed, so the performance of the copying came to my mind.
The test I have looks like this:
CayenneContext context1;
CayenneContext context2;
SelectQuery q = new SelectQuery(Painting.class);
q.addPrefetch(Painting.GALLERY_PROPERTY);
q.addPrefetch(Painting. GALLERY_PROPERTY + "." + Gallery.CITY_PROPERTY);
List l = context1.performQuery(q);
//now the context1 contain all the records I wanted it to contain
Painting p1 = (Painting) l.get(0);
Painting p2 = (Painting) context2.localObject(p1.getObjectId(), null);
//now the context2 contain only the single record I copied
p2.getGallery().getCity();
//now context2 contains the same objects as context1
All is very quick, but my question is how it will scale when the
relationship would be to-many and there will be hundreds related
records.
What do you think would be a threshold number of related objects which
have to be localised over which it would be worth doing a new,
specific select query for that single object (with prefetches). Would
there be advantage of doing that at all ?
We are using cutting edge version of cayenne (I think we use build
about 2-3 weeks old).
Marcin
Re: localObject vs prefetches: would there be a benefit ?
Posted by Kevin Menard <km...@servprise.com>.
I think his question is really about the speed of resolving faults. That
doesn't really change anything, however. If the object is cached, the
resolved fault will pull from cache. Otherwise, it will only pull the
necessary object out of the DB.
Now, it is possible for this to degrade under the right conditions, but I'm
having a hard time envisioning them. Particularly because if you're using
localObject(), you really should be using shared caches. If not, you're
going to see some pretty strange behavior and I would not recommend it all.
--
Kevin
On 11/3/07 1:25 PM, "Mike Kienenberger" <mk...@gmail.com> wrote:
> Fetching one painting from the database is slower than copying one object.
> Fetching 300 paintings from the database is slower than copying 300 objects.
>
> Copy operations run at the speed of memory. Database operations run
> at the speed of networking and/or file I/O.
>
>
> On 11/2/07, Marcin Skladaniec <ma...@ish.com.au> wrote:
>> Hi Mike
>> Thats exactly what I mean: localObject is copying one object at the
>> time. So say I have in my contextA 1 Gallery with 300 related
>> Paintings, all with PersistentState.COMMITED. I do
>> contextB.localObject on Gallery, then I access every Painting via
>> relaionship. this means that the localObject will be invoked 301
>> times, everytime with single object. I believe this will be faster
>> than re-fetching the data, but what if there will be 3000 Paintings ?
>> Is there a limit after which the localObject becomes slower than re-
>> fetch ?
>>
>> Marcin
Re: localObject vs prefetches: would there be a benefit ?
Posted by Mike Kienenberger <mk...@gmail.com>.
Fetching one painting from the database is slower than copying one object.
Fetching 300 paintings from the database is slower than copying 300 objects.
Copy operations run at the speed of memory. Database operations run
at the speed of networking and/or file I/O.
On 11/2/07, Marcin Skladaniec <ma...@ish.com.au> wrote:
> Hi Mike
> Thats exactly what I mean: localObject is copying one object at the
> time. So say I have in my contextA 1 Gallery with 300 related
> Paintings, all with PersistentState.COMMITED. I do
> contextB.localObject on Gallery, then I access every Painting via
> relaionship. this means that the localObject will be invoked 301
> times, everytime with single object. I believe this will be faster
> than re-fetching the data, but what if there will be 3000 Paintings ?
> Is there a limit after which the localObject becomes slower than re-
> fetch ?
>
> Marcin
>
>
> On 02/11/2007, at 11:23 PM, Mike Kienenberger wrote:
>
> > I am no expert in this subject, but the cost of doing I/O is always
> > going to exceed the cost of in-system memory copies, probably by
> > orders of magnitude. The only place where this might not be true is
> > using an in-memory database such as an HSQLDB mem jdbc connection.
> >
> > So unless you're copying orders-of-magnitude more data than you'd be
> > fetching, always copy.
> >
> > Remember too that a localObject only copies the object, not any of its
> > relationships. And the other objects are only pulled in if you
> > access them, which is even more fine-grained than prefetching.
> >
> > So my guess is that localObject is ALWAYS orders of magnitude faster.
> > I suspect it'd be extremely difficult to define a situation where this
> > was not the case.
> >
> > On 11/1/07, Marcin Skladaniec <ma...@ish.com.au> wrote:
> >> Hi
> >>
> >> We are improving the performance of our client application (ROP). Not
> >> surprisingly we have found that adding prefetches did improve the
> >> speed significantly.
> >>
> >> I have a question though about which is about efficiency of
> >> localObject. We are fetching a list of objects (query with
> >> prefetches)
> >> to a specific one non-editable context. If the object is to be edited
> >> it has to be copied to another context which allows committing
> >> changes. After the object is copied all the related object are
> >> accessed, so the performance of the copying came to my mind.
> >>
> >> The test I have looks like this:
> >>
> >> CayenneContext context1;
> >> CayenneContext context2;
> >>
> >> SelectQuery q = new SelectQuery(Painting.class);
> >> q.addPrefetch(Painting.GALLERY_PROPERTY);
> >> q.addPrefetch(Painting. GALLERY_PROPERTY + "." +
> >> Gallery.CITY_PROPERTY);
> >> List l = context1.performQuery(q);
> >> //now the context1 contain all the records I wanted it to contain
> >>
> >> Painting p1 = (Painting) l.get(0);
> >> Painting p2 = (Painting) context2.localObject(p1.getObjectId(),
> >> null);
> >>
> >> //now the context2 contain only the single record I copied
> >> p2.getGallery().getCity();
> >> //now context2 contains the same objects as context1
> >>
> >> All is very quick, but my question is how it will scale when the
> >> relationship would be to-many and there will be hundreds related
> >> records.
> >>
> >> What do you think would be a threshold number of related objects
> >> which
> >> have to be localised over which it would be worth doing a new,
> >> specific select query for that single object (with prefetches). Would
> >> there be advantage of doing that at all ?
> >>
> >> We are using cutting edge version of cayenne (I think we use build
> >> about 2-3 weeks old).
> >>
> >> Marcin
> >>
>
>
>
Re: localObject vs prefetches: would there be a benefit ?
Posted by Marcin Skladaniec <ma...@ish.com.au>.
Hi Mike
Thats exactly what I mean: localObject is copying one object at the
time. So say I have in my contextA 1 Gallery with 300 related
Paintings, all with PersistentState.COMMITED. I do
contextB.localObject on Gallery, then I access every Painting via
relaionship. this means that the localObject will be invoked 301
times, everytime with single object. I believe this will be faster
than re-fetching the data, but what if there will be 3000 Paintings ?
Is there a limit after which the localObject becomes slower than re-
fetch ?
Marcin
On 02/11/2007, at 11:23 PM, Mike Kienenberger wrote:
> I am no expert in this subject, but the cost of doing I/O is always
> going to exceed the cost of in-system memory copies, probably by
> orders of magnitude. The only place where this might not be true is
> using an in-memory database such as an HSQLDB mem jdbc connection.
>
> So unless you're copying orders-of-magnitude more data than you'd be
> fetching, always copy.
>
> Remember too that a localObject only copies the object, not any of its
> relationships. And the other objects are only pulled in if you
> access them, which is even more fine-grained than prefetching.
>
> So my guess is that localObject is ALWAYS orders of magnitude faster.
> I suspect it'd be extremely difficult to define a situation where this
> was not the case.
>
> On 11/1/07, Marcin Skladaniec <ma...@ish.com.au> wrote:
>> Hi
>>
>> We are improving the performance of our client application (ROP). Not
>> surprisingly we have found that adding prefetches did improve the
>> speed significantly.
>>
>> I have a question though about which is about efficiency of
>> localObject. We are fetching a list of objects (query with
>> prefetches)
>> to a specific one non-editable context. If the object is to be edited
>> it has to be copied to another context which allows committing
>> changes. After the object is copied all the related object are
>> accessed, so the performance of the copying came to my mind.
>>
>> The test I have looks like this:
>>
>> CayenneContext context1;
>> CayenneContext context2;
>>
>> SelectQuery q = new SelectQuery(Painting.class);
>> q.addPrefetch(Painting.GALLERY_PROPERTY);
>> q.addPrefetch(Painting. GALLERY_PROPERTY + "." +
>> Gallery.CITY_PROPERTY);
>> List l = context1.performQuery(q);
>> //now the context1 contain all the records I wanted it to contain
>>
>> Painting p1 = (Painting) l.get(0);
>> Painting p2 = (Painting) context2.localObject(p1.getObjectId(),
>> null);
>>
>> //now the context2 contain only the single record I copied
>> p2.getGallery().getCity();
>> //now context2 contains the same objects as context1
>>
>> All is very quick, but my question is how it will scale when the
>> relationship would be to-many and there will be hundreds related
>> records.
>>
>> What do you think would be a threshold number of related objects
>> which
>> have to be localised over which it would be worth doing a new,
>> specific select query for that single object (with prefetches). Would
>> there be advantage of doing that at all ?
>>
>> We are using cutting edge version of cayenne (I think we use build
>> about 2-3 weeks old).
>>
>> Marcin
>>
Re: localObject vs prefetches: would there be a benefit ?
Posted by Mike Kienenberger <mk...@gmail.com>.
I am no expert in this subject, but the cost of doing I/O is always
going to exceed the cost of in-system memory copies, probably by
orders of magnitude. The only place where this might not be true is
using an in-memory database such as an HSQLDB mem jdbc connection.
So unless you're copying orders-of-magnitude more data than you'd be
fetching, always copy.
Remember too that a localObject only copies the object, not any of its
relationships. And the other objects are only pulled in if you
access them, which is even more fine-grained than prefetching.
So my guess is that localObject is ALWAYS orders of magnitude faster.
I suspect it'd be extremely difficult to define a situation where this
was not the case.
On 11/1/07, Marcin Skladaniec <ma...@ish.com.au> wrote:
> Hi
>
> We are improving the performance of our client application (ROP). Not
> surprisingly we have found that adding prefetches did improve the
> speed significantly.
>
> I have a question though about which is about efficiency of
> localObject. We are fetching a list of objects (query with prefetches)
> to a specific one non-editable context. If the object is to be edited
> it has to be copied to another context which allows committing
> changes. After the object is copied all the related object are
> accessed, so the performance of the copying came to my mind.
>
> The test I have looks like this:
>
> CayenneContext context1;
> CayenneContext context2;
>
> SelectQuery q = new SelectQuery(Painting.class);
> q.addPrefetch(Painting.GALLERY_PROPERTY);
> q.addPrefetch(Painting. GALLERY_PROPERTY + "." + Gallery.CITY_PROPERTY);
> List l = context1.performQuery(q);
> //now the context1 contain all the records I wanted it to contain
>
> Painting p1 = (Painting) l.get(0);
> Painting p2 = (Painting) context2.localObject(p1.getObjectId(), null);
>
> //now the context2 contain only the single record I copied
> p2.getGallery().getCity();
> //now context2 contains the same objects as context1
>
> All is very quick, but my question is how it will scale when the
> relationship would be to-many and there will be hundreds related
> records.
>
> What do you think would be a threshold number of related objects which
> have to be localised over which it would be worth doing a new,
> specific select query for that single object (with prefetches). Would
> there be advantage of doing that at all ?
>
> We are using cutting edge version of cayenne (I think we use build
> about 2-3 weeks old).
>
> Marcin
>