You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cayenne.apache.org by Tomi NA <he...@gmail.com> on 2006/05/11 17:14:27 UTC
a...bug with pgsql and addPrefetch? (Cay 1.2B2)
I have a table with 50-100 thousand records which I want to search on
a find-as-you-type basis.
This is what was executed when a key was pressed:
results = new ArrayList();
List<MyRec> myRecs = null;
String[] searchKeys = dissassemble(pattern);
SelectQuery query = prepQuery(searchKeys);
query.addPrefetch(MyRec.TO_OTHER_REC_PROPERTY);
query.setFetchLimit(100);
myRecs = myContext.performQuery(query);
for (MyRec mr : myRecs) {
results.add(mr.getToOtherRec());
}
Now, what happens is that 2 queries are run. The first is limited to
100, just the way I wanted it. The second, however, is not: it will
return 50k records no questions asked.
I can work arround the problem just by not doing the prefetch: then
everything works i.e. my app doesn't break down due to an
OutOfMemoryException.
If I missunderstood the intended behaviour of addPrefetch, someone
please correct me. Otherwise, I'll proceed to submiting a bug
report...
Tomislav
Re: a...bug with pgsql and addPrefetch? (Cay 1.2B2)
Posted by Cris Daniluk <cr...@gmail.com>.
> I'm sorry I didn't make it completely clear: I'm using cayenne 1.2B2
> (see subject line).
GMail cut off the subject... sorry bout that :)
> Also, I'm very thankfull for the hint about prefetch semantics: I
> didn't really think about what was going on under the hood, up untill
> now.
> I gave PrefetchTreeNode.JOINT_PREFETCH_SEMANTICS a try and got
> *exactly* the join I wanted:
>
> QueryLogger:
> SELECT (...t0.fieldlist, t1.fieldlist...)
> FROM main.first_table t0, main.second_table t1
> WHERE t0.id = t1.id AND (t0.attr ILIKE ?) LIMIT 50 [bind: '%a%'] -
> prepared in 18 ms.
>
> Now, the problem is that not knowing about prefetch semantics, I
> expected the prefetch to *just work* i.e. deduce which records in the
> other table I really require, instead of fetching all of them.
> I still think there's no use case to support the default prefetch
> behaviour that gave me problems in the first place: if the query on
> the primary table is limited to no more than a 100 records, why would
> the prefetch go for all the records of the secondary table?
>
The reason is that if you were including certain expressions, you
might find that the prefetch didn't include all results, meaning a
getRelationshipArray call on your base object could deceivingly return
less than the number of results that are actually present. This has
been documented in a few bugs and on the list, and is definitely a
priority, but quite difficult to actually fix :)
http://issues.apache.org/cayenne/browse/CAY-319
Maybe you could submit a bug report about this, though... the
semantics should be detected correctly most of the times, and this
might be a time where it could have been detected better. Would have
to look in greater detail to be sure.
Cris
Re: a...bug with pgsql and addPrefetch? (Cay 1.2B2)
Posted by Tomi NA <he...@gmail.com>.
On 5/12/06, Cris Daniluk <cr...@gmail.com> wrote:
> I think this is more a limitation of the current prefetch behavior,
> than an actual bug. You may want to experiment with the prefetch
> "advice", or semantics:
>
> http://objectstyle.org/confluence/display/CAYDOC/Prefetching
>
> I'm assuming you're using 1.2 latest release, of course. If you're
> using something that's not 1.1 latest or 1.2 latest, then it will be
> hard to talk about it. Anyway, if the docs don't provide any helpful
> info, please post the debugging output - specifically, the queries
> that are executed in this case.
I'm sorry I didn't make it completely clear: I'm using cayenne 1.2B2
(see subject line).
Also, I'm very thankfull for the hint about prefetch semantics: I
didn't really think about what was going on under the hood, up untill
now.
I gave PrefetchTreeNode.JOINT_PREFETCH_SEMANTICS a try and got
*exactly* the join I wanted:
QueryLogger:
SELECT (...t0.fieldlist, t1.fieldlist...)
FROM main.first_table t0, main.second_table t1
WHERE t0.id = t1.id AND (t0.attr ILIKE ?) LIMIT 50 [bind: '%a%'] -
prepared in 18 ms.
Now, the problem is that not knowing about prefetch semantics, I
expected the prefetch to *just work* i.e. deduce which records in the
other table I really require, instead of fetching all of them.
I still think there's no use case to support the default prefetch
behaviour that gave me problems in the first place: if the query on
the primary table is limited to no more than a 100 records, why would
the prefetch go for all the records of the secondary table?
t.n.a.
Re: a...bug with pgsql and addPrefetch? (Cay 1.2B2)
Posted by Cris Daniluk <cr...@gmail.com>.
I think this is more a limitation of the current prefetch behavior,
than an actual bug. You may want to experiment with the prefetch
"advice", or semantics:
http://objectstyle.org/confluence/display/CAYDOC/Prefetching
I'm assuming you're using 1.2 latest release, of course. If you're
using something that's not 1.1 latest or 1.2 latest, then it will be
hard to talk about it. Anyway, if the docs don't provide any helpful
info, please post the debugging output - specifically, the queries
that are executed in this case.
Cris
On 5/11/06, Tomi NA <he...@gmail.com> wrote:
> I have a table with 50-100 thousand records which I want to search on
> a find-as-you-type basis.
> This is what was executed when a key was pressed:
>
> results = new ArrayList();
> List<MyRec> myRecs = null;
>
> String[] searchKeys = dissassemble(pattern);
> SelectQuery query = prepQuery(searchKeys);
> query.addPrefetch(MyRec.TO_OTHER_REC_PROPERTY);
> query.setFetchLimit(100);
> myRecs = myContext.performQuery(query);
> for (MyRec mr : myRecs) {
> results.add(mr.getToOtherRec());
> }
>
> Now, what happens is that 2 queries are run. The first is limited to
> 100, just the way I wanted it. The second, however, is not: it will
> return 50k records no questions asked.
>
> I can work arround the problem just by not doing the prefetch: then
> everything works i.e. my app doesn't break down due to an
> OutOfMemoryException.
> If I missunderstood the intended behaviour of addPrefetch, someone
> please correct me. Otherwise, I'll proceed to submiting a bug
> report...
>
> Tomislav
>