You are viewing a plain text version of this content. The canonical link for it is here.
Posted to torque-dev@db.apache.org by Thomas Fischer <tf...@apache.org> on 2005/09/11 19:45:34 UTC
LargeSelect and databases which do not support native limit/offset
Hi,
As was explained in http://issues.apache.org/scarab/issues/id/TRQS318,
LargeSelect has a very poor performance for databases with no native
limit/offset. I lookead at the code myself and also am of the opinion that
using LargesSelect makes no sense if the db does not support native
limit/offset.
Therefore I am inclined to follow the proposed solution and throw an
exception in the constructor of LargeSelect if the chosen DB does not
support native limit and offset. The code would look loke:
private void init(Criteria criteria, int pageSize, int
memoryLimitPages)
throws IllegalArgumentException
{
...
{
String dbName = criteria.getDbName();
if (dbName == null)
{
dbName = Torque.getDefaultDB();
}
if (dbName == null)
{
throw new RuntimeException(
"Torque is not initialized yet");
}
DB db;
try
{
db = Torque.getDB(dbName);
}
catch (TorqueException e)
{
throw new RuntimeException(
"cannot retieve Database for DB name " + dbName);
}
if (db.getLimitStyle() == DB.LIMIT_STYLE_NONE)
{
throw new IllegalArgumentException(
"criteria must have a db which supports "
+ "native Limit and Offset");
}
}
...
Are there any objections ? Is everybody ok with throwing a
RuntimeException, or would it be better to add the TorqueException to the
throws clause, possibly breaking people's code because TorqueException is
not caught ?
I would also guess that the problems with databases which do not support
native limit/offset have lead to the exclusion of the LargeSelectTest from
the runtimetest. Does anybody object to include the LargeSelectTest
and print an error but not execute the test if the database dose not
support native limit/offset ? Sample code would be
public class LargeSelectTest extends BaseRuntimeTestCase
{
....
public void testLargeSelect() throws TorqueException
{
if (Torque.getDB(Torque.getDefaultDB()).getLimitStyle()
== DB.LIMIT_STYLE_NONE)
{
log.error("LargeSelect is known not to work for databases "
+ "which do not support native limit/offset");
return;
}
....
If one adds this, the LargeSelectTest also runs for hsqldb which does not
support native limit/offset.
Thomas
---------------------------------------------------------------------
To unsubscribe, e-mail: torque-dev-unsubscribe@db.apache.org
For additional commands, e-mail: torque-dev-help@db.apache.org
Re: LargeSelect and databases which do not support native limit/offset
Posted by Thomas Vandahl <th...@tewisoft.de>.
Scott Eade wrote:
> My only comment would be that as a heavy user of LargeSelect, if I were
> to ever switch to a database that did not support limit and offset I
> could not easily produce an executing (though obviously poorly
> performing when it comes to LargeSelect) version of my application.
>
> I would prefer us to add an appropriately worded statement in the docs
> and javadocs highlighting the issue with a recommendation that it not be
> used for the affected databases.
I agree with that. I would rather log a warning than throwing an
exception. I used part of the code of LargeSelect to provide a similar
interface to a server application of ours. This one does not have
limit/offset support at all, but using the LargeSelect approach is still
much faster than loading the whole result list. So basically I
recommed to leave the decision about performance to the developer.
BTW: Findbugs complains about poor synchronization in LargeSelect. I
could not really make sense of it. If someone could give me a hint, I
hereby volunteer to look into it.
Bye, Thomas.
---------------------------------------------------------------------
To unsubscribe, e-mail: torque-dev-unsubscribe@db.apache.org
For additional commands, e-mail: torque-dev-help@db.apache.org
Re: LargeSelect and databases which do not support native limit/offset
Posted by Thomas Fischer <tf...@apache.org>.
On Mon, 12 Sep 2005, Scott Eade wrote:
>> Therefore I am inclined to follow the proposed solution and throw an
>> exception in the constructor of LargeSelect if the chosen DB does not
>> support native limit and offset.
>> ...
>>
>> Are there any objections ? Is everybody ok with throwing a
>> RuntimeException, or would it be better to add the TorqueException to
>> the throws clause, possibly breaking people's code because
>> TorqueException is not caught ?
>
> My only comment would be that as a heavy user of LargeSelect, if I were
> to ever switch to a database that did not support limit and offset I
> could not easily produce an executing (though obviously poorly
> performing when it comes to LargeSelect) version of my application.
>
> I would prefer us to add an appropriately worded statement in the docs
> and javadocs highlighting the issue with a recommendation that it not be
> used for the affected databases.
>
The comment already exists, though maybe one could highlight it more. The
problem is that the performance is much poorer
than it need be, because fetching one page at a time is exactly the thing
you should not do if you do not have native limit/offset.
Thinking over it again, the real solution would be to fetch the whole
memoryLinit at once and do the offset by hand if the database does not
support native limit/offset. The bad thing is that I do not know the
LargeSelect code and have not the time to do it at the moment (sigh).
>>
>> I would also guess that the problems with databases which do not
>> support native limit/offset have lead to the exclusion of the
>> LargeSelectTest from the runtimetest. Does anybody object to include
>> the LargeSelectTest and print an error but not execute the test if the
>> database dose not support native limit/offset ? Sample code would be
>>
>> public class LargeSelectTest extends BaseRuntimeTestCase
>> {
>> ....
>>
>> public void testLargeSelect() throws TorqueException
>> {
>> if (Torque.getDB(Torque.getDefaultDB()).getLimitStyle()
>> == DB.LIMIT_STYLE_NONE)
>> {
>> log.error("LargeSelect is known not to work for databases "
>> + "which do not support native limit/offset");
>> return;
>> }
>>
>> ....
>>
>> If one adds this, the LargeSelectTest also runs for hsqldb which does
>> not support native limit/offset.
>
> Does the test case fail at present?
At the moment, the method LargeSelectTest.testLargeSelect() fails.
> If we continue to let the code code
> execute for the reason given above then the test case should at least be
> enabled to make sure that the simulated limit and offset are indeed
> compatible with the behaviour when the database supports them.
>
If we manage to do the limit/offset manually, one could take it out again.
Thomas
---------------------------------------------------------------------
To unsubscribe, e-mail: torque-dev-unsubscribe@db.apache.org
For additional commands, e-mail: torque-dev-help@db.apache.org
Re: LargeSelect and databases which do not support native limit/offset
Posted by Scott Eade <se...@backstagetech.com.au>.
Thomas Fischer wrote:
> Hi,
>
> As was explained in http://issues.apache.org/scarab/issues/id/TRQS318,
> LargeSelect has a very poor performance for databases with no native
> limit/offset. I lookead at the code myself and also am of the opinion
> that using LargesSelect makes no sense if the db does not support
> native limit/offset.
>
> Therefore I am inclined to follow the proposed solution and throw an
> exception in the constructor of LargeSelect if the chosen DB does not
> support native limit and offset. The code would look loke:
>
> private void init(Criteria criteria, int pageSize, int
> memoryLimitPages)
> throws IllegalArgumentException
> {
> ...
>
> {
> String dbName = criteria.getDbName();
> if (dbName == null)
> {
> dbName = Torque.getDefaultDB();
> }
> if (dbName == null)
> {
> throw new RuntimeException(
> "Torque is not initialized yet");
> }
> DB db;
> try
> {
> db = Torque.getDB(dbName);
> }
> catch (TorqueException e)
> {
> throw new RuntimeException(
> "cannot retieve Database for DB name " + dbName);
> }
> if (db.getLimitStyle() == DB.LIMIT_STYLE_NONE)
> {
> throw new IllegalArgumentException(
> "criteria must have a db which supports "
> + "native Limit and Offset");
> }
> }
>
> ...
>
> Are there any objections ? Is everybody ok with throwing a
> RuntimeException, or would it be better to add the TorqueException to
> the throws clause, possibly breaking people's code because
> TorqueException is not caught ?
My only comment would be that as a heavy user of LargeSelect, if I were
to ever switch to a database that did not support limit and offset I
could not easily produce an executing (though obviously poorly
performing when it comes to LargeSelect) version of my application.
I would prefer us to add an appropriately worded statement in the docs
and javadocs highlighting the issue with a recommendation that it not be
used for the affected databases.
>
> I would also guess that the problems with databases which do not
> support native limit/offset have lead to the exclusion of the
> LargeSelectTest from the runtimetest. Does anybody object to include
> the LargeSelectTest and print an error but not execute the test if the
> database dose not support native limit/offset ? Sample code would be
>
> public class LargeSelectTest extends BaseRuntimeTestCase
> {
> ....
>
> public void testLargeSelect() throws TorqueException
> {
> if (Torque.getDB(Torque.getDefaultDB()).getLimitStyle()
> == DB.LIMIT_STYLE_NONE)
> {
> log.error("LargeSelect is known not to work for databases "
> + "which do not support native limit/offset");
> return;
> }
>
> ....
>
> If one adds this, the LargeSelectTest also runs for hsqldb which does
> not support native limit/offset.
Does the test case fail at present? If we continue to let the code code
execute for the reason given above then the test case should at least be
enabled to make sure that the simulated limit and offset are indeed
compatible with the behaviour when the database supports them.
Scott
--
Scott Eade
Backstage Technologies Pty. Ltd.
http://www.backstagetech.com.au
---------------------------------------------------------------------
To unsubscribe, e-mail: torque-dev-unsubscribe@db.apache.org
For additional commands, e-mail: torque-dev-help@db.apache.org