You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@wicket.apache.org by Dan Retzlaff <dr...@gmail.com> on 2012/04/12 03:59:14 UTC
IDataProvider#size()
Hi all. Time to start a thread of my own. :)
Many of Wicket's powerful repeaters depend on IDataProvider. This interface
has a size() method which returns a non-null integer. This makes it easy to
determine the total number of pages in a pageable view, but IMO the
required computation and application complexity are not always called for.
In many cases, a pageable but open-ended data view is adequate. Have you
experienced this impedance mismatch yourselves? What was your solution?
To elaborate on my experience:
For SQL-based views, the application complexity comes from the need to
construct a count(*) query with exactly the same criteria as the subsequent
result query. In my experience, this pollutes DAO interfaces and
IDataProvider implementation non-trivially. We initially had separate
methods for counting and querying (same args), but eventually moved to a
single method that returns a <List,Integer>-tuple with both the results and
total size which our IDataProvider caches. This lets us do some Hibernate
trickery to introduce a MySQL SQL_CALC_FOUND_ROWS query hint, avoiding
separate count/results queries in most cases. It's still not simple, and
for large counts is still expensive.
The situation is worse for non-SQL data stores which don't have a
fully-functional count(*) capability. We use Cassandra whose native "where
clause" support is limited, requiring significant client-side filtering.
Paging through an entire column (or CF) in this way is prohibitively
expensive, especially considering our users rarely even go to page 2. To
solve this, we've created a parallel set of view/paging classes that define
windows using previously discovered result keys instead of start indices
(tokens and column names in Cassandra). But having a full suite of
IUnsizedDataProvider-based classes smells. I love that Wicket devs have
solved some tough/tedious problems with DataViewBase and friends, and I
want to make use of them!
Comments or suggestions?
Cheers,
Dan
Re: IDataProvider#size()
Posted by Martin Grigorov <mg...@apache.org>.
On Thu, Apr 12, 2012 at 9:12 AM, Martin Grigorov <mg...@apache.org> wrote:
> On Thu, Apr 12, 2012 at 4:59 AM, Dan Retzlaff <dr...@gmail.com> wrote:
>> Hi all. Time to start a thread of my own. :)
>>
>> Many of Wicket's powerful repeaters depend on IDataProvider. This interface
>> has a size() method which returns a non-null integer. This makes it easy to
>> determine the total number of pages in a pageable view, but IMO the
>> required computation and application complexity are not always called for.
>> In many cases, a pageable but open-ended data view is adequate. Have you
>> experienced this impedance mismatch yourselves? What was your solution?
>
> Wicketstuff InMethod Grid supports this use case.
> One of its examples shows it.
> I'll deploy the examples at wicket-library.com soon.
http://www.wicket-library.com/inmethod-grid/data-grid/unknown-count
>
>>
>> To elaborate on my experience:
>>
>> For SQL-based views, the application complexity comes from the need to
>> construct a count(*) query with exactly the same criteria as the subsequent
>> result query. In my experience, this pollutes DAO interfaces and
>> IDataProvider implementation non-trivially. We initially had separate
>> methods for counting and querying (same args), but eventually moved to a
>> single method that returns a <List,Integer>-tuple with both the results and
>> total size which our IDataProvider caches. This lets us do some Hibernate
>> trickery to introduce a MySQL SQL_CALC_FOUND_ROWS query hint, avoiding
>> separate count/results queries in most cases. It's still not simple, and
>> for large counts is still expensive.
>>
>> The situation is worse for non-SQL data stores which don't have a
>> fully-functional count(*) capability. We use Cassandra whose native "where
>> clause" support is limited, requiring significant client-side filtering.
>> Paging through an entire column (or CF) in this way is prohibitively
>> expensive, especially considering our users rarely even go to page 2. To
>> solve this, we've created a parallel set of view/paging classes that define
>> windows using previously discovered result keys instead of start indices
>> (tokens and column names in Cassandra). But having a full suite of
>> IUnsizedDataProvider-based classes smells. I love that Wicket devs have
>> solved some tough/tedious problems with DataViewBase and friends, and I
>> want to make use of them!
>>
>> Comments or suggestions?
>>
>> Cheers,
>> Dan
>
>
>
> --
> Martin Grigorov
> jWeekend
> Training, Consulting, Development
> http://jWeekend.com
--
Martin Grigorov
jWeekend
Training, Consulting, Development
http://jWeekend.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: IDataProvider#size()
Posted by Martin Grigorov <mg...@apache.org>.
On Thu, Apr 12, 2012 at 4:59 AM, Dan Retzlaff <dr...@gmail.com> wrote:
> Hi all. Time to start a thread of my own. :)
>
> Many of Wicket's powerful repeaters depend on IDataProvider. This interface
> has a size() method which returns a non-null integer. This makes it easy to
> determine the total number of pages in a pageable view, but IMO the
> required computation and application complexity are not always called for.
> In many cases, a pageable but open-ended data view is adequate. Have you
> experienced this impedance mismatch yourselves? What was your solution?
Wicketstuff InMethod Grid supports this use case.
One of its examples shows it.
I'll deploy the examples at wicket-library.com soon.
>
> To elaborate on my experience:
>
> For SQL-based views, the application complexity comes from the need to
> construct a count(*) query with exactly the same criteria as the subsequent
> result query. In my experience, this pollutes DAO interfaces and
> IDataProvider implementation non-trivially. We initially had separate
> methods for counting and querying (same args), but eventually moved to a
> single method that returns a <List,Integer>-tuple with both the results and
> total size which our IDataProvider caches. This lets us do some Hibernate
> trickery to introduce a MySQL SQL_CALC_FOUND_ROWS query hint, avoiding
> separate count/results queries in most cases. It's still not simple, and
> for large counts is still expensive.
>
> The situation is worse for non-SQL data stores which don't have a
> fully-functional count(*) capability. We use Cassandra whose native "where
> clause" support is limited, requiring significant client-side filtering.
> Paging through an entire column (or CF) in this way is prohibitively
> expensive, especially considering our users rarely even go to page 2. To
> solve this, we've created a parallel set of view/paging classes that define
> windows using previously discovered result keys instead of start indices
> (tokens and column names in Cassandra). But having a full suite of
> IUnsizedDataProvider-based classes smells. I love that Wicket devs have
> solved some tough/tedious problems with DataViewBase and friends, and I
> want to make use of them!
>
> Comments or suggestions?
>
> Cheers,
> Dan
--
Martin Grigorov
jWeekend
Training, Consulting, Development
http://jWeekend.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: IDataProvider#size()
Posted by Eric Jablow <er...@gmail.com>.
On Wed, Apr 11, 2012 at 9:59 PM, Dan Retzlaff <dr...@gmail.com> wrote:
> For SQL-based views, the application complexity comes from the need to
> construct a count(*) query with exactly the same criteria as the subsequent
> result query. In my experience, this pollutes DAO interfaces and
> IDataProvider implementation non-trivially. We initially had separate
> methods for counting and querying (same args), but eventually moved to a
> single method that returns a <List,Integer>-tuple with both the results and
> total size which our IDataProvider caches. This lets us do some Hibernate
> trickery to introduce a MySQL SQL_CALC_FOUND_ROWS query hint, avoiding
> separate count/results queries in most cases. It's still not simple, and
> for large counts is still expensive.
>
> I faced something similar on an earlier project, just using straight JDBC
(in Spring),
and I ended up taking advice from a book; I made my query
SELECT ..., COUNT(*) AS ct
FROM ...
WHERE ...
and I added a ct variable to the class I was mapping to. A lot of extra
data was
returned, but it didn't slow things too much. I dout this would work well
with a
NoSQL database, and it is clumsy.
Eric
Re: IDataProvider#size()
Posted by Igor Vaynberg <ig...@gmail.com>.
but if you do not know how many results you have then that can happen
using any approach...
-igor
On Thu, Apr 12, 2012 at 2:36 AM, Michal Wegrzyn
<mi...@onior.com> wrote:
> That's the solution which I have used for the case, where count is not possible.
> It has only one slight disadvantage - if count == n*pageSize then the last page will be blank.
>
> Best regards,
> Michal Wegrzyn
>
>> -----Original Message-----
>> From: Igor Vaynberg [mailto:igor.vaynberg@gmail.com]
>> Sent: Thursday, April 12, 2012 6:06
>> To: users@wicket.apache.org
>> Subject: Re: IDataProvider#size()
>>
>> why not just fake the size to current page+1? that way you always have
>> a "next" link and once you receive the current page you should know if
>> you have more or not so you dont have to add the one on the last
>> page....
>>
>> -igor
>>
>> On Wed, Apr 11, 2012 at 6:59 PM, Dan Retzlaff <dr...@gmail.com>
>> wrote:
>> > Hi all. Time to start a thread of my own. :)
>> >
>> > Many of Wicket's powerful repeaters depend on IDataProvider. This
>> > interface has a size() method which returns a non-null integer. This
>> > makes it easy to determine the total number of pages in a pageable
>> > view, but IMO the required computation and application complexity are
>> not always called for.
>> > In many cases, a pageable but open-ended data view is adequate. Have
>> > you experienced this impedance mismatch yourselves? What was your
>> solution?
>> >
>> > To elaborate on my experience:
>> >
>> > For SQL-based views, the application complexity comes from the need
>> to
>> > construct a count(*) query with exactly the same criteria as the
>> > subsequent result query. In my experience, this pollutes DAO
>> > interfaces and IDataProvider implementation non-trivially. We
>> > initially had separate methods for counting and querying (same args),
>> > but eventually moved to a single method that returns a
>> > <List,Integer>-tuple with both the results and total size which our
>> > IDataProvider caches. This lets us do some Hibernate trickery to
>> > introduce a MySQL SQL_CALC_FOUND_ROWS query hint, avoiding separate
>> > count/results queries in most cases. It's still not simple, and for
>> large counts is still expensive.
>> >
>> > The situation is worse for non-SQL data stores which don't have a
>> > fully-functional count(*) capability. We use Cassandra whose native
>> > "where clause" support is limited, requiring significant client-side
>> filtering.
>> > Paging through an entire column (or CF) in this way is prohibitively
>> > expensive, especially considering our users rarely even go to page 2.
>> > To solve this, we've created a parallel set of view/paging classes
>> > that define windows using previously discovered result keys instead
>> of
>> > start indices (tokens and column names in Cassandra). But having a
>> > full suite of IUnsizedDataProvider-based classes smells. I love that
>> > Wicket devs have solved some tough/tedious problems with DataViewBase
>> > and friends, and I want to make use of them!
>> >
>> > Comments or suggestions?
>> >
>> > Cheers,
>> > Dan
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>> For additional commands, e-mail: users-help@wicket.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
RE: IDataProvider#size()
Posted by Michal Wegrzyn <mi...@onior.com>.
That's the solution which I have used for the case, where count is not possible.
It has only one slight disadvantage - if count == n*pageSize then the last page will be blank.
Best regards,
Michal Wegrzyn
> -----Original Message-----
> From: Igor Vaynberg [mailto:igor.vaynberg@gmail.com]
> Sent: Thursday, April 12, 2012 6:06
> To: users@wicket.apache.org
> Subject: Re: IDataProvider#size()
>
> why not just fake the size to current page+1? that way you always have
> a "next" link and once you receive the current page you should know if
> you have more or not so you dont have to add the one on the last
> page....
>
> -igor
>
> On Wed, Apr 11, 2012 at 6:59 PM, Dan Retzlaff <dr...@gmail.com>
> wrote:
> > Hi all. Time to start a thread of my own. :)
> >
> > Many of Wicket's powerful repeaters depend on IDataProvider. This
> > interface has a size() method which returns a non-null integer. This
> > makes it easy to determine the total number of pages in a pageable
> > view, but IMO the required computation and application complexity are
> not always called for.
> > In many cases, a pageable but open-ended data view is adequate. Have
> > you experienced this impedance mismatch yourselves? What was your
> solution?
> >
> > To elaborate on my experience:
> >
> > For SQL-based views, the application complexity comes from the need
> to
> > construct a count(*) query with exactly the same criteria as the
> > subsequent result query. In my experience, this pollutes DAO
> > interfaces and IDataProvider implementation non-trivially. We
> > initially had separate methods for counting and querying (same args),
> > but eventually moved to a single method that returns a
> > <List,Integer>-tuple with both the results and total size which our
> > IDataProvider caches. This lets us do some Hibernate trickery to
> > introduce a MySQL SQL_CALC_FOUND_ROWS query hint, avoiding separate
> > count/results queries in most cases. It's still not simple, and for
> large counts is still expensive.
> >
> > The situation is worse for non-SQL data stores which don't have a
> > fully-functional count(*) capability. We use Cassandra whose native
> > "where clause" support is limited, requiring significant client-side
> filtering.
> > Paging through an entire column (or CF) in this way is prohibitively
> > expensive, especially considering our users rarely even go to page 2.
> > To solve this, we've created a parallel set of view/paging classes
> > that define windows using previously discovered result keys instead
> of
> > start indices (tokens and column names in Cassandra). But having a
> > full suite of IUnsizedDataProvider-based classes smells. I love that
> > Wicket devs have solved some tough/tedious problems with DataViewBase
> > and friends, and I want to make use of them!
> >
> > Comments or suggestions?
> >
> > Cheers,
> > Dan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: IDataProvider#size()
Posted by Igor Vaynberg <ig...@gmail.com>.
why not just fake the size to current page+1? that way you always have
a "next" link and once you receive the current page you should know if
you have more or not so you dont have to add the one on the last
page....
-igor
On Wed, Apr 11, 2012 at 6:59 PM, Dan Retzlaff <dr...@gmail.com> wrote:
> Hi all. Time to start a thread of my own. :)
>
> Many of Wicket's powerful repeaters depend on IDataProvider. This interface
> has a size() method which returns a non-null integer. This makes it easy to
> determine the total number of pages in a pageable view, but IMO the
> required computation and application complexity are not always called for.
> In many cases, a pageable but open-ended data view is adequate. Have you
> experienced this impedance mismatch yourselves? What was your solution?
>
> To elaborate on my experience:
>
> For SQL-based views, the application complexity comes from the need to
> construct a count(*) query with exactly the same criteria as the subsequent
> result query. In my experience, this pollutes DAO interfaces and
> IDataProvider implementation non-trivially. We initially had separate
> methods for counting and querying (same args), but eventually moved to a
> single method that returns a <List,Integer>-tuple with both the results and
> total size which our IDataProvider caches. This lets us do some Hibernate
> trickery to introduce a MySQL SQL_CALC_FOUND_ROWS query hint, avoiding
> separate count/results queries in most cases. It's still not simple, and
> for large counts is still expensive.
>
> The situation is worse for non-SQL data stores which don't have a
> fully-functional count(*) capability. We use Cassandra whose native "where
> clause" support is limited, requiring significant client-side filtering.
> Paging through an entire column (or CF) in this way is prohibitively
> expensive, especially considering our users rarely even go to page 2. To
> solve this, we've created a parallel set of view/paging classes that define
> windows using previously discovered result keys instead of start indices
> (tokens and column names in Cassandra). But having a full suite of
> IUnsizedDataProvider-based classes smells. I love that Wicket devs have
> solved some tough/tedious problems with DataViewBase and friends, and I
> want to make use of them!
>
> Comments or suggestions?
>
> Cheers,
> Dan
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org