Paging query relating to IDataProvider and Lucene search

[rgoodwin <rg...@yahoo.com>]

Hello,

I'm trying to solve a variation on a standard problem regarding running
searches for paged results and displaying paging links.

Unfortunately for me, I cannot use the standard PagingNavigator / DataView /
IDataProvider solution for this. Wicket's top-down approach to processing
has a consequence in that there must be separate queries run for size and
data.

But our current web app retrieves its data from Lucene search results.
A Lucene query returns both results and size simultaneously.
Only 1 query is performed.


The paging related and DataView are calling IDataProvider.size() early on,
before calling iterator() and passing the offset. Of course this fits
perfectly for most users and I see from the code why it has to be done like
this. But this also seems to conflict with a data service tier expecting to
perform a single query to return both results ans size in one, e.g. like
Lucene or when using a Transfer Object pattern.

Is there any easy way to subclass either PagingNavigator or associated
classes so that they don't get involved with IPageable until after all data
is loaded?


---

Not as weird a suggestion as you might think.
Having paging defer it's configuration until last is exactly how our current
servlet/JSP app works.
So no need to get hold of size before querying for paged results.

1. User clicks paging link which sends an offset value
2. Server performs Lucene search which returns size and hits
3. A paged subset of results starting at offset are made ready for
presentation
4. Server creates Paging class with the latest offset and size etc
5. Server hands over to JSP
6. JSP renders results and renders paging links


Wicket sequence seems approximately the opposite:

1. User clicks paging link which sends page number
2. PagingNavigationLink calls its IPageable.getPageCount()
3. IPageable then calls IDataProvider.size()
4. Query 1 is run to get size
3. DataView calls IDataProvider to get size()
4. DataView then calls iterator() to get results
5. Query 2 is run to get results
4. Then page rendering starts?

With only a basic understanding of the Wicket api, it seems to me data
loading and component configuration stages are interwoven?


I already wrote an alternative to IDataProvider and DataView that supports
single query for size and data,
and posted the code for this to Jira ticket WICKET-1784.

Would now like to develop an alternative paging component to solve the
problem completely, in that the paging behaves like our current solution,
e.g. it 'reacts' to the latest data after it's loaded, rather than try to
predict it.


Many thanks.
-- 
View this message in context: http://www.nabble.com/Paging-query-relating-to-IDataProvider-and-Lucene-search-tp19083164p19083164.html
Sent from the Wicket - User mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org

Re: Paging query relating to IDataProvider and Lucene search

[John Patterson <jd...@gmail.com>]

Why don't you just cache your hits in your dataprovider in the constructor
and access it for both size() and iterator(int, int)?

I use Lucene for a DataView this way without a problem.


rgoodwin wrote:
> 
> Hello,
> 
> I'm trying to solve a variation on a standard problem regarding running
> searches for paged results and displaying paging links.
> 
> Unfortunately for me, I cannot use the standard PagingNavigator / DataView
> / IDataProvider solution for this. Wicket's top-down approach to
> processing has a consequence in that there must be separate queries run
> for size and data.
> 
> But our current web app retrieves its data from Lucene search results.
> A Lucene query returns both results and size simultaneously.
> Only 1 query is performed.
> 
> 
> The paging related and DataView are calling IDataProvider.size() early on,
> before calling iterator() and passing the offset. Of course this fits
> perfectly for most users and I see from the code why it has to be done
> like this. But this also seems to conflict with a data service tier
> expecting to perform a single query to return both results ans size in
> one, e.g. like Lucene or when using a Transfer Object pattern.
> 
> Is there any easy way to subclass either PagingNavigator or associated
> classes so that they don't get involved with IPageable until after all
> data is loaded?
> 
> 
> ---
> 
> Not as weird a suggestion as you might think.
> Having paging defer it's configuration until last is exactly how our
> current servlet/JSP app works.
> So no need to get hold of size before querying for paged results.
> 
> 1. User clicks paging link which sends an offset value
> 2. Server performs Lucene search which returns size and hits
> 3. A paged subset of results starting at offset are made ready for
> presentation
> 4. Server creates Paging class with the latest offset and size etc
> 5. Server hands over to JSP
> 6. JSP renders results and renders paging links
> 
> 
> Wicket sequence seems approximately the opposite:
> 
> 1. User clicks paging link which sends page number
> 2. PagingNavigationLink calls its IPageable.getPageCount()
> 3. IPageable then calls IDataProvider.size()
> 4. Query 1 is run to get size
> 3. DataView calls IDataProvider to get size()
> 4. DataView then calls iterator() to get results
> 5. Query 2 is run to get results
> 4. Then page rendering starts?
> 
> With only a basic understanding of the Wicket api, it seems to me data
> loading and component configuration stages are interwoven?
> 
> 
> I already wrote an alternative to IDataProvider and DataView that supports
> single query for size and data,
> and posted the code for this to Jira ticket WICKET-1784.
> 
> Would now like to develop an alternative paging component to solve the
> problem completely, in that the paging behaves like our current solution,
> e.g. it 'reacts' to the latest data after it's loaded, rather than try to
> predict it.
> 
> 
> Many thanks.
> 

-- 
View this message in context: http://www.nabble.com/Paging-query-relating-to-IDataProvider-and-Lucene-search-tp19083164p19083944.html
Sent from the Wicket - User mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org

Re: Paging query relating to IDataProvider and Lucene search

[rgoodwin <rg...@yahoo.com>]

Hi John,

Thanks for replying. Would be a neat solution if only I could cache Hits.
:(

But in my desire to simplify the problem statement I omitted to mention that
our data services tier runs in another cluster and converts Lucene
hits/documents/fields to a collection of our in-house search results
objects. No Lucene objects are sent back to the UI server.

These search result objects are a bit like DTOs, but not in the pre-EJB3
sense. They're really just to standardise our search results architecture,
as certain search result subclasses actually do db hits. So we want
everything unified for the web app.

This is why I made reference in the Wicket Jira ticket WICKET-1784 to the
Transfer Object pattern, as solving this problem with respect to DTOs
(instead of just Lucene) would solve for all interaction with legacy systems
using DTOs, me thinks.

(DTOs get a bad name these days now that entities are detachable, but I
believe they still have their uses, e.g. for abstracting)

Thanks again, and very open to simple solutions.
Not keen on re-inventing the Wicket wheel.
-- 
View this message in context: http://www.nabble.com/Paging-query-relating-to-IDataProvider-and-Lucene-search-tp19083164p19094471.html
Sent from the Wicket - User mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org