IDataProvider advice needed – caching?

[Troy Cauble <tr...@gmail.com>]

I need an IDataProvider for web services calls to a Flickr search.

To complicate matters, I need to cull the Flickr results based on
local criteria.  So the Nth item to the user is the N + Mth item from Flickr.
And I can't know M without looking at the all the Flickr results up to
N + M.

So to handle a user paging back and forth, I either need to
a)  cache the Flickr data (tuples, not images)
b)  cache a mapping from the user index to the Flickr index
c)  start from 0 with flickr and count up each time the Provider is used.

Even with a cache (a & b), I'll have to fetch the data in between if
a user jumps from page 1 to page 5.

So my questions are

1)  How much data is it reasonable to let an IDataProvider serialize?
     2000 short strings?  4000?  Or how large of a TreeMap<int, int>?
2)  If I wanted to cache outside the IDataProvider serialization (global
     mem or another process), there's no way to know when a user is
     done with the cache, right?  (When they close that window or tab?)
3)  Have I missed any approaches?  Something besides IDataProvider?
     Anything?

Thanks,
-troy

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org

Re: IDataProvider advice needed – caching?

[Jeremy Thomerson <je...@wickettraining.com>]

On Thu, Sep 30, 2010 at 11:20 AM, Troy Cauble <tr...@gmail.com> wrote:

> I need an IDataProvider for web services calls to a Flickr search.
>
> To complicate matters, I need to cull the Flickr results based on
> local criteria.  So the Nth item to the user is the N + Mth item from
> Flickr.
> And I can't know M without looking at the all the Flickr results up to
> N + M.
>
> So to handle a user paging back and forth, I either need to
> a)  cache the Flickr data (tuples, not images)
> b)  cache a mapping from the user index to the Flickr index
> c)  start from 0 with flickr and count up each time the Provider is used.
>
> Even with a cache (a & b), I'll have to fetch the data in between if
> a user jumps from page 1 to page 5.
>
> So my questions are
>
> 1)  How much data is it reasonable to let an IDataProvider serialize?
>     2000 short strings?  4000?  Or how large of a TreeMap<int, int>?
> 2)  If I wanted to cache outside the IDataProvider serialization (global
>     mem or another process), there's no way to know when a user is
>     done with the cache, right?  (When they close that window or tab?)
> 3)  Have I missed any approaches?  Something besides IDataProvider?
>     Anything?
>
> Thanks,
> -troy
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
I'd think that this kind of logic and caching should be put into your
service layer.  Depending on your application, it is very likely that the
cached data could be used for multiple users.  Besides, your UI should just
be able to call into the service layer and not worry about these details.

At the service layer, you can use something like memcached and set the
timeout appropriately for your session length / usage combo.


-- 
Jeremy Thomerson
http://www.wickettraining.com