You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@wicket.apache.org by Jan Juno <ja...@gmail.com> on 2011/03/09 13:00:40 UTC
Storing ArrayList with over 10000 objects
I have One Array-list(with over 10000 objects in it) what is the best
practice for caching it so I don't have to load it over and over again in
each request?
Jan
Re: Storing ArrayList with over 10000 objects
Posted by Jan Juno <ja...@gmail.com>.
can you point me to a good wicket + ehcache tutorial?
On 9 March 2011 13:13, Martijn Dashorst <ma...@gmail.com> wrote:
> On Wed, Mar 9, 2011 at 1:00 PM, Jan Juno <ja...@gmail.com> wrote:
> > I have One Array-list(with over 10000 objects in it) what is the best
> > practice for caching it so I don't have to load it over and over again in
> > each request?
>
> Usually at our company we expect our ORM mapper (hibernate) to take
> care of caching, so it is not a problem for us to ask the ORM for the
> same query again and again and again.
>
> I'd use a cache for that. Either something home grown (usually not a
> great idea, but for a single use case it might work), or something
> like ehcache. If the item can't be found in the cache, retrieve it
> from data store and put it in cache.
>
> I wouldn't keep the 10k objects in a page instance or the user session
> since that will be serialized with each request.
>
> Martijn
>
> --
> Become a Wicket expert, learn from the best: http://wicketinaction.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
Re: Storing ArrayList with over 10000 objects
Posted by Martijn Dashorst <ma...@gmail.com>.
Should work, though perhaps going to the datastore for each
back/forward button press might be overkill. Retrieving it from a
cache, like ehcache, allows you to set an expiry on the data, keep it
thread safe and even memory safe (i.e. let ehcache throw away the
objects when memory is tight, or let it write to disk if not used for
a while).
AFAIK there are no examples on the web utilizing ehcache and wicket
specifically. Just use a plain ehcache example, and access the cache
store in a loadable detacheble model, or possibly even in your service
method.
Martijn
On Wed, Mar 9, 2011 at 1:48 PM, robert.mcguinness
<ro...@gmail.com> wrote:
> Martijn,
>
> Wouldn't this be a safe way to cache a list at the expense of memory (using
> transient)? In a lot of my projects I have to do some aggregation of the
> results from multiple data stores that are slow even with the caching layer.
>
>
> private class SomeDataProvider extends SortableDataProvider {
>
> /* cache the page results but don't serialize them to session or disk */
> private transient SearchResults searchResults;
>
> private static final long serialVersionUID = 1L;
> private SortParam sortParam;
>
> public SomeDataProvider () {}
>
> @Override
> public int size() {
>
> /*
> * if null (back button/forward button case) rebuild the results using
> the search parmas that are stored with page
> */
> if (searchResults == null) {
>
> searchResults = searchService.findSomeStuff(serializableSearchParams);
> }
>
> return searchResults.size();;
> }
>
> @Override
> public IModel model(Object object) {
> return new SearchHitModel(object).setShouldDetach(false);
> }
>
> @Override
> public Iterator iterator(int first, int count) {
>
> SortParam sp = getSort();
>
> // only sort results when the sort command changes
> if (!sortParam.equals(sp)) {
> Sorter sorter = new Sorter(sp.getProperty(), sp.isAscending());
> Collections.sort(searchResults.getSearchResultList(), sorter);
> sortParam = sp;
> }
>
> return searchResults.getPagedResults(first, count).iterator();
> }
>
> }
>
>
> --
> View this message in context: http://apache-wicket.1842946.n4.nabble.com/Storing-ArrayList-with-over-10000-objects-tp3343442p3343499.html
> Sent from the Users forum mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
--
Become a Wicket expert, learn from the best: http://wicketinaction.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: Storing ArrayList with over 10000 objects
Posted by "robert.mcguinness" <ro...@gmail.com>.
Martijn,
Wouldn't this be a safe way to cache a list at the expense of memory (using
transient)? In a lot of my projects I have to do some aggregation of the
results from multiple data stores that are slow even with the caching layer.
private class SomeDataProvider extends SortableDataProvider {
/* cache the page results but don't serialize them to session or disk */
private transient SearchResults searchResults;
private static final long serialVersionUID = 1L;
private SortParam sortParam;
public SomeDataProvider () {}
@Override
public int size() {
/*
* if null (back button/forward button case) rebuild the results using
the search parmas that are stored with page
*/
if (searchResults == null) {
searchResults = searchService.findSomeStuff(serializableSearchParams);
}
return searchResults.size();;
}
@Override
public IModel model(Object object) {
return new SearchHitModel(object).setShouldDetach(false);
}
@Override
public Iterator iterator(int first, int count) {
SortParam sp = getSort();
// only sort results when the sort command changes
if (!sortParam.equals(sp)) {
Sorter sorter = new Sorter(sp.getProperty(), sp.isAscending());
Collections.sort(searchResults.getSearchResultList(), sorter);
sortParam = sp;
}
return searchResults.getPagedResults(first, count).iterator();
}
}
--
View this message in context: http://apache-wicket.1842946.n4.nabble.com/Storing-ArrayList-with-over-10000-objects-tp3343442p3343499.html
Sent from the Users forum mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: Storing ArrayList with over 10000 objects
Posted by Martijn Dashorst <ma...@gmail.com>.
On Wed, Mar 9, 2011 at 1:00 PM, Jan Juno <ja...@gmail.com> wrote:
> I have One Array-list(with over 10000 objects in it) what is the best
> practice for caching it so I don't have to load it over and over again in
> each request?
Usually at our company we expect our ORM mapper (hibernate) to take
care of caching, so it is not a problem for us to ask the ORM for the
same query again and again and again.
I'd use a cache for that. Either something home grown (usually not a
great idea, but for a single use case it might work), or something
like ehcache. If the item can't be found in the cache, retrieve it
from data store and put it in cache.
I wouldn't keep the 10k objects in a page instance or the user session
since that will be serialized with each request.
Martijn
--
Become a Wicket expert, learn from the best: http://wicketinaction.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: Storing ArrayList with over 10000 objects
Posted by Michael O'Cleirigh <mi...@rivulet.ca>.
Hi,
Another way is to have a service manage the list. We have a reference
data list (list of streets) in our application that has >100k elements
that is stored in memory and we have a service hold one instance that is
shared between all accessors (typically the autocomplete search fields)
of the data.
The list is built when the application starts up and has defined refresh
points and then all users can just access the data through the service.
e.g.
public List<StreetData> referenceDataService.find(String streetNamePrefix);
Since we use spring and inject the service using @SpringBean it is
wrapped in a proxy that prevents the 100k element list from being
serialized with the page.
Regards,
Mike
> I have One Array-list(with over 10000 objects in it) what is the best
> practice for caching it so I don't have to load it over and over again in
> each request?
>
> Jan
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org