cache sizes

[Andrew Nagy <an...@villanova.edu>]

Is their a science to choosing a cache sizes?  I have about 500,000 
records and am seeing a lot of evictions, about 50% of lookups.  What 
factors can i look at to determine what my cache sizes should be?

Here are my cache statistics:

filterCache 
class:     org.apache.solr.search.LRUCache 
version:     1.0 
description:     LRU Cache(maxSize=100000, initialSize=50000, 
autowarmCount=50000, 
regenerator=org.apache.solr.search.SolrIndexSearcher$1@1efb4be) 
stats:     lookups : 3195074
hits : 1541249
hitratio : 0.48
inserts : 1653824
evictions : 1553824
size : 100000
cumulative_lookups : 3195074
cumulative_hits : 1541249
cumulative_hitratio : 0.48
cumulative_inserts : 1653825
cumulative_evictions : 1553825
   
name:     queryResultCache 
class:     org.apache.solr.search.LRUCache 
version:     1.0 
description:     LRU Cache(maxSize=100000, initialSize=5000, 
autowarmCount=5000, 
regenerator=org.apache.solr.search.SolrIndexSearcher$2@77eaf8) 
stats:     lookups : 1976
hits : 872
hitratio : 0.44
inserts : 1199
evictions : 0
size : 1104
cumulative_lookups : 1976
cumulative_hits : 872
cumulative_hitratio : 0.44
cumulative_inserts : 1199
cumulative_evictions : 0
   
name:     documentCache 
class:     org.apache.solr.search.LRUCache 
version:     1.0 
description:     LRU Cache(maxSize=512, initialSize=512) 
stats:     lookups : 177530
hits : 13255
hitratio : 0.07
inserts : 164275
evictions : 163763
size : 512
cumulative_lookups : 177530
cumulative_hits : 13255
cumulative_hitratio : 0.07
cumulative_inserts : 164275
cumulative_evictions : 163763

Re: cache sizes

[Erik Hatcher <er...@ehatchersolutions.com>]

On Mar 16, 2007, at 2:21 PM, Andrew Nagy wrote:
> Is their a science to choosing a cache sizes?  I have about 500,000  
> records and am seeing a lot of evictions, about 50% of lookups.   
> What factors can i look at to determine what my cache sizes should be?

Roughly you could start with getting a count of all unique values in  
all the fields you're faceting on.  This would get you practically  
there, though other filters get used if you're leveraging other  
aspects of Solr, but the bulk of them are your facets I'm sure.

You can use Luke's first screen to get some counts, or do one facet  
at a time from a fresh Solr instance and see how many filters each  
one creates.

	Erik