Re: What's your query result cache's stats?

[Shawn Heisey <so...@elyograg.org>]

On 5/31/2011 3:02 PM, Markus Jelsma wrote:
> Hi,
>
> I've seen the stats page many times, of quite a few installations and even
> more servers. There's one issue that keeps bothering me: the cumulative hit
> ratio of the query result cache, it's almost never higher than 50%.
>
> What are your stats? How do you deal with it?

Below are my stats.

I will be lowering my warmcounts dramatically when I respin for 3.1.  
The 28 second warm time is too high for me.  I don't think it's going to 
make a lot of difference in performance.  I think most of the warming 
benefit is realized after the first few queries.

queryResultCache:
Concurrent LRU Cache(maxSize=1024, initialSize=1024, minSize=921, 
acceptableSize=972, cleanupThread=true, autowarmCount=64, 
regenerator=org.apache.solr.search.SolrIndexSearcher$3@60c0c8b5)

lookups : 932
hits : 528
hitratio : 0.56
inserts : 403
evictions : 0
size : 449
warmupTime : 28198
cumulative_lookups : 980357
cumulative_hits : 622726
cumulative_hitratio : 0.63
cumulative_inserts : 369692
cumulative_evictions : 83711

lookups : 68543
hits : 57286
hitratio : 0.83
inserts : 11357
evictions : 0
size : 11357
warmupTime : 0
cumulative_lookups : 219118491
cumulative_hits : 179119106
cumulative_hitratio : 0.81
cumulative_inserts : 39999385
cumulative_evictions : 32833254


documentCache:
LRU Cache(maxSize=16384, initialSize=4096)

lookups : 68543
hits : 57286
hitratio : 0.83
inserts : 11357
evictions : 0
size : 11357
warmupTime : 0
cumulative_lookups : 219118491
cumulative_hits : 179119106
cumulative_hitratio : 0.81
cumulative_inserts : 39999385
cumulative_evictions : 32833254


filterCache:
LRU Cache(maxSize=512, initialSize=512, autowarmCount=32, 
regenerator=org.apache.solr.search.SolrIndexSearcher$2@6910b640)

lookups : 859
hits : 464
hitratio : 0.54
inserts : 465
evictions : 0
size : 464
warmupTime : 27747
cumulative_lookups : 682600
cumulative_hits : 355130
cumulative_hitratio : 0.52
cumulative_inserts : 327479
cumulative_evictions : 161624

Re: What's your query result cache's stats?

[Jonathan Rochkind <ro...@jhu.edu>]

I believe you need SOME query cache even with low hit counts, for things 
like a user paging through results. You want the query to still be in 
the cache when they go to the next page or what have you. Other 
operations like this may depend on the query cache too for good 
performance.

So even with a low hit rate, you still want enough query cache that all 
the "current" queries, all the queries someone is in the middle of doing 
something with and may do more with can stay in the cache. (what things 
those are can depend on your particular client interface).   So the 
cache hit count may not actually be a good guide to sizing your query 
cache.

Correct me if I'm wrong, but this is what I've been thinking.

On 6/1/2011 12:03 PM, Shawn Heisey wrote:
> On 5/31/2011 3:02 PM, Markus Jelsma wrote:
>> Hi,
>>
>> I've seen the stats page many times, of quite a few installations and 
>> even
>> more servers. There's one issue that keeps bothering me: the 
>> cumulative hit
>> ratio of the query result cache, it's almost never higher than 50%.
>>
>> What are your stats? How do you deal with it?
>
> Below are my stats.
>
> I will be lowering my warmcounts dramatically when I respin for 3.1.  
> The 28 second warm time is too high for me.  I don't think it's going 
> to make a lot of difference in performance.  I think most of the 
> warming benefit is realized after the first few queries.
>
> queryResultCache:
> Concurrent LRU Cache(maxSize=1024, initialSize=1024, minSize=921, 
> acceptableSize=972, cleanupThread=true, autowarmCount=64, 
> regenerator=org.apache.solr.search.SolrIndexSearcher$3@60c0c8b5)
>
> lookups : 932
> hits : 528
> hitratio : 0.56
> inserts : 403
> evictions : 0
> size : 449
> warmupTime : 28198
> cumulative_lookups : 980357
> cumulative_hits : 622726
> cumulative_hitratio : 0.63
> cumulative_inserts : 369692
> cumulative_evictions : 83711
>
> lookups : 68543
> hits : 57286
> hitratio : 0.83
> inserts : 11357
> evictions : 0
> size : 11357
> warmupTime : 0
> cumulative_lookups : 219118491
> cumulative_hits : 179119106
> cumulative_hitratio : 0.81
> cumulative_inserts : 39999385
> cumulative_evictions : 32833254
>
>
> documentCache:
> LRU Cache(maxSize=16384, initialSize=4096)
>
> lookups : 68543
> hits : 57286
> hitratio : 0.83
> inserts : 11357
> evictions : 0
> size : 11357
> warmupTime : 0
> cumulative_lookups : 219118491
> cumulative_hits : 179119106
> cumulative_hitratio : 0.81
> cumulative_inserts : 39999385
> cumulative_evictions : 32833254
>
>
> filterCache:
> LRU Cache(maxSize=512, initialSize=512, autowarmCount=32, 
> regenerator=org.apache.solr.search.SolrIndexSearcher$2@6910b640)
>
> lookups : 859
> hits : 464
> hitratio : 0.54
> inserts : 465
> evictions : 0
> size : 464
> warmupTime : 27747
> cumulative_lookups : 682600
> cumulative_hits : 355130
> cumulative_hitratio : 0.52
> cumulative_inserts : 327479
> cumulative_evictions : 161624
>
>
>