Filter query cache issues

["Matt M." <go...@gmail.com>]

Hi,

I'm working with an index that contains 4,447,390 documents. The response
time for querying using facets is pretty darn slow. I'm fairly new to more
advanced Solr usage and today have started looking into the solrconfig.xml.
In the solr admin app, I noticed that the filterCache evictions were around
14,194,010 - is this saying that 14,194,010 items that were supposed to be
cached were not? Here are the stats as they stand currently. Would someone
mind looking at this and giving me an analysis of sorts?

Thank you - matt

filterCache - STATS:

**
lookups : 14576040
hits : 49737
hitratio : 0.00
inserts : 14526389
evictions : 14194010
size : 2048
cumulative_lookups : 14576040
cumulative_hits : 49737
cumulative_hitratio : 0.00
cumulative_inserts : 14526389
cumulative_evictions : 14194010

Re: Filter query cache issues

[Yonik Seeley <yo...@apache.org>]

On Fri, Feb 22, 2008 at 12:22 PM, Matt M. <go...@gmail.com> wrote:
>  I'm working with an index that contains 4,447,390 documents. The response
>  time for querying using facets is pretty darn slow. I'm fairly new to more
>  advanced Solr usage and today have started looking into the solrconfig.xml.
>  In the solr admin app, I noticed that the filterCache evictions were around
>  14,194,010 - is this saying that 14,194,010 items that were supposed to be
>  cached were not? Here are the stats as they stand currently. Would someone
>  mind looking at this and giving me an analysis of sorts?

The current faceting code only works well for certain term distributions:
1) single valued fields (where the Lucene FieldCache is used)
2) multi-valued fields with the number of unique terms <1000-10000 or so.

Do a single faceted query, and then check how the lookups in the
filterCache changed.
That will be the number of unique terms, and the filterCache size
should be set to be larger than this so everything will be cached.
Right now, the hit rate is 0.  You may or may not have enough memory
to use this method... just try it out to find out.

Sometime in the near future, we'll have a better faceting method for
multi-valued fields with many terms, provided that each document only
has a few terms on average:
https://issues.apache.org/jira/browse/SOLR-475

-Yonik

>  Thank you - matt
>
>  filterCache - STATS:
>
>  **
>  lookups : 14576040
>  hits : 49737
>  hitratio : 0.00
>  inserts : 14526389
>  evictions : 14194010
>  size : 2048
>  cumulative_lookups : 14576040
>  cumulative_hits : 49737
>  cumulative_hitratio : 0.00
>  cumulative_inserts : 14526389
>  cumulative_evictions : 14194010
>