RE: facet optimizing

["Gunther, Andrew" <Gu...@si.edu>]

Back on facet optimizing again. 

Can someone post their magic formula for filterCache (Erik?)  We've hit
a plateau around 1.7mill docs and my response times have suffered when
filtering.  Have adjusted filtercache up and down all day but can't seem
to get a good handle on these values.  What does size actually correlate
to (number or named-values pairs in hashmap?)  Can adjusting JVM memory
on startup factor in any.

Cheers,

Andrew

-----Original Message-----
From: Erik Hatcher [mailto:erik@ehatchersolutions.com] 
Sent: Wednesday, February 07, 2007 5:08 PM
To: solr-user@lucene.apache.org
Subject: Re: facet optimizing


On Feb 7, 2007, at 4:42 PM, Yonik Seeley wrote:
> Solr relies on the filter cache for faceting, and if it's not big
> enough you're going to get a near 0% hit rate.  Check the statistics
> page and make sure there aren't any evictions after you do a query
> with facets.  If there are, make the cache larger.

Yonik - thanks!   I was too deep into other things to worry about the  
slowness of massive multiValued facets, mainly because I was going to  
use the mess of all those nasty values we have in typical library  
data to push back and have it cleaned up.  But, I just adjusted my  
filter cache settings and my responses went from 2000+ ms to 85 ms!   
Now it takes longer to render the pie charts than it does to get the  
results back :)

	Erik

Re: facet optimizing

[Yonik Seeley <yo...@apache.org>]

On 3/1/07, Gunther, Andrew <Gu...@si.edu> wrote:
> Can someone post their magic formula for filterCache (Erik?)  We've hit
> a plateau around 1.7mill docs and my response times have suffered when
> filtering.

Is this for field faceting (facet.field)?

> Have adjusted filtercache up and down all day but can't seem
> to get a good handle on these values.  What does size actually correlate
> to (number or named-values pairs in hashmap?)

Yes.
Right now, field faceting isn't incredibly scalable if the number of
unique values for the field is high (it iterates through all the
values).  About all you can do is increase the filterCache to be
larger than the number of unique values in your field (check the admin
statistics to see if it's large enough).

There are plans to try and remedy this (the ideas for tree faceting I
talked about in this thread previously), but I haven't had a chance to
work on it recently.  I will eventually though... this is the fun
stuff to work on :-)

http://issues.apache.org/jira/browse/SOLR-153

-Yonik