OOME diagnosis - possible to disable caching?

[James Brady <ja...@gmail.com>]

Hi all, I have 20 indices, each ~10GB in size, being searched by a single
Solr slave instance (using the multicore features in a slightly old 1.2 dev
build)
I'm getting unpredictable, but inevitable, OutOfMemoryError from the slave,
and I have no more physical memory to throw at the problem (HotSpot 1.6 with
Xmx=4000m). At this point, I'd like to see how much memory Lucene is eating
by disabling all supplemental Solr caches.

Which solrconfig settings do I need to be paying attention to
here? filterCache, queryResultCache and documentCache?

I'm not faceting, sorting, highlighting or anything like that (all in an
effort to get more docs in a searchable index!)

Thanks!
James

Re: OOME diagnosis - possible to disable caching?

[Mike Klaas <mi...@gmail.com>]

On 19-Jan-09, at 2:44 PM, James Brady wrote:

> Hi all, I have 20 indices, each ~10GB in size, being searched by a  
> single
> Solr slave instance (using the multicore features in a slightly old  
> 1.2 dev
> build)
> I'm getting unpredictable, but inevitable, OutOfMemoryError from the  
> slave,
> and I have no more physical memory to throw at the problem (HotSpot  
> 1.6 with
> Xmx=4000m). At this point, I'd like to see how much memory Lucene is  
> eating
> by disabling all supplemental Solr caches.
>
> Which solrconfig settings do I need to be paying attention to
> here? filterCache, queryResultCache and documentCache?

That should be it.  Note that it is unadvisable to reduce  
documentCache size to less than 100 or so, as Solr assumes that enough  
cache for the docs for one query.

A heap dump should help point you in to the relevant problems.  Also,  
it should be easy to get a rough estimate of the filtercache mem usage  
by looking at its size.

-Mike