What options would you recommend for the Sun JVM?

[Jérôme Etévé <je...@gmail.com>]

Hi solr addicts,

I know there's no one size fits all set of options for the sun JVM,
but I think It'd be useful to everyone to share your tips on using the
sun JVM with solr.

For instance, I recently figured out that setting the tenured
generation garbage collection to Concurrent mark and sweep (
-XX:+UseConcMarkSweepGC )  have dramatically decreased the amount of
time java hangs on tenured gen. garbage collecting. On my settings,
the old gen. garbage collection went from big time chunks of 1~2
second to multiple small slices of ~0.2 s.

As a result, the commits (hence the searcher drop/rebuild) are much
less painful from the application performance point of view.

What are the other options you would recommend?

Cheers!

Jerome.

-- 
Jerome Eteve.
http://www.eteve.net
jerome@eteve.net

Re: What options would you recommend for the Sun JVM?

[Grant Ingersoll <gs...@apache.org>]

On Sep 25, 2009, at 7:30 AM, Jérôme Etévé wrote:

> Hi solr addicts,
>
> I know there's no one size fits all set of options for the sun JVM,
> but I think It'd be useful to everyone to share your tips on using the
> sun JVM with solr.
>
> For instance, I recently figured out that setting the tenured
> generation garbage collection to Concurrent mark and sweep (
> -XX:+UseConcMarkSweepGC )  have dramatically decreased the amount of
> time java hangs on tenured gen. garbage collecting. On my settings,
> the old gen. garbage collection went from big time chunks of 1~2
> second to multiple small slices of ~0.2 s.
>
> As a result, the commits (hence the searcher drop/rebuild) are much
> less painful from the application performance point of view.
>
> What are the other options you would recommend?


Mark M. just posted some on this: http://www.lucidimagination.com/blog/2009/09/19/java-garbage-collection-boot-camp-draft/

I'm usually wary of futzing too much w/ the parameters.  With a proper  
machine (multiple cores), the concurrent low pause collector is the  
way to go and I usually leave it at that.  Beyond that, I don't  
usually recommend getting involved with too many parameters.  GC  
settings are often a black art and I've seen many cases of people  
going down the rat hole of trying to figure out how to get their GC  
right, when they could have spent far less time thinking about how  
they model their domain in Lucene/Solr to produce less garbage in the  
first place without effecting functionality one bit.  After all, most  
shops have domain expertise and not GC expertise.

The other thing to be wary of is too big of a heap.  Basically, take a  
pragmatic approach and test under load using real queries/indexing and  
see what the heap high water mark is (which is almost always evident  
during and right after a commit) and then set it at that plus maybe 1  
more GB just to be on the safe side.

--------------------------
Grant Ingersoll
http://www.lucidimagination.com/

Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)  
using Solr/Lucene:
http://www.lucidimagination.com/search