You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Furkan KAMACI <fu...@gmail.com> on 2013/04/24 10:02:36 UTC
JVM Parameters to Startup Solr?
Lucidworks Solr Guide says that:
"If you are using Sun's JVM, add the -server command-line option when you
start Solr. This tells the JVM that it should optimize for a long running,
server process. If the Java runtime on your system is a JRE, rather than a
full JDK distribution (including javac and other development tools), then
it is possible that it may not support the -server JVM option"
Does any folks using -server parameter? Also what parameters you are using
to start up Solr? I mean parallel garbage collector vs.?
Re: JVM Parameters to Startup Solr?
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/24/2013 2:02 AM, Furkan KAMACI wrote:
> Lucidworks Solr Guide says that:
>
> "If you are using Sun's JVM, add the -server command-line option when you
> start Solr. This tells the JVM that it should optimize for a long running,
> server process. If the Java runtime on your system is a JRE, rather than a
> full JDK distribution (including javac and other development tools), then
> it is possible that it may not support the -server JVM option"
>
> Does any folks using -server parameter? Also what parameters you are using
> to start up Solr? I mean parallel garbage collector vs.?
The answers to your questions are hotly debated in Java communities.
This is treading on religious ground. :)
I never actually use the -server parameter. When java runs on my
multiprocessor 64-bit Linux machines, it already knows it should be in
server mode. If you run on a platform that Java decides is a client
machine, you might need the -server parameter.
Most people agree that you should use the CMS collector. You won't find
much agreement about anything else on the startup commandline. I can
tell you what I use. It may work for you, apart from the specific value
of the -Xmx parameter. These parameters result in fairly low GC pause
times for me. I can tell you that I have arrived at these parameters
through testing that wasn't very methodical, so they are probably not
the optimal settings:
-Xmx6144M -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75
-XX:NewRatio=3 -XX:MaxTenuringThreshold=8 -XX:+CMSParallelRemarkEnabled
-XX:+ParallelRefProcEnabled -XX:+UseLargePages -XX:+AggressiveOpts
The G1 collector is supposed to work for all situations without tuning,
but it didn't work for me. GC pause times were just as long as when I
had a badly tuned CMS setup.
Thanks,
Shawn
Re: JVM Parameters to Startup Solr?
Posted by Furkan KAMACI <fu...@gmail.com>.
Could you explain that what you mean with such kind of scripts? What it
checks and do exactly?
2013/4/25 Toke Eskildsen <te...@statsbiblioteket.dk>
> On Wed, 2013-04-24 at 18:03 +0200, Mark Miller wrote:
> > On Apr 24, 2013, at 12:00 PM, Mark Miller <ma...@gmail.com> wrote:
> >
> > >> -XX:OnOutOfMemoryError="kill -9 %p" -XX:+HeapDumpOnOutOfMemoryError
> >
> > The way I like to handle this is to have the OOM trigger a little script
> or set of cmds that logs the issue and kills the process.
>
> We treat all Errors as fatal by writing to a dedicated log and shutting
> down the JVM (which triggers the load balancer etc.). Unfortunately that
> means that some XML + XSLT combinations can bring the JVM down due to
> StackOverflowError. This might be a little too diligent as the Oracle
> JVM running on Linux (our current setup) is resilient to Threads hitting
> stack overflow.
>
> - Toke Eskildsen, State and University Library, Denmark
>
>
Re: JVM Parameters to Startup Solr?
Posted by Toke Eskildsen <te...@statsbiblioteket.dk>.
On Wed, 2013-04-24 at 18:03 +0200, Mark Miller wrote:
> On Apr 24, 2013, at 12:00 PM, Mark Miller <ma...@gmail.com> wrote:
>
> >> -XX:OnOutOfMemoryError="kill -9 %p" -XX:+HeapDumpOnOutOfMemoryError
>
> The way I like to handle this is to have the OOM trigger a little script or set of cmds that logs the issue and kills the process.
We treat all Errors as fatal by writing to a dedicated log and shutting
down the JVM (which triggers the load balancer etc.). Unfortunately that
means that some XML + XSLT combinations can bring the JVM down due to
StackOverflowError. This might be a little too diligent as the Oracle
JVM running on Linux (our current setup) is resilient to Threads hitting
stack overflow.
- Toke Eskildsen, State and University Library, Denmark
Re: JVM Parameters to Startup Solr?
Posted by Mark Miller <ma...@gmail.com>.
On Apr 24, 2013, at 12:22 PM, Timothy Potter <th...@gmail.com> wrote:
> I like the idea of running a script vs. kill -9 ;-) Right now when a
> node fails, we have monitors for whether a node is up and serving
> queries. If not, that triggers some manual investigation and restart
> process. Part of the process was to capture the logs and heap dump
> file. What happened previously is that the log capture part wasn't
> scripted into the restart process and so the logs got wiped out when
> the restart happened :-(
>
> One question about this - when you say "logs the issue" from your
> script - what type of things do you log? I've been relying on the
> timestamp of the heap dump (hprof) as a way to trace back into our log
> files.
Yeah, that's pretty much it - the time of the event and the fact that an OOM occurred. If you are dropping a heap dump, that has the same info, but a log is just a nice compact little history of events.
- Mark
>
> Thanks.
> Tim
>
> On Wed, Apr 24, 2013 at 10:03 AM, Mark Miller <ma...@gmail.com> wrote:
>>
>> On Apr 24, 2013, at 12:00 PM, Mark Miller <ma...@gmail.com> wrote:
>>
>>>> -XX:OnOutOfMemoryError="kill -9 %p" -XX:+HeapDumpOnOutOfMemoryError
>>
>> The way I like to handle this is to have the OOM trigger a little script or set of cmds that logs the issue and kills the process.
>>
>> Then if you have the process supervised (via runit or something), it will just start back up (what else do you do after an OOM?), but you will have logged something, triggered a notification, whatever.
>>
>> - Mark
Re: JVM Parameters to Startup Solr?
Posted by Timothy Potter <th...@gmail.com>.
I like the idea of running a script vs. kill -9 ;-) Right now when a
node fails, we have monitors for whether a node is up and serving
queries. If not, that triggers some manual investigation and restart
process. Part of the process was to capture the logs and heap dump
file. What happened previously is that the log capture part wasn't
scripted into the restart process and so the logs got wiped out when
the restart happened :-(
One question about this - when you say "logs the issue" from your
script - what type of things do you log? I've been relying on the
timestamp of the heap dump (hprof) as a way to trace back into our log
files.
Thanks.
Tim
On Wed, Apr 24, 2013 at 10:03 AM, Mark Miller <ma...@gmail.com> wrote:
>
> On Apr 24, 2013, at 12:00 PM, Mark Miller <ma...@gmail.com> wrote:
>
>>> -XX:OnOutOfMemoryError="kill -9 %p" -XX:+HeapDumpOnOutOfMemoryError
>
> The way I like to handle this is to have the OOM trigger a little script or set of cmds that logs the issue and kills the process.
>
> Then if you have the process supervised (via runit or something), it will just start back up (what else do you do after an OOM?), but you will have logged something, triggered a notification, whatever.
>
> - Mark
Re: JVM Parameters to Startup Solr?
Posted by Mark Miller <ma...@gmail.com>.
On Apr 24, 2013, at 12:00 PM, Mark Miller <ma...@gmail.com> wrote:
>> -XX:OnOutOfMemoryError="kill -9 %p" -XX:+HeapDumpOnOutOfMemoryError
The way I like to handle this is to have the OOM trigger a little script or set of cmds that logs the issue and kills the process.
Then if you have the process supervised (via runit or something), it will just start back up (what else do you do after an OOM?), but you will have logged something, triggered a notification, whatever.
- Mark
Re: JVM Parameters to Startup Solr?
Posted by Mark Miller <ma...@gmail.com>.
Yes, I recommend this. You can't predict a JVM that has had an OOM - so it's best to neutralize it. We have seen cases where the node was messed up but still advertised as active and good in zk due to OOM's. Behavior after an OOM is undefined.
I was actually going to ask if you were positive you had restarted that node in the other OOM thread, because that sounded similar. Just a straw to grasp for, as I'd guess you are sure you did restart it.
- Mark
On Apr 24, 2013, at 11:37 AM, Timothy Potter <th...@gmail.com> wrote:
> Just verifying that it is also recommended to use the JVM options to
> kill on OOM? I vaguely recall a message from Mark about this sometime
> ago:
>
> -XX:OnOutOfMemoryError="kill -9 %p" -XX:+HeapDumpOnOutOfMemoryError
>
> On Wed, Apr 24, 2013 at 9:13 AM, Mark Miller <ma...@gmail.com> wrote:
>>
>> On Apr 24, 2013, at 4:02 AM, Furkan KAMACI <fu...@gmail.com> wrote:
>>
>>> Lucidworks Solr Guide says that:
>>>
>>> "If you are using Sun's JVM, add the -server command-line option when you
>>> start Solr. This tells the JVM that it should optimize for a long running,
>>> server process. If the Java runtime on your system is a JRE, rather than a
>>> full JDK distribution (including javac and other development tools), then
>>> it is possible that it may not support the -server JVM option"
>>>
>>> Does any folks using -server parameter? Also what parameters you are using
>>> to start up Solr? I mean parallel garbage collector vs.?
>>
>> Unless you are using 32-bit Windows, you are probably getting the server JVM. It's not a bad idea to use -server to be sure - it's certainly preferable to -client for Solr.
>>
>> You should generally use the concurrent low pause garbage collector with Solr.
>>
>> - Mark
>>
Re: JVM Parameters to Startup Solr?
Posted by Timothy Potter <th...@gmail.com>.
Just verifying that it is also recommended to use the JVM options to
kill on OOM? I vaguely recall a message from Mark about this sometime
ago:
-XX:OnOutOfMemoryError="kill -9 %p" -XX:+HeapDumpOnOutOfMemoryError
On Wed, Apr 24, 2013 at 9:13 AM, Mark Miller <ma...@gmail.com> wrote:
>
> On Apr 24, 2013, at 4:02 AM, Furkan KAMACI <fu...@gmail.com> wrote:
>
>> Lucidworks Solr Guide says that:
>>
>> "If you are using Sun's JVM, add the -server command-line option when you
>> start Solr. This tells the JVM that it should optimize for a long running,
>> server process. If the Java runtime on your system is a JRE, rather than a
>> full JDK distribution (including javac and other development tools), then
>> it is possible that it may not support the -server JVM option"
>>
>> Does any folks using -server parameter? Also what parameters you are using
>> to start up Solr? I mean parallel garbage collector vs.?
>
> Unless you are using 32-bit Windows, you are probably getting the server JVM. It's not a bad idea to use -server to be sure - it's certainly preferable to -client for Solr.
>
> You should generally use the concurrent low pause garbage collector with Solr.
>
> - Mark
>
Re: JVM Parameters to Startup Solr?
Posted by Mark Miller <ma...@gmail.com>.
On Apr 24, 2013, at 4:02 AM, Furkan KAMACI <fu...@gmail.com> wrote:
> Lucidworks Solr Guide says that:
>
> "If you are using Sun's JVM, add the -server command-line option when you
> start Solr. This tells the JVM that it should optimize for a long running,
> server process. If the Java runtime on your system is a JRE, rather than a
> full JDK distribution (including javac and other development tools), then
> it is possible that it may not support the -server JVM option"
>
> Does any folks using -server parameter? Also what parameters you are using
> to start up Solr? I mean parallel garbage collector vs.?
Unless you are using 32-bit Windows, you are probably getting the server JVM. It's not a bad idea to use -server to be sure - it's certainly preferable to -client for Solr.
You should generally use the concurrent low pause garbage collector with Solr.
- Mark