[jira] [Commented] (SOLR-13394) Change default GC from CMS to G1

["Robert Bunch (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-13394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16960387#comment-16960387 ] 

Robert Bunch commented on SOLR-13394:
-------------------------------------

late chiming in on this...  but I agree, it'd be very, very helpful for it to be a knob that can be turned (like -debug).  Perhaps also enable JMX w/ the same switch?

I manage a lot of clusters, Solr and ES, and any means of gathering info – specifically old gen FGC's – is invaluable.  Discovering the JVMs in the first place is critical.
{code:java}
 #!/bin/bash 
for pid in `pgrep -f start.jar`; do
# for pid in `pgrep java`; do # normally I use this, as I don't have a local ZK...
   st=`ps -eo pid,etime | grep ${pid} | awk '{$1=""; print $0}'`
   cli=`${JAVA_HOME}/bin/jinfo $pid 2>/dev/null | grep -i xmx | awk '{print $3}'` # placement of -Xmx will vary
   if [[ "`uname`" == "Darwin" ]]; then 
      port=`lsof -p $pid | grep TCP | grep LISTEN | grep :89 | awk '{print $9}'`
   else
      port=`netstat -nap 2>/dev/null | grep $pid | grep :89 | grep LISTEN | awk '{print $4}'` # this will vary, JMX port conflict?
   fi
   echo $port | grep ":89" >/dev/null; is_solr_port=$?
   if [[ $is_solr_port -eq 0 ]]; then
      fgc=`${JAVA_HOME}/bin/jstat -gcutil $pid | awk '{print $9, $10}' | tail -1` # this will vary, java 7 vs 8 vs... get count, time
      if [[ "$cli" == "" ]]; then cli="UNKNOWN"; fi
      printf "%s %6d %'9d %'11.2f %s %s\n" $port $pid $fgc $cli $st
   fi
done | sort | egrep --color=always '^.*?[0-9]{4} +[0-9]{3,} +[0-9,]{4,} .*|$' # change throttle of redline at 3rd grouping
for j in $(seq 1 60); do echo -n '.'; sleep 60; done{code}
when it works (port, pid, # old gen FGCs, total old gen FGC time, -Xmx param, process uptime (days-hours:min:sec)
{code:java}
$ ./fgc-jira.sh 
*:8981  15685         0        0.00 -Xmx1g 06:15:13
*:8982  15779         0        0.00 -Xmx1g 06:15:09
*:8983  15873         0        0.00 -Xmx1g 06:15:04
*:8984  15967         0        0.00 -Xmx1g 06:15:00
...^C  (it'll tick off a dot for each minute, then an hour later here's another sample){code}
 

when I can't get to that info (in this case, :8985 is 8.2 running vanilla w/ -XX:+PerfDisableSharedMem)
{code:java}
$ ./fgc-jira.sh 
sun.jvmstat.monitor.MonitorException: 17297 not found
	at jdk.internal.jvmstat/sun.jvmstat.perfdata.monitor.protocol.local.PerfDataBuffer.<init>(PerfDataBuffer.java:84)
	at jdk.internal.jvmstat/sun.jvmstat.perfdata.monitor.protocol.local.LocalMonitoredVm.<init>(LocalMonitoredVm.java:68)
	at jdk.internal.jvmstat/sun.jvmstat.perfdata.monitor.protocol.local.MonitoredHostProvider.getMonitoredVm(MonitoredHostProvider.java:77)
	at jdk.jcmd/sun.tools.jstat.Jstat.logSamples(Jstat.java:107)
	at jdk.jcmd/sun.tools.jstat.Jstat.main(Jstat.java:70)
Caused by: java.lang.IllegalArgumentException: Could not map vmid to user Name
	at java.base/jdk.internal.perf.Perf.attach(Native Method)
	at java.base/jdk.internal.perf.Perf.attachImpl(Perf.java:270)
	at java.base/jdk.internal.perf.Perf.attach(Perf.java:200)
	at jdk.internal.jvmstat/sun.jvmstat.perfdata.monitor.protocol.local.PerfDataBuffer.<init>(PerfDataBuffer.java:64)
	... 4 more
./fgc-jira.sh: line 15: printf: -XX:+UseG1GC: invalid number
./fgc-jira.sh: line 15: printf: 58:12: invalid number
 $ ./fgc-jira.sh 
*:8981  15685         0        0.00 -Xmx1g 06:15:13
*:8982  15779         0        0.00 -Xmx1g 06:15:09
*:8983  15873         0        0.00 -Xmx1g 06:15:04
*:8984  15967         0        0.00 -Xmx1g 06:15:00
*:8985  17297         0        0.00 (blew up trying to get jinfo for this node)
...^C{code}
 

> Change default GC from CMS to G1
> --------------------------------
>
>                 Key: SOLR-13394
>                 URL: https://issues.apache.org/jira/browse/SOLR-13394
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: Ishan Chattopadhyaya
>            Assignee: Ishan Chattopadhyaya
>            Priority: Major
>             Fix For: 8.1
>
>         Attachments: SOLR-13394.patch, SOLR-13394.patch
>
>          Time Spent: 1h
>  Remaining Estimate: 0h
>
> CMS has been deprecated in new versions of Java (http://openjdk.java.net/jeps/291). This issue is to switch Solr default from CMS to G1.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org