You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Otis Gospodnetic <ot...@gmail.com> on 2012/12/19 02:24:47 UTC
Re: occasional GC crashes
Robert,
Step 1 is to get the latest Java 7 or if you have to remain on 6 then use
the latest 6.
Otis
--
SOLR Performance Monitoring - http://sematext.com/spm
On Dec 18, 2012 7:54 PM, "Petersen, Robert" <ro...@buy.com> wrote:
> Hi solr user group,****
>
> ** **
>
> Sorry if this isn’t directly a Solr question. Seems like once in a blue
> moon the GC crashes on a server in our Solr 3.6.1 slave farm. This seems
> to only happen on a couple of the twelve slaves we have deployed and only
> very rarely on those. It seems like this doesn’t directly affect solr
> because in the logs it looks like solr keeps working after the time of the
> exception but our external monitoring tool reports that the solr service is
> down so our operations department restarts solr on that box and alerts me.
> The solr logs show nothing unusual. The exception does show up in the
> catalina.out log file though. Does this happen to anyone else? Here is
> the basic error and I have attached the crash dump file also. Our total
> uptime on these boxes is over a year now BTW.****
>
> ** **
>
> #****
>
> # A fatal error has been detected by the Java Runtime Environment:****
>
> #****
>
> # SIGSEGV (0xb) at pc=0x00002b5379346612, pid=13724, tid=1082353984****
>
> #****
>
> # JRE version: 6.0_25-b06****
>
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.0-b11 mixed mode
> linux-amd64 )****
>
> # Problematic frame:****
>
> # V [libjvm.so+0x3c4612] Par_ConcMarkingClosure::trim_queue(unsigned
> long)+0x82****
>
> #****
>
> # An error report file with more information is saved as:****
>
> # /var/LucidWorks/lucidworks/hs_err_pid13724.log****
>
> #****
>
> # If you would like to submit a bug report, please visit:****
>
> # http://java.sun.com/webapps/bugreport/crash.jsp****
>
> #****
>
> ** **
>
> VM Arguments:****
>
> jvm_args:
> -Djava.util.logging.config.file=/var/LucidWorks/lucidworks/tomcat/conf/logging.properties
> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> -Xmx32768m -Xms32768m -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode
> -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.port=6060
> -Djava.endorsed.dirs=/var/LucidWorks/lucidworks/tomcat/endorsed
> -Dcatalina.base=/var/LucidWorks/lucidworks/tomcat
> -Dcatalina.home=/var/LucidWorks/lucidworks/tomcat
> -Djava.io.tmpdir=/var/LucidWorks/lucidworks/tomcat/temp ****
>
> java_command: org.apache.catalina.startup.Bootstrap -server
> -Dsolr.solr.home=lucidworks/solr start****
>
> Launcher Type: SUN_STANDARD****
>
> ** **
>
> Stack: [0x0000000000000000,0x0000000000000000], sp=0x0000000040835eb0,
> free space=1056983k****
>
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
> code)****
>
> V [libjvm.so+0x3c4612] Par_ConcMarkingClosure::trim_queue(unsigned
> long)+0x82****
>
> V [libjvm.so+0x3c481a] CMSConcMarkingTask::do_work_steal(int)+0xfa****
>
> V [libjvm.so+0x3c3dcf] CMSConcMarkingTask::work(int)+0xef****
>
> V [libjvm.so+0x8783dc] YieldingFlexibleGangWorker::loop()+0xbc****
>
> V [libjvm.so+0x8755b4] GangWorker::run()+0x24****
>
> V [libjvm.so+0x71096f] java_start(Thread*)+0x13f****
>
> ** **
>
> Heap****
>
> par new generation total 345024K, used 180672K [0x00002aaaae120000,
> 0x00002aaac5780000, 0x00002aaac5780000)****
>
> eden space 306688K, 53% used [0x00002aaaae120000, 0x00002aaab8243c28,
> 0x00002aaac0ca0000)****
>
> from space 38336K, 40% used [0x00002aaac3210000, 0x00002aaac415c3f8,
> 0x00002aaac5780000)****
>
> to space 38336K, 0% used [0x00002aaac0ca0000, 0x00002aaac0ca0000,
> 0x00002aaac3210000)****
>
> concurrent mark-sweep generation total 33171072K, used 12144213K
> [0x00002aaac5780000, 0x00002ab2ae120000, 0x00002ab2ae120000)****
>
> concurrent-mark-sweep perm gen total 83968K, used 50650K
> [0x00002ab2ae120000, 0x00002ab2b3320000, 0x00002ab2b3320000)****
>
> ** **
>
> Code Cache [0x00002aaaab054000, 0x00002aaaab9a4000, 0x00002aaaae054000)**
> **
>
> total_blobs=2800 nmethods=2273 adapters=480 free_code_cache=40752512
> largest_free_block=15808****
>
> ** **
>
> ** **
>
> ** **
>
> Thanks,****
>
> ** **
>
> *Robert (Robi) Petersen*
>
> Senior Software Engineer****
>
> Search Department****
>
> ** **
>
RE: occasional GC crashes
Posted by "Petersen, Robert" <ro...@buy.com>.
Thanks Otis, will do.
-----Original Message-----
From: Otis Gospodnetic [mailto:otis.gospodnetic@gmail.com]
Sent: Thursday, December 20, 2012 7:01 PM
To: solr-user@lucene.apache.org
Subject: RE: occasional GC crashes
Hi Robi,
Oh that's the thing of the past, go for the latest Java 7 if they let you!
Otis
--
Performance Monitoring - http://sematext.com/spm On Dec 20, 2012 6:29 PM, "Petersen, Robert" <ro...@buy.com> wrote:
> Hi Otis,
>
> I thought Java 7 had a bug which wasn't being addressed by Oracle
> which was making it not suitable for Solr. Did that get fixed now?
> http://searchhub.org/2011/07/28/dont-use-java-7-for-anything/
>
> I did see this but it doesn't really mention the bug:
> http://opensearchnews.com/2012/04/announcing-java7-support-with-apache
> -solr-and-lucene/
>
> Thanks
> Robi
>
>
> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis.gospodnetic@gmail.com]
> Sent: Tuesday, December 18, 2012 5:25 PM
> To: solr-user@lucene.apache.org
> Subject: Re: occasional GC crashes
>
> Robert,
>
> Step 1 is to get the latest Java 7 or if you have to remain on 6 then
> use the latest 6.
>
> Otis
> --
> SOLR Performance Monitoring - http://sematext.com/spm On Dec 18, 2012
> 7:54 PM, "Petersen, Robert" <ro...@buy.com> wrote:
>
> > Hi solr user group,****
> >
> > ** **
> >
> > Sorry if this isn't directly a Solr question. Seems like once in a
> > blue moon the GC crashes on a server in our Solr 3.6.1 slave farm.
> > This seems to only happen on a couple of the twelve slaves we have
> > deployed and only very rarely on those. It seems like this doesn't
> > directly affect solr because in the logs it looks like solr keeps
> > working after the time of the exception but our external monitoring
> > tool reports that the solr service is down so our operations
> > department
> restarts solr on that box and alerts me.
> > The solr logs show nothing unusual. The exception does show up in
> > the catalina.out log file though. Does this happen to anyone else? Here is
> > the basic error and I have attached the crash dump file also. Our total
> > uptime on these boxes is over a year now BTW.****
> >
> > ** **
> >
> > #****
> >
> > # A fatal error has been detected by the Java Runtime
> > Environment:****
> >
> > #****
> >
> > # SIGSEGV (0xb) at pc=0x00002b5379346612, pid=13724,
> > tid=1082353984****
> >
> > #****
> >
> > # JRE version: 6.0_25-b06****
> >
> > # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.0-b11 mixed mode
> > linux-amd64 )****
> >
> > # Problematic frame:****
> >
> > # V [libjvm.so+0x3c4612]
> > Par_ConcMarkingClosure::trim_queue(unsigned
> > long)+0x82****
> >
> > #****
> >
> > # An error report file with more information is saved as:****
> >
> > # /var/LucidWorks/lucidworks/hs_err_pid13724.log****
> >
> > #****
> >
> > # If you would like to submit a bug report, please visit:****
> >
> > # http://java.sun.com/webapps/bugreport/crash.jsp****
> >
> > #****
> >
> > ** **
> >
> > VM Arguments:****
> >
> > jvm_args:
> > -Djava.util.logging.config.file=/var/LucidWorks/lucidworks/tomcat/co
> > nf
> > /logging.properties
> > -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> > -Xmx32768m -Xms32768m -XX:+UseConcMarkSweepGC
> > -XX:+CMSIncrementalMode -Dcom.sun.management.jmxremote
> > -Dcom.sun.management.jmxremote.ssl=false
> > -Dcom.sun.management.jmxremote.authenticate=false
> > -Dcom.sun.management.jmxremote.port=6060
> > -Djava.endorsed.dirs=/var/LucidWorks/lucidworks/tomcat/endorsed
> > -Dcatalina.base=/var/LucidWorks/lucidworks/tomcat
> > -Dcatalina.home=/var/LucidWorks/lucidworks/tomcat
> > -Djava.io.tmpdir=/var/LucidWorks/lucidworks/tomcat/temp ****
> >
> > java_command: org.apache.catalina.startup.Bootstrap -server
> > -Dsolr.solr.home=lucidworks/solr start****
> >
> > Launcher Type: SUN_STANDARD****
> >
> > ** **
> >
> > Stack: [0x0000000000000000,0x0000000000000000],
> > sp=0x0000000040835eb0, free space=1056983k****
> >
> > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
> > C=native
> > code)****
> >
> > V [libjvm.so+0x3c4612] Par_ConcMarkingClosure::trim_queue(unsigned
> > long)+0x82****
> >
> > V [libjvm.so+0x3c481a]
> > CMSConcMarkingTask::do_work_steal(int)+0xfa****
> >
> > V [libjvm.so+0x3c3dcf] CMSConcMarkingTask::work(int)+0xef****
> >
> > V [libjvm.so+0x8783dc] YieldingFlexibleGangWorker::loop()+0xbc****
> >
> > V [libjvm.so+0x8755b4] GangWorker::run()+0x24****
> >
> > V [libjvm.so+0x71096f] java_start(Thread*)+0x13f****
> >
> > ** **
> >
> > Heap****
> >
> > par new generation total 345024K, used 180672K [0x00002aaaae120000,
> > 0x00002aaac5780000, 0x00002aaac5780000)****
> >
> > eden space 306688K, 53% used [0x00002aaaae120000,
> > 0x00002aaab8243c28,
> > 0x00002aaac0ca0000)****
> >
> > from space 38336K, 40% used [0x00002aaac3210000,
> > 0x00002aaac415c3f8,
> > 0x00002aaac5780000)****
> >
> > to space 38336K, 0% used [0x00002aaac0ca0000, 0x00002aaac0ca0000,
> > 0x00002aaac3210000)****
> >
> > concurrent mark-sweep generation total 33171072K, used 12144213K
> > [0x00002aaac5780000, 0x00002ab2ae120000, 0x00002ab2ae120000)****
> >
> > concurrent-mark-sweep perm gen total 83968K, used 50650K
> > [0x00002ab2ae120000, 0x00002ab2b3320000, 0x00002ab2b3320000)****
> >
> > ** **
> >
> > Code Cache [0x00002aaaab054000, 0x00002aaaab9a4000,
> > 0x00002aaaae054000)**
> > **
> >
> > total_blobs=2800 nmethods=2273 adapters=480 free_code_cache=40752512
> > largest_free_block=15808****
> >
> > ** **
> >
> > ** **
> >
> > ** **
> >
> > Thanks,****
> >
> > ** **
> >
> > *Robert (Robi) Petersen*
> >
> > Senior Software Engineer****
> >
> > Search Department****
> >
> > ** **
> >
>
>
RE: occasional GC crashes
Posted by Otis Gospodnetic <ot...@gmail.com>.
Hi Robi,
Oh that's the thing of the past, go for the latest Java 7 if they let you!
Otis
--
Performance Monitoring - http://sematext.com/spm
On Dec 20, 2012 6:29 PM, "Petersen, Robert" <ro...@buy.com> wrote:
> Hi Otis,
>
> I thought Java 7 had a bug which wasn't being addressed by Oracle which
> was making it not suitable for Solr. Did that get fixed now?
> http://searchhub.org/2011/07/28/dont-use-java-7-for-anything/
>
> I did see this but it doesn't really mention the bug:
> http://opensearchnews.com/2012/04/announcing-java7-support-with-apache-solr-and-lucene/
>
> Thanks
> Robi
>
>
> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis.gospodnetic@gmail.com]
> Sent: Tuesday, December 18, 2012 5:25 PM
> To: solr-user@lucene.apache.org
> Subject: Re: occasional GC crashes
>
> Robert,
>
> Step 1 is to get the latest Java 7 or if you have to remain on 6 then use
> the latest 6.
>
> Otis
> --
> SOLR Performance Monitoring - http://sematext.com/spm On Dec 18, 2012
> 7:54 PM, "Petersen, Robert" <ro...@buy.com> wrote:
>
> > Hi solr user group,****
> >
> > ** **
> >
> > Sorry if this isn't directly a Solr question. Seems like once in a
> > blue moon the GC crashes on a server in our Solr 3.6.1 slave farm.
> > This seems to only happen on a couple of the twelve slaves we have
> > deployed and only very rarely on those. It seems like this doesn't
> > directly affect solr because in the logs it looks like solr keeps
> > working after the time of the exception but our external monitoring
> > tool reports that the solr service is down so our operations department
> restarts solr on that box and alerts me.
> > The solr logs show nothing unusual. The exception does show up in the
> > catalina.out log file though. Does this happen to anyone else? Here is
> > the basic error and I have attached the crash dump file also. Our total
> > uptime on these boxes is over a year now BTW.****
> >
> > ** **
> >
> > #****
> >
> > # A fatal error has been detected by the Java Runtime Environment:****
> >
> > #****
> >
> > # SIGSEGV (0xb) at pc=0x00002b5379346612, pid=13724,
> > tid=1082353984****
> >
> > #****
> >
> > # JRE version: 6.0_25-b06****
> >
> > # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.0-b11 mixed mode
> > linux-amd64 )****
> >
> > # Problematic frame:****
> >
> > # V [libjvm.so+0x3c4612] Par_ConcMarkingClosure::trim_queue(unsigned
> > long)+0x82****
> >
> > #****
> >
> > # An error report file with more information is saved as:****
> >
> > # /var/LucidWorks/lucidworks/hs_err_pid13724.log****
> >
> > #****
> >
> > # If you would like to submit a bug report, please visit:****
> >
> > # http://java.sun.com/webapps/bugreport/crash.jsp****
> >
> > #****
> >
> > ** **
> >
> > VM Arguments:****
> >
> > jvm_args:
> > -Djava.util.logging.config.file=/var/LucidWorks/lucidworks/tomcat/conf
> > /logging.properties
> > -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> > -Xmx32768m -Xms32768m -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode
> > -Dcom.sun.management.jmxremote
> > -Dcom.sun.management.jmxremote.ssl=false
> > -Dcom.sun.management.jmxremote.authenticate=false
> > -Dcom.sun.management.jmxremote.port=6060
> > -Djava.endorsed.dirs=/var/LucidWorks/lucidworks/tomcat/endorsed
> > -Dcatalina.base=/var/LucidWorks/lucidworks/tomcat
> > -Dcatalina.home=/var/LucidWorks/lucidworks/tomcat
> > -Djava.io.tmpdir=/var/LucidWorks/lucidworks/tomcat/temp ****
> >
> > java_command: org.apache.catalina.startup.Bootstrap -server
> > -Dsolr.solr.home=lucidworks/solr start****
> >
> > Launcher Type: SUN_STANDARD****
> >
> > ** **
> >
> > Stack: [0x0000000000000000,0x0000000000000000],
> > sp=0x0000000040835eb0, free space=1056983k****
> >
> > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
> > C=native
> > code)****
> >
> > V [libjvm.so+0x3c4612] Par_ConcMarkingClosure::trim_queue(unsigned
> > long)+0x82****
> >
> > V [libjvm.so+0x3c481a]
> > CMSConcMarkingTask::do_work_steal(int)+0xfa****
> >
> > V [libjvm.so+0x3c3dcf] CMSConcMarkingTask::work(int)+0xef****
> >
> > V [libjvm.so+0x8783dc] YieldingFlexibleGangWorker::loop()+0xbc****
> >
> > V [libjvm.so+0x8755b4] GangWorker::run()+0x24****
> >
> > V [libjvm.so+0x71096f] java_start(Thread*)+0x13f****
> >
> > ** **
> >
> > Heap****
> >
> > par new generation total 345024K, used 180672K [0x00002aaaae120000,
> > 0x00002aaac5780000, 0x00002aaac5780000)****
> >
> > eden space 306688K, 53% used [0x00002aaaae120000,
> > 0x00002aaab8243c28,
> > 0x00002aaac0ca0000)****
> >
> > from space 38336K, 40% used [0x00002aaac3210000,
> > 0x00002aaac415c3f8,
> > 0x00002aaac5780000)****
> >
> > to space 38336K, 0% used [0x00002aaac0ca0000, 0x00002aaac0ca0000,
> > 0x00002aaac3210000)****
> >
> > concurrent mark-sweep generation total 33171072K, used 12144213K
> > [0x00002aaac5780000, 0x00002ab2ae120000, 0x00002ab2ae120000)****
> >
> > concurrent-mark-sweep perm gen total 83968K, used 50650K
> > [0x00002ab2ae120000, 0x00002ab2b3320000, 0x00002ab2b3320000)****
> >
> > ** **
> >
> > Code Cache [0x00002aaaab054000, 0x00002aaaab9a4000,
> > 0x00002aaaae054000)**
> > **
> >
> > total_blobs=2800 nmethods=2273 adapters=480 free_code_cache=40752512
> > largest_free_block=15808****
> >
> > ** **
> >
> > ** **
> >
> > ** **
> >
> > Thanks,****
> >
> > ** **
> >
> > *Robert (Robi) Petersen*
> >
> > Senior Software Engineer****
> >
> > Search Department****
> >
> > ** **
> >
>
>
RE: occasional GC crashes
Posted by "Petersen, Robert" <ro...@buy.com>.
Hi Otis,
I thought Java 7 had a bug which wasn't being addressed by Oracle which was making it not suitable for Solr. Did that get fixed now?
http://searchhub.org/2011/07/28/dont-use-java-7-for-anything/
I did see this but it doesn't really mention the bug: http://opensearchnews.com/2012/04/announcing-java7-support-with-apache-solr-and-lucene/
Thanks
Robi
-----Original Message-----
From: Otis Gospodnetic [mailto:otis.gospodnetic@gmail.com]
Sent: Tuesday, December 18, 2012 5:25 PM
To: solr-user@lucene.apache.org
Subject: Re: occasional GC crashes
Robert,
Step 1 is to get the latest Java 7 or if you have to remain on 6 then use the latest 6.
Otis
--
SOLR Performance Monitoring - http://sematext.com/spm On Dec 18, 2012 7:54 PM, "Petersen, Robert" <ro...@buy.com> wrote:
> Hi solr user group,****
>
> ** **
>
> Sorry if this isn't directly a Solr question. Seems like once in a
> blue moon the GC crashes on a server in our Solr 3.6.1 slave farm.
> This seems to only happen on a couple of the twelve slaves we have
> deployed and only very rarely on those. It seems like this doesn't
> directly affect solr because in the logs it looks like solr keeps
> working after the time of the exception but our external monitoring
> tool reports that the solr service is down so our operations department restarts solr on that box and alerts me.
> The solr logs show nothing unusual. The exception does show up in the
> catalina.out log file though. Does this happen to anyone else? Here is
> the basic error and I have attached the crash dump file also. Our total
> uptime on these boxes is over a year now BTW.****
>
> ** **
>
> #****
>
> # A fatal error has been detected by the Java Runtime Environment:****
>
> #****
>
> # SIGSEGV (0xb) at pc=0x00002b5379346612, pid=13724,
> tid=1082353984****
>
> #****
>
> # JRE version: 6.0_25-b06****
>
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.0-b11 mixed mode
> linux-amd64 )****
>
> # Problematic frame:****
>
> # V [libjvm.so+0x3c4612] Par_ConcMarkingClosure::trim_queue(unsigned
> long)+0x82****
>
> #****
>
> # An error report file with more information is saved as:****
>
> # /var/LucidWorks/lucidworks/hs_err_pid13724.log****
>
> #****
>
> # If you would like to submit a bug report, please visit:****
>
> # http://java.sun.com/webapps/bugreport/crash.jsp****
>
> #****
>
> ** **
>
> VM Arguments:****
>
> jvm_args:
> -Djava.util.logging.config.file=/var/LucidWorks/lucidworks/tomcat/conf
> /logging.properties
> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> -Xmx32768m -Xms32768m -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode
> -Dcom.sun.management.jmxremote
> -Dcom.sun.management.jmxremote.ssl=false
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.port=6060
> -Djava.endorsed.dirs=/var/LucidWorks/lucidworks/tomcat/endorsed
> -Dcatalina.base=/var/LucidWorks/lucidworks/tomcat
> -Dcatalina.home=/var/LucidWorks/lucidworks/tomcat
> -Djava.io.tmpdir=/var/LucidWorks/lucidworks/tomcat/temp ****
>
> java_command: org.apache.catalina.startup.Bootstrap -server
> -Dsolr.solr.home=lucidworks/solr start****
>
> Launcher Type: SUN_STANDARD****
>
> ** **
>
> Stack: [0x0000000000000000,0x0000000000000000],
> sp=0x0000000040835eb0, free space=1056983k****
>
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
> C=native
> code)****
>
> V [libjvm.so+0x3c4612] Par_ConcMarkingClosure::trim_queue(unsigned
> long)+0x82****
>
> V [libjvm.so+0x3c481a]
> CMSConcMarkingTask::do_work_steal(int)+0xfa****
>
> V [libjvm.so+0x3c3dcf] CMSConcMarkingTask::work(int)+0xef****
>
> V [libjvm.so+0x8783dc] YieldingFlexibleGangWorker::loop()+0xbc****
>
> V [libjvm.so+0x8755b4] GangWorker::run()+0x24****
>
> V [libjvm.so+0x71096f] java_start(Thread*)+0x13f****
>
> ** **
>
> Heap****
>
> par new generation total 345024K, used 180672K [0x00002aaaae120000,
> 0x00002aaac5780000, 0x00002aaac5780000)****
>
> eden space 306688K, 53% used [0x00002aaaae120000,
> 0x00002aaab8243c28,
> 0x00002aaac0ca0000)****
>
> from space 38336K, 40% used [0x00002aaac3210000,
> 0x00002aaac415c3f8,
> 0x00002aaac5780000)****
>
> to space 38336K, 0% used [0x00002aaac0ca0000, 0x00002aaac0ca0000,
> 0x00002aaac3210000)****
>
> concurrent mark-sweep generation total 33171072K, used 12144213K
> [0x00002aaac5780000, 0x00002ab2ae120000, 0x00002ab2ae120000)****
>
> concurrent-mark-sweep perm gen total 83968K, used 50650K
> [0x00002ab2ae120000, 0x00002ab2b3320000, 0x00002ab2b3320000)****
>
> ** **
>
> Code Cache [0x00002aaaab054000, 0x00002aaaab9a4000,
> 0x00002aaaae054000)**
> **
>
> total_blobs=2800 nmethods=2273 adapters=480 free_code_cache=40752512
> largest_free_block=15808****
>
> ** **
>
> ** **
>
> ** **
>
> Thanks,****
>
> ** **
>
> *Robert (Robi) Petersen*
>
> Senior Software Engineer****
>
> Search Department****
>
> ** **
>