You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by prathap <pr...@gmail.com> on 2017/12/29 11:31:18 UTC

With 100% CPU usage giving out of memory exception and solr is not responding

ERROR - 2017-12-21 08:39:13.326; org.apache.solr.common.SolrException;
null:java.lang.RuntimeException: java.lang.OutOfMemoryError: GC overhead
limit exceeded
	at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:793)
	at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:434)
	at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Unknown
Source)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(Unknown Source)
	at org.apache.catalina.core.StandardWrapperValve.invoke(Unknown Source)
	at org.apache.catalina.core.StandardContextValve.invoke(Unknown Source)
	at org.apache.catalina.core.StandardHostValve.invoke(Unknown Source)
	at org.apache.catalina.valves.ErrorReportValve.invoke(Unknown Source)
	at org.apache.catalina.valves.AccessLogValve.invoke(Unknown Source)
	at org.apache.catalina.core.StandardEngineValve.invoke(Unknown Source)
	at org.apache.catalina.connector.CoyoteAdapter.service(Unknown Source)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(Unknown Source)
	at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(Unknown
Source)
	at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(Unknown
Source)
	at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(Unknown
Source)
	at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(Unknown
Source)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

INFO  - 2017-12-21 08:43:43.351; org.apache.solr.core.SolrCore;
[SA_PROD_SPK_QC] webapp=/solr path=/select
params={q=430+Gl&fq=channel:(20052)&fq=languageId:(9)&rows=6&wt=xml&version=2.2}
hits=0 status=0 QTime=6732700 
ERROR - 2017-12-21 08:43:43.351; org.apache.solr.common.SolrException;
null:org.apache.catalina.connector.ClientAbortException: java.io.IOException
	at org.apache.catalina.connector.OutputBuffer.doFlush(Unknown Source)
	at org.apache.catalina.connector.OutputBuffer.flush(Unknown Source)
	at org.apache.catalina.connector.CoyoteOutputStream.flush(Unknown Source)
	at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:297)
	at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
	at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
	at org.apache.solr.util.FastWriter.flush(FastWriter.java:137)
	at
org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:766)
	at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:426)
	at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Unknown
Source)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(Unknown Source)
	at org.apache.catalina.core.StandardWrapperValve.invoke(Unknown Source)
	at org.apache.catalina.core.StandardContextValve.invoke(Unknown Source)
	at org.apache.catalina.core.StandardHostValve.invoke(Unknown Source)
	at org.apache.catalina.valves.ErrorReportValve.invoke(Unknown Source)
	at org.apache.catalina.valves.AccessLogValve.invoke(Unknown Source)
	at org.apache.catalina.core.StandardEngineValve.invoke(Unknown Source)
	at org.apache.catalina.connector.CoyoteAdapter.service(Unknown Source)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(Unknown Source)
	at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(Unknown
Source)
	at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(Unknown
Source)
	at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(Unknown
Source)
	at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(Unknown
Source)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException
	at org.apache.coyote.http11.InternalAprOutputBuffer.flushBuffer(Unknown
Source)
	at org.apache.coyote.http11.InternalAprOutputBuffer.flush(Unknown Source)
	at org.apache.coyote.http11.AbstractHttp11Processor.action(Unknown Source)
	at org.apache.coyote.Response.action(Unknown Source)
	... 27 more



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: With 100% CPU usage giving out of memory exception and solr is not responding

Posted by prathap <pr...@gmail.com>.
What is your Xmx?  - 20GB
* How many documents in your index? 12GB
* What is your filterCache size? 512 MB



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: With 100% CPU usage giving out of memory exception and solr is not responding

Posted by prathap <pr...@gmail.com>.
What is your Xmx? 20GB
* How many documents in your index? 12GB
* What is your filterCache size? 512 MB



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: With 100% CPU usage giving out of memory exception and solr is not responding

Posted by Toke Eskildsen <to...@kb.dk>.
prathap <pr...@gmail.com> wrote:
> ERROR - 2017-12-21 08:39:13.326; org.apache.solr.common.SolrException;
> null:java.lang.RuntimeException: java.lang.OutOfMemoryError: GC overhead
> limit exceeded
...
> INFO  - 2017-12-21 08:43:43.351; org.apache.solr.core.SolrCore;
> [SA_PROD_SPK_QC] webapp=/solr path=/select
> params={q=430+Gl&fq=channel:(20052)&fq=languageId:(9)&rows=6&wt=xml
> &version=2.2} hits=0 status=0 QTime=6732700

Seems like your heap is too small for what your are doing. Common culprits are a combination of many documents in the index and a large filterCache or faceting on a high-cardinality field.

If the query above is representative of your general queries, I'll guess it's the many docs + large filterCache one. It's fairly easy to check:
* What is your Xmx?
* How many documents in your index?
* What is your filterCache size?

- Toke Eskildsen