Re: Random IllegalStateExceptions on Solr slave (3.6.1)

[Erick Erickson <er...@gmail.com>]

My first guess would be that your tomcat container timeouts need to be
lengthened, but that's mostly a guess based on the socket timeout
error message. Not sure where in Tomcat that needs to be configured
though...

Best
Erick

On Tue, Apr 30, 2013 at 12:37 PM, Arun Rangarajan
<ar...@gmail.com> wrote:
> We have a master-slave Solr set up and run live queries only against the
> slave. Full import (with optimize) happens on master every day at 2 a.m.
> Delta imports happen every 10 min for one entity and every hour for another
> entity.
>
> The following exceptions occur a few times every day in our app logs:
>
> ...
> org.apache.solr.client.solrj.SolrServerException: Error executing query
>         at
> org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:95)
>         at
> org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:118)
> ...
>
> and this one:
> ...
> org.apache.solr.client.solrj.SolrServerException: java.net.SocketException:
> Connection reset
>         at
> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:478)
>         at
> org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:244)
>         at
> org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:89)
>         at
> org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:118)
>         ...
>
> These happen on different queries at different times, so they are not
> query-dependent.
>
> If I inspect the localhost.log file in tomcat on the solr slave server, the
> following exception occurs at the same time:
>
> Apr 06, 2013 7:16:33 AM org.apache.catalina.core.StandardWrapperValve invoke
> SEVERE: Servlet.service() for servlet default threw exception
> java.lang.IllegalStateException
>     at
> org.apache.catalina.connector.ResponseFacade.sendError(ResponseFacade.java:407)
>     at
> org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:389)
>     at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:291)
>     at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>     at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>     at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>     at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>     at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>     at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>     at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>     at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
>     at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
>     at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
>     at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
>     at java.lang.Thread.run(Thread.java:722)
> ...
>
>
> The replication set-up is as follows:
>
> <requestHandler name="/replication" class="solr.ReplicationHandler" >
>       <lst name="master">
>         <str name="enable">${enable.master:false}</str>
>         <str name="replicateAfter">startup</str>
>         <str name="replicateAfter">commit</str>
>         <str name="replicateAfter">optimize</str>
>         <str
> name="confFiles">solrconfig.xml,data-config.xml,schema.xml,stopwords.txt,synonyms.txt,elevate.xml</str>
>       </lst>
>       <lst name="slave">
>         <str name="enable">${enable.slave:false}</str>
>         <str name="masterUrl">http://${master.ip}:${master.port}/solr/${
> solr.core.name}/replication</str>
>         <str name="pollInterval">00:01:00</str>
>       </lst>
>   </requestHandler>
>
> Aside from the full and delta imports, we also have external file fields
> which are loaded every 1 hour with reloadCache on both master and slave.
>
> What is causing these exceptions and how do I fix it?
>
> Thanks.