Solr / Tomcat bottleneck while parsing http headers

[Christophe Fondacci <ch...@nextep-softwares.com>]

Hello all,

I've searched the web and this forum without finding any answer to the
problem I have...

So here it is :

Our application performs queries to solr.
Our application is deployed on a Tomcat 6.0.14 on machine A.
Solr is deployed on a dedicated Tomcat 6.0.14 server running on a distinct
machine B.
We are using JDK 1.6.0_02-b05.

We perform queries to solr in 2 ways :
1. Performing a http get with an URL containing our query (We use the
HttpClient / GetMethod from apache commons).
URL is like :
"http://xxxx:8080/solr/select/?q=cityuri%3AXEABDBFDDACCXglasgowXEABDBFDDACCX&facet=true&facet.field=price&fl=id&facet.sort=false&facet.mincount=1&facet.limit=-1")

2. Performing a http post on this URL : http://xxxx:8080/solr/select/ and a
set of 12 NameValuePair.


Everything works fine when the server load is light...
When the server load is heavy, we observe a serious bottleneck on solr
responses. We added logs to monitor the time of the remote call to solr, and
sometimes we get response times of hundreds of seconds (!). The average of
those peaks is 200-300s. It happens randomly on both methods we use to query
solr. 

We performed deep jvm analysis (using jprofiler) to observe what was going
on on the solr server. When the problem occurs, we can see threads which are
stucked with the following stack trace :

at java.net.SocketInputStream.socketRead0(Native Method) 
at java.net.SocketInputStream.read(SocketInputStream.java:129) 
at
org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:700) 
at
org.apache.coyote.http11.InternalInputBuffer.parseRequestLine(InternalInputBuffer.java:366) 
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:805) 
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:584) 
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) 
at java.lang.Thread.run(Thread.java:619) 


The problem might be related to tomcat... 
Does anyone know what is going on ? 
Or if solr hold some "locks" which may affect Tomcat connector ?
May it be related to solr configuration ? I tried to switch the
useColdSearcher parameter to true to avoid waiting threads. Using most of
the default solr settings. We do use lazy fields.

If it helps I can publish our solrconfig.xml file...

Any help / clue / suggestions would be highly appreciated.
Thank you.

Christophe

-- 
View this message in context: http://www.nabble.com/Solr---Tomcat-bottleneck-while-parsing-http-headers-tp18214387p18214387.html
Sent from the Solr - User mailing list archive at Nabble.com.