You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Rallavagu <ra...@gmail.com> on 2015/10/01 19:50:25 UTC
PoolingClientConnectionManager
Solr 4.6.1, single Shard, cloud with 4 nodes
Solr is running on Tomcat configured with 200 threads for thread pool.
As Solr uses "org.apache.http.impl.conn.PoolingClientConnectionManager"
for replication, my question is does Solr threads use connections from
tomcat thread pool or they create their own thread pool? I am trying to
find out if it would be 200 + Solr threads or not. Thanks.
Re: PoolingClientConnectionManager
Posted by Rallavagu <ra...@gmail.com>.
Thanks Shawn. This is good data.
On 10/1/15 11:43 AM, Shawn Heisey wrote:
> On 10/1/2015 11:50 AM, Rallavagu wrote:
>> Solr 4.6.1, single Shard, cloud with 4 nodes
>>
>> Solr is running on Tomcat configured with 200 threads for thread pool.
>> As Solr uses
>> "org.apache.http.impl.conn.PoolingClientConnectionManager" for
>> replication, my question is does Solr threads use connections from
>> tomcat thread pool or they create their own thread pool? I am trying
>> to find out if it would be 200 + Solr threads or not. Thanks.
>
> I don't know the answer to the actual question you have asked ... but I
> do know that keeping the container maxThreads at 200 can cause serious
> problems for Solr. It does not take a very big installation to exceed
> 200 threads, and users have had problems fixed by increasing
> maxThreads. This implies that the container is able to control the
> threads in Solr to some degree.
>
> The Jetty included with all versions of Solr that I have actually
> checked (back to 3.2.0) has maxThreads set to 10000, which effectively
> removes the thread limit for any typical install. Very large installs
> might need it bumped higher than 10000.
>
> Thanks,
> Shawn
>
Re: PoolingClientConnectionManager
Posted by Shawn Heisey <ap...@elyograg.org>.
On 10/1/2015 11:50 AM, Rallavagu wrote:
> Solr 4.6.1, single Shard, cloud with 4 nodes
>
> Solr is running on Tomcat configured with 200 threads for thread pool.
> As Solr uses
> "org.apache.http.impl.conn.PoolingClientConnectionManager" for
> replication, my question is does Solr threads use connections from
> tomcat thread pool or they create their own thread pool? I am trying
> to find out if it would be 200 + Solr threads or not. Thanks.
I don't know the answer to the actual question you have asked ... but I
do know that keeping the container maxThreads at 200 can cause serious
problems for Solr. It does not take a very big installation to exceed
200 threads, and users have had problems fixed by increasing
maxThreads. This implies that the container is able to control the
threads in Solr to some degree.
The Jetty included with all versions of Solr that I have actually
checked (back to 3.2.0) has maxThreads set to 10000, which effectively
removes the thread limit for any typical install. Very large installs
might need it bumped higher than 10000.
Thanks,
Shawn
Re: PoolingClientConnectionManager
Posted by Rallavagu <ra...@gmail.com>.
Awesome. This is what I was looking for. Will try these. Thanks.
On 10/1/15 1:31 PM, Shawn Heisey wrote:
> On 10/1/2015 12:39 PM, Rallavagu wrote:
>> Thanks for the response Andrea.
>>
>> Assuming that Solr has it's own thread pool, it appears that
>> "PoolingClientConnectionManager" has a maximum 20 threads per host as
>> default. Is there a way to changes this increase to handle heavy
>> update traffic? Thanks.
>
> You can configure all ShardHandler instances with the solr.xml file.
> The shard handler controls SolrJ (and HttpClient) within Solr.
>
> https://cwiki.apache.org/confluence/display/solr/Moving+to+the+New+solr.xml+Format
>
> That page does not go into all the shard handler options, though. For
> that, you need to look at the page for distributed requests ... but
> don't configure it in solrconfig.xml as the following link shows,
> configure it in solr.xml as shown by the earlier link.
>
> https://cwiki.apache.org/confluence/display/solr/Distributed+Requests#DistributedRequests-ConfiguringtheShardHandlerFactory
>
> Thanks,
> Shawn
>
Re: PoolingClientConnectionManager
Posted by Shawn Heisey <ap...@elyograg.org>.
On 10/1/2015 12:39 PM, Rallavagu wrote:
> Thanks for the response Andrea.
>
> Assuming that Solr has it's own thread pool, it appears that
> "PoolingClientConnectionManager" has a maximum 20 threads per host as
> default. Is there a way to changes this increase to handle heavy
> update traffic? Thanks.
You can configure all ShardHandler instances with the solr.xml file.
The shard handler controls SolrJ (and HttpClient) within Solr.
https://cwiki.apache.org/confluence/display/solr/Moving+to+the+New+solr.xml+Format
That page does not go into all the shard handler options, though. For
that, you need to look at the page for distributed requests ... but
don't configure it in solrconfig.xml as the following link shows,
configure it in solr.xml as shown by the earlier link.
https://cwiki.apache.org/confluence/display/solr/Distributed+Requests#DistributedRequests-ConfiguringtheShardHandlerFactory
Thanks,
Shawn
Re: PoolingClientConnectionManager
Posted by Rallavagu <ra...@gmail.com>.
Thanks for the response Andrea.
Assuming that Solr has it's own thread pool, it appears that
"PoolingClientConnectionManager" has a maximum 20 threads per host as
default. Is there a way to changes this increase to handle heavy update
traffic? Thanks.
On 10/1/15 11:05 AM, Andrea Gazzarini wrote:
> Hi,
> Maybe I could be wrong as your question is related with Solr internals (I
> believe the dev list is a better candidate for such questions).
>
> Anyway, my thoughts: unless you're within a JCA inbound component (and Solr
> isn't), the JEE specs say you shouldn' start new threads. For this reason,
> there's no a (standard) way to directly connect to and use the servlet
> container threads.
>
> As far as I know Solr 4.x is a standard and JEE compliant web application
> so the answer to your question *should* be: "yes, it is using its own
> threads"
>
> Best,
> Andrea
> Solr 4.6.1, single Shard, cloud with 4 nodes
>
> Solr is running on Tomcat configured with 200 threads for thread pool. As
> Solr uses "org.apache.http.impl.conn.PoolingClientConnectionManager" for
> replication, my question is does Solr threads use connections from tomcat
> thread pool or they create their own thread pool? I am trying to find out
> if it would be 200 + Solr threads or not. Thanks.
>
Re: PoolingClientConnectionManager
Posted by Andrea Gazzarini <a....@gmail.com>.
Hi,
Maybe I could be wrong as your question is related with Solr internals (I
believe the dev list is a better candidate for such questions).
Anyway, my thoughts: unless you're within a JCA inbound component (and Solr
isn't), the JEE specs say you shouldn' start new threads. For this reason,
there's no a (standard) way to directly connect to and use the servlet
container threads.
As far as I know Solr 4.x is a standard and JEE compliant web application
so the answer to your question *should* be: "yes, it is using its own
threads"
Best,
Andrea
Solr 4.6.1, single Shard, cloud with 4 nodes
Solr is running on Tomcat configured with 200 threads for thread pool. As
Solr uses "org.apache.http.impl.conn.PoolingClientConnectionManager" for
replication, my question is does Solr threads use connections from tomcat
thread pool or they create their own thread pool? I am trying to find out
if it would be 200 + Solr threads or not. Thanks.