Apache Solr warning on 6.2.1

[Nitin Midas <mi...@gmail.com>]

Hello,

We have Apache Solr version 6.2.1 installed on server and we are getting
this warning on Apache Solr log from few days which has affected
performance of solr queries and put latency on our App:

SolrCore [user_details] PERFORMANCE WARNING: Overlapping onDeckSearchers=2

So we have followed this article
https://support.datastax.com/hc/en-us/articles/207690673-FAQ-Solr-logging-PERFORMANCE-WARNING-Overlapping-onDeckSearchers-and-its-meaning
and made changes in SolrConfig.xml file of user_details like this:

<maxWarmingSearchers>16</maxWarmingSearchers>

and also we have reduced number of autowarmCount

<cache name="perSegFilter"
           class="solr.search.LRUCache"
           size="10"
           initialSize="0"
           autowarmCount="5"
           regenerator="solr.NoOpRegenerator" />

however still we are getting this warning. Can you please help us how can
we improve the performance of solr queries on our app.

Regards,
Nitin.

Re: Apache Solr warning on 6.2.1

[Erick Erickson <er...@gmail.com>]

The warning isn’t what’s affecting performance, it’s just an indication that you’re committing too often. Technically you’re opening searchers too often.

Searchers are opened for several reasons;
1> your autocommit with openSearcher=true interval expires
2> your soft commit interval expires
3> you issue commits from some client

<1> and <2> are simple configuration changes. Make these intervals as long as you can tolerate, minutes if possible.

You should avoid <3> all together. It’s a common mistake to do horrible things like send a commit  from the indexing client after every batch add for instance. It’s usually far better (and easier to explain) to rely on your autocommit settings in solrconfig.xml and _not_ commit from the client. I’ll bend a little on this if you can absolutely _guarantee_ that the client commits once and only  once at the very end of the indexing run and there’s only one client.

You can specify “commitWithin” when the client is adding docs if you want to to insure that the docs are visible in some specified time frame, but even that is unnecessary if you rely on your solrconfig settings.

And if, for some reason, you _do_ require very short intervals between opening searchers, just set your cache sizes to zero since they aren’t doing you much good anyway (see your hit ratios).

Best,
Erick

> On Jul 1, 2019, at 2:12 AM, Nitin Midas <mi...@gmail.com> wrote:
> 
> Hello,
> 
> We have Apache Solr version 6.2.1 installed on server and we are getting
> this warning on Apache Solr log from few days which has affected
> performance of solr queries and put latency on our App:
> 
> SolrCore [user_details] PERFORMANCE WARNING: Overlapping onDeckSearchers=2
> 
> So we have followed this article
> https://support.datastax.com/hc/en-us/articles/207690673-FAQ-Solr-logging-PERFORMANCE-WARNING-Overlapping-onDeckSearchers-and-its-meaning
> and made changes in SolrConfig.xml file of user_details like this:
> 
> <maxWarmingSearchers>16</maxWarmingSearchers>
> 
> and also we have reduced number of autowarmCount
> 
> <cache name="perSegFilter"
>           class="solr.search.LRUCache"
>           size="10"
>           initialSize="0"
>           autowarmCount="5"
>           regenerator="solr.NoOpRegenerator" />
> 
> however still we are getting this warning. Can you please help us how can
> we improve the performance of solr queries on our app.
> 
> Regards,
> Nitin.