Apache solr queries are taking time to fetch results

[Nitin Midas <mi...@gmail.com>]

Hello,

We have Apache Solr version 6.2.1 installed on server and we are getting
this warning on Apache Solr log from few days which has affected
performance of solr queries and put latency on our App:

SolrCore [user_details] PERFORMANCE WARNING: Overlapping onDeckSearchers=2

and also org.apache.solr.handler.component.QueryComponent.process is taking
time on our server when returning data from Apache solr queries.

Can you please help us to fix it.

Thank You,
Nitin.

Re: Apache solr queries are taking time to fetch results

[Shawn Heisey <ap...@elyograg.org>]

On 11/7/2019 2:33 AM, Nitin Midas wrote:
> We have Apache Solr version 6.2.1 installed on server and we are getting 
> this warning on Apache Solr log from few days which has affected 
> performance of solr queries and put latency on our App:
> 
> SolrCore [user_details] PERFORMANCE WARNING: Overlapping onDeckSearchers=2

This is the wrong mailing list for your question.  It belongs on 
solr-user, not dev.  This mailing list is for discussions around the 
development/code of Lucene and Solr.

That message means that you are doing commits, those that open a new 
searcher, too frequently.  This has resulted in a situation where you 
have more than one searcher warming up at the same time.  That is the 
source of your performance woes.

The solution is to do commits less frequently.

The default configurations that Solr ships with include an autoCommit 
section that it set to a maxTime of 15 seconds and has openSearcher set 
to false.  That default configuration will NOT cause this message to be 
logged.

The configuration can also have an autoSoftCommit setting, which *will* 
open a new searcher, and thus could cause that message to be logged if 
its intervals are too frequent.

Commit operations that are submitted manually or by an indexing program 
will normally have openSearcher set to true (which is the default 
setting) and could cause the same problem.

Solr ships with autoSoftCommit disabled ... but depending on exactly how 
an index is created, it MIGHT automatically have autoSoftCommit enabled 
with a 3 second maxTime.  I think that three seconds is far too short an 
interval for most indexes.  It doesn't take much for warming a new 
searcher to take more than 3 seconds.

Here's a blog post that provides some useful information:

https://lucidworks.com/post/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/

Note that this page says a soft commit is less expensive than a hard 
commit.  This is only *sometimes* true.  There are situations that can 
make a soft commit just as expensive as a hard commit.  And it's also 
worthwhile to note that a hard commit with openSearcher=false is NOT 
more expensive than a soft commit.  Opening a new searcher is the 
expensive part, so a hard commit with openSearcher=false is far less 
expensive than a soft commit.

Here's the wiki FAQ page on the particular log message you mentioned:

https://cwiki.apache.org/confluence/display/solr/FAQ#FAQ-Whatdoes%22PERFORMANCEWARNING:OverlappingonDeckSearchers=X%22meaninmylogs?

Thanks,
Shawn

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org