Is there way to autowarm new searcher using recently ran queries

[Pushkar Raste <pu...@gmail.com>]

Hi,

A rookie question. We have a Solr cluster that doesn't get too much
traffic. We see that our queries take long time unless we run a script to
send more traffic to Solr.

We are indexing data all the time and use autoCommit.

I am wondering if there is a way to warmup new searcher on commit by
rerunning queries processed by the last searcher. May be it happens by
default but then I can't understand why we see high query times if those
searchers are being warmed.

Re: Is there way to autowarm new searcher using recently ran queries

[Joel Bernstein <jo...@gmail.com>]

Typically what you would do is add static warming queries to warm all the
caches. These queries are hardcoded into the solrconfig.xml. You'll want to
run the facets you're using in the warming queries particularly facets on
string fields.

Once you add these it will take longer to warm the new searcher so you may
need to change the auto-commit intervals.




Joel Bernstein
http://joelsolr.blogspot.com/


On Wed, Jan 27, 2021 at 5:30 PM Pushkar Raste <pu...@gmail.com>
wrote:

> Hi,
>
> A rookie question. We have a Solr cluster that doesn't get too much
> traffic. We see that our queries take long time unless we run a script to
> send more traffic to Solr.
>
> We are indexing data all the time and use autoCommit.
>
> I am wondering if there is a way to warmup new searcher on commit by
> rerunning queries processed by the last searcher. May be it happens by
> default but then I can't understand why we see high query times if those
> searchers are being warmed.
>

Re: Is there way to autowarm new searcher using recently ran queries

[Chris Hostetter <ho...@fucit.org>]

: I am wondering if there is a way to warmup new searcher on commit by
: rerunning queries processed by the last searcher. May be it happens by
: default but then I can't understand why we see high query times if those
: searchers are being warmed.

it only happens by default if you have an 'autowarmCount' enabled for each 
cache...

https://lucene.apache.org/solr/guide/8_7/query-settings-in-solrconfig.html#caches

But note that this warms the caches *individually* --it doesn't re-simular 
a "full request" so some things (like stored fields) may still be "cold" 
on disk.

This typically isn't a problem -- eccept for people relying on FieldCache 
-- which is a query tie "un-inversion" of fields for sorting/faceting -- 
and has no explicit solr configuration or warming.

for that you have to use soemthing like joel described -- static 
'newSearcher' QuerySenderListenr queries that will sort/facet on those 
fields....

https://lucene.apache.org/solr/guide/8_7/query-settings-in-solrconfig.html#query-related-listeners

...but a better solution is to make sure you use DocValues on these fields 
instead.




-Hoss
http://www.lucidworks.com/