Autowarm and filtercache invalidation

[Jeff Wartes <jw...@whitepages.com>]

If I configure my filterCache like this:
<filterCache class="solr.FastLRUCache" size="512" initialSize="512"
autowarmCount="10"/>

and I have <= 10 distinct filter queries I ever use, does that mean I’ve
effectively disabled cache invalidation? So my cached filter query results
will never change? (short of JVM restart)

I’m unclear on whether autowarm simply copies the value into the new
searcher’s cache or whether it tries to rebuild the results of the cached
filter query based on the new searcher’s view of the data.

Re: Autowarm and filtercache invalidation

[Erick Erickson <er...@gmail.com>]

Jeff:

Yes, exactly. Otherwise the autowarming would be quite useless since
what's stored in the cache is the _lucene_ doc ID (either as a bitmap
or as a list of IDs). And the lucene doc ID can change when merging,
so the old IDs are useless.

Best,
Erick

On Thu, Sep 24, 2015 at 2:11 PM, Jeff Wartes <jw...@whitepages.com> wrote:

> Answering my own question: Looks like the default filterCache regenerator
> uses the old cache to re-executes queries in the context of the new
> searcher and does nothing with the old cache value.
>
> So, the new searcher’s cache contents will be consistent with that
> searcher’s view, regardless of whether it was populated via autowarm.
>
>
> On 9/24/15, 11:28 AM, "Jeff Wartes" <jw...@whitepages.com> wrote:
>
> >
> >If I configure my filterCache like this:
> ><filterCache class="solr.FastLRUCache" size="512" initialSize="512"
> >autowarmCount="10"/>
> >
> >and I have <= 10 distinct filter queries I ever use, does that mean I’ve
> >effectively disabled cache invalidation? So my cached filter query results
> >will never change? (short of JVM restart)
> >
> >I’m unclear on whether autowarm simply copies the value into the new
> >searcher’s cache or whether it tries to rebuild the results of the cached
> >filter query based on the new searcher’s view of the data.
> >
>
>

Re: Autowarm and filtercache invalidation

[Shawn Heisey <ap...@elyograg.org>]

On 9/24/2015 3:11 PM, Jeff Wartes wrote:
> Answering my own question: Looks like the default filterCache regenerator
> uses the old cache to re-executes queries in the context of the new
> searcher and does nothing with the old cache value.
> 
> So, the new searcher’s cache contents will be consistent with that
> searcher’s view, regardless of whether it was populated via autowarm.

That is how cache warming works in general.  The entries in the old
cache contain the query that was used to produce the cache entry.
During warming, the same query is executed on the new searcher to build
a new cache entry.

Thanks,
Shawn

Re: Autowarm and filtercache invalidation

[Jeff Wartes <jw...@whitepages.com>]

Answering my own question: Looks like the default filterCache regenerator
uses the old cache to re-executes queries in the context of the new
searcher and does nothing with the old cache value.

So, the new searcher’s cache contents will be consistent with that
searcher’s view, regardless of whether it was populated via autowarm.


On 9/24/15, 11:28 AM, "Jeff Wartes" <jw...@whitepages.com> wrote:

>
>If I configure my filterCache like this:
><filterCache class="solr.FastLRUCache" size="512" initialSize="512"
>autowarmCount="10"/>
>
>and I have <= 10 distinct filter queries I ever use, does that mean I’ve
>effectively disabled cache invalidation? So my cached filter query results
>will never change? (short of JVM restart)
>
>I’m unclear on whether autowarm simply copies the value into the new
>searcher’s cache or whether it tries to rebuild the results of the cached
>filter query based on the new searcher’s view of the data.
>