FilterCache Tuning Question

["Sanders, Marshall (AT - Atlanta)" <Ma...@autotrader.com>]

Greetings,

We've successfully figured out how to tune AND queries to make effective use (re-use) of the filter cache, is there a way to do this with OR queries as well?  The way our data is structured I feel we could really benefit from it.  For example with AND queries instead of doing fq=field1:value1 AND field2:value2 we split them into fq=field1:value1&fq=field2:value2 which will generate 2 separate cache entries which can then be used by any subsequent queries which use either one, or both, or any combination with other fields/values as long as the same format is followed.

My question is regarding if there is a way to structure/configure an OR query where it would do something similar.

Currently if you have
fq=field1:value1 OR field2:value2

It generates a filter cache item with a key like this:
item_field1:value1 field2

Ideally it would create 2 additional ones like so (configured or on demand maybe?):
item_field1:value1 field2:value2
item_field1:value1
item_field2:value2

That way we can reuse existing cache keys that might be in the cache already and/or set up the cache for future queries.

Is there a way to specify/reformat the fq parameter to accomplish something along these lines?


Marshall Sanders
Technical Lead - Software Engineer
Autotrader.com
404-568-7130

Re: FilterCache Tuning Question

[Mikhail Khludnev <mk...@griddynamics.com>]

Hello,

See filter() in http://yonik.com/solr-5-4/
also
http://blog.griddynamics.com/2014/01/segmented-filter-cache-in-solr.html

On Tue, Sep 15, 2015 at 10:35 PM, Sanders, Marshall (AT - Atlanta) <
Marshall.Sanders@autotrader.com> wrote:

> Greetings,
>
> We've successfully figured out how to tune AND queries to make effective
> use (re-use) of the filter cache, is there a way to do this with OR queries
> as well?  The way our data is structured I feel we could really benefit
> from it.  For example with AND queries instead of doing fq=field1:value1
> AND field2:value2 we split them into fq=field1:value1&fq=field2:value2
> which will generate 2 separate cache entries which can then be used by any
> subsequent queries which use either one, or both, or any combination with
> other fields/values as long as the same format is followed.
>
> My question is regarding if there is a way to structure/configure an OR
> query where it would do something similar.
>
> Currently if you have
> fq=field1:value1 OR field2:value2
>
> It generates a filter cache item with a key like this:
> item_field1:value1 field2
>
> Ideally it would create 2 additional ones like so (configured or on demand
> maybe?):
> item_field1:value1 field2:value2
> item_field1:value1
> item_field2:value2
>
> That way we can reuse existing cache keys that might be in the cache
> already and/or set up the cache for future queries.
>
> Is there a way to specify/reformat the fq parameter to accomplish
> something along these lines?
>
>
> Marshall Sanders
> Technical Lead - Software Engineer
> Autotrader.com
> 404-568-7130
>
>


-- 
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics

<http://www.griddynamics.com>
<mk...@griddynamics.com>

Re: FilterCache Tuning Question

[Shawn Heisey <ap...@elyograg.org>]

On 9/15/2015 1:35 PM, Sanders, Marshall (AT - Atlanta) wrote:
> Currently if you have
> fq=field1:value1 OR field2:value2
>
> It generates a filter cache item with a key like this:
> item_field1:value1 field2
>
> Ideally it would create 2 additional ones like so (configured or on demand maybe?):
> item_field1:value1 field2:value2
> item_field1:value1
> item_field2:value2
>
> That way we can reuse existing cache keys that might be in the cache already and/or set up the cache for future queries.

There is at least one (and possibly more than one) issue in Jira for
this.  This is what I was able to find -- you'll see that I commented on
it quite a long time ago:

https://issues.apache.org/jira/browse/SOLR-1223

I had not ever seen the 5.4 filter() syntax that Mikhail mentioned, so
I'm completely unfamiliar with whether it will give you what you're
after.  It looks like it might do exactly what you need.

Thanks,
Shawn