Facet count cache (Solr)

[Michael Gibney <mi...@michaelgibney.net>]

I'm interested in caching facet counts, keyed to query/DocSet domain (like
filterCache, but caching facet counts instead of DocSets).

Trying not to open a duplicate issue, I found SOLR-1617
<https://issues.apache.org/jira/browse/SOLR-1617>, but I wasn't clear on
whether it's applicable, or whether it's about maintaining per-segment UIF
data structures (which could be considered a type of cache).

I'd appreciate any guidance/suggestions on how to proceed -- revive
SOLR-1617, or open a new issue; thanks!

Michael

ps- A version of the functionality I'm interested in is implemented and
folded in to SOLR-13132 <https://issues.apache.org/jira/browse/SOLR-13132> (as
a prerequisite for performance on that issue). But I think I may have
buried the lead by including it there, since sort-by-relatedness is a
relatively specialized use case, and a facet cache is more generally
useful. The cache as implemented for SOLR-13132 is compatible across
SimpleFacets and JSON facets, and is segment-aware (so should be
NRT-friendly). It's particularly useful for high-cardinality fields and
high-cardinality DocSet domains (e.g., facets on a main landing page).

Re: Facet count cache (Solr)

[Michael Gibney <mi...@michaelgibney.net>]

Thank you, David! I'll take another look at the existing per-segment caches
and hopefully create a new issue soon.
Michael

On Wed, Jul 24, 2019 at 11:54 PM David Smiley <da...@gmail.com>
wrote:

> I think a new issue is appropriate, not the old one SOLR-1617 that feels
> like the reporter's wishful TODO that is underspecified.
>
> FYI on your implementation consider that Solr has some existing
> per-segment caches; one somewhere in the joins perSegment blah blah blah,
> and another inside RptWithGeometrySpatialField.  It's a bit awkward; I wish
> Solr supported per-segment caches better as a first class notion.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Wed, Jul 24, 2019 at 9:57 AM Michael Gibney <mi...@michaelgibney.net>
> wrote:
>
>> I'm interested in caching facet counts, keyed to query/DocSet domain
>> (like filterCache, but caching facet counts instead of DocSets).
>>
>> Trying not to open a duplicate issue, I found SOLR-1617
>> <https://issues.apache.org/jira/browse/SOLR-1617>, but I wasn't clear on
>> whether it's applicable, or whether it's about maintaining per-segment UIF
>> data structures (which could be considered a type of cache).
>>
>> I'd appreciate any guidance/suggestions on how to proceed -- revive
>> SOLR-1617, or open a new issue; thanks!
>>
>> Michael
>>
>> ps- A version of the functionality I'm interested in is implemented and
>> folded in to SOLR-13132
>> <https://issues.apache.org/jira/browse/SOLR-13132> (as a prerequisite
>> for performance on that issue). But I think I may have buried the lead by
>> including it there, since sort-by-relatedness is a relatively specialized
>> use case, and a facet cache is more generally useful. The cache as
>> implemented for SOLR-13132 is compatible across SimpleFacets and JSON
>> facets, and is segment-aware (so should be NRT-friendly). It's particularly
>> useful for high-cardinality fields and high-cardinality DocSet domains
>> (e.g., facets on a main landing page).
>>
>

Re: Facet count cache (Solr)

[David Smiley <da...@gmail.com>]

I think a new issue is appropriate, not the old one SOLR-1617 that feels
like the reporter's wishful TODO that is underspecified.

FYI on your implementation consider that Solr has some existing per-segment
caches; one somewhere in the joins perSegment blah blah blah, and another
inside RptWithGeometrySpatialField.  It's a bit awkward; I wish Solr
supported per-segment caches better as a first class notion.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Wed, Jul 24, 2019 at 9:57 AM Michael Gibney <mi...@michaelgibney.net>
wrote:

> I'm interested in caching facet counts, keyed to query/DocSet domain (like
> filterCache, but caching facet counts instead of DocSets).
>
> Trying not to open a duplicate issue, I found SOLR-1617
> <https://issues.apache.org/jira/browse/SOLR-1617>, but I wasn't clear on
> whether it's applicable, or whether it's about maintaining per-segment UIF
> data structures (which could be considered a type of cache).
>
> I'd appreciate any guidance/suggestions on how to proceed -- revive
> SOLR-1617, or open a new issue; thanks!
>
> Michael
>
> ps- A version of the functionality I'm interested in is implemented and
> folded in to SOLR-13132 <https://issues.apache.org/jira/browse/SOLR-13132> (as
> a prerequisite for performance on that issue). But I think I may have
> buried the lead by including it there, since sort-by-relatedness is a
> relatively specialized use case, and a facet cache is more generally
> useful. The cache as implemented for SOLR-13132 is compatible across
> SimpleFacets and JSON facets, and is segment-aware (so should be
> NRT-friendly). It's particularly useful for high-cardinality fields and
> high-cardinality DocSet domains (e.g., facets on a main landing page).
>