You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "Chris M. Hostetter (Jira)" <ji...@apache.org> on 2025/05/06 18:17:00 UTC
[jira] [Updated] (SOLR-17736) Introduce support for KNN search on nested vector documents
[ https://issues.apache.org/jira/browse/SOLR-17736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris M. Hostetter updated SOLR-17736:
--------------------------------------
Attachment: SOLR-17736.patch
Status: Open (was: Open)
I haven't fully wrapped my head around everything in the linked GH PR#3316, but IIUC the "meat" of the idea is that:
* IF:
** a block join {{parent}} QParser is used
** AND that {{parent}} QParser directly wraps a (single) {{KnnXxxVectorQuery}}
* THEN:
** Extract some of the properties from the {{KnnXxxVectorQuery}} (field name, vector, topK, etc...)
** Ignore the original KNN query and replace it with a new {{DiversifyingChildrenXxxKnnVectorQuery}} query
(correct?)
To me, this approach seems kind of "kludgy" and brittle.
In my day job, we have a small plugin that we use with Solr which creates instances of {{DiversifyingChildren...}} queries via a simple subclass of the existing {{KnnQParserPlugin}} using a new {{childOf}} local params (modeled after the {{of}} param in Solr's {{child}} QParser)
This approach supports several usecases that (AFAICT) the current PR does not:
* Create {{DiversifyingChildrenXxxKnnVectorQuery}} instances even w/o a {{parent}} wrapper
** When you want to return diverse child docs w/o joining to the parent doc
* Wrap {{parent}} queries around BooleanQuery containing multiple clauses (one or more of which might be {{{}DiversifyingChildrenXxxKnnVectorQuery{}}})
** When you want the top scoring parents based on child scores using multiple vector queries, either because of multiple input vectors, or because of multiple vector fields.
** Or when you want to return parent docs based on *either* diverse topK children *OR* some other non-vector child criteria
* Wrap {{parent}} queries around plain {{KnnXxxVectorQuery}} against children, w/o using {{DiversifyingChildren...}}
** When you don't care about the overhead of diversification, perhaps because you know each parent has at most one child (of a particular type) with a vector
I'm attaching a patch that adapts my current custom plugin to re-implement it as a simple addition to the existing {{{}KnnQParserPlugin{}}}, that kicks in if and only if a {{childOf}} local param is specified.
To my mind this approach is a lot cleaner and more versatile then the " {{parent}} QParser wrapped around {{knn}} QParser should always throw away the original query and build it's own" approach in the PR – but if folks prefer the approach in PR#3316 then can we please at least make it configurable? (maybe via a new variant of the {{parent}} QParser?)
Because as it stands right now – since {{DiversifyingChildrenXxxxKnnVectorQuery}} extends {{KnnXxxVectorQuery}} – it will become impossible to support some of the use cases I listed above (even with a custom plugin) because the Solr {{parent}} QParser will start treating _any_ {{KnnXxxVectorQuery}} it wraps (including {{DiversifyingChildrenXxxxKnnVectorQuery}} created by a custom plugin) as special, throwing them away and creating it's own.
> Introduce support for KNN search on nested vector documents
> -----------------------------------------------------------
>
> Key: SOLR-17736
> URL: https://issues.apache.org/jira/browse/SOLR-17736
> Project: Solr
> Issue Type: New Feature
> Components: query
> Affects Versions: 9.8
> Reporter: Alessandro Benedetti
> Priority: Major
> Labels: pull-request-available
> Attachments: SOLR-17736.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> This issue tracks the work of introducing the support for KNN search on nested vector documents, surfacing the Lucene implementation in here:
> https://github.com/apache/lucene/pull/12434
> This allows both:
> -KNN retrieval of children, applying parent filters with no denormalisation needed
> -KNN retrieval of parents (based on children KNN, children level prefiltering and parent level prefiltering)
> It's one way of having multi-valued vectors per field, per document in Solr.
> More will come soon
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org