[jira] [Updated] (SOLR-13892) Add postfilter support to {!join} queries

["Jason Gerlowski (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SOLR-13892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Gerlowski updated SOLR-13892:
-----------------------------------
    Attachment: SOLR-13892.patch
        Status: Open  (was: Open)

Attaching a patch with a first pass at this:

*Design*
The bulk of this code lives in PostFIlterJoinQuery.  On post-filter creation, we run the {{from}} query using a Collector implementation that populates a terms bitset ({{fromOrdBitSet}}) as each {{from}} match is found.  We then build a corresponding bitset ({{toOrdBitSet}}) for the {{to}} field by iterating through the ordinals set in {{fromOrdBitSet}} and checking whether each set term exists in the {{to}} field.  At post-filter execution time, the postfilter just looks at the individual doc and checks whether it's value in the {{to}} field matches ones of the bits set in {{toOrdBitSet}}.

In some test scenarios this increases join performance up to 5x over standard execution.  Since bitsets are used, the postfilter scales well as {{from}} query matches increase.  Like most post-filter implementations, it starts to get slower than standard execution as the {{to}} query results increase (since postfilters are invoked on each doc matched, this is to be expected).


*Current Limitations*
* only possible on fields with docValues
* only tested with string fields.
* currently only supports single-value fields.

*TODO*
* Tests
* Documentation
* Maybe merge PostFilterJoinQuery in with the existing JoinQuery implementation.
* add support for multivalue fields.

> Add postfilter support to {!join} queries
> -----------------------------------------
>
>                 Key: SOLR-13892
>                 URL: https://issues.apache.org/jira/browse/SOLR-13892
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: query parsers
>    Affects Versions: master (9.0)
>            Reporter: Jason Gerlowski
>            Priority: Major
>         Attachments: SOLR-13892.patch
>
>
> The JoinQParserPlugin would be a lot performant in many use-cases if it could operate as a post-filter, especially when doc-values for the involved fields are available.
> With this issue, I'd like to propose a post-filter implementation for the {{join}} qparser.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org