You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "jpountz (via GitHub)" <gi...@apache.org> on 2024/01/25 16:46:40 UTC
[PR] Early exit range queries on non-matching segments. [lucene]
jpountz opened a new pull request, #13033:
URL: https://github.com/apache/lucene/pull/13033
This change makes `PointRangeQuery` exit early when it knows that it doesn't match a segment. In the case when this query is part of a conjunction, this helps make sure `ScorerSupplier#get` doesn't get called on other required clauses, which is sometimes an expensive operation (e.g. multi-term queries).
This is especially relevant for time-based data combined with `LogByteSizePolicy`, which gives non-adjacent ranges of timestamps to each segment, which in-turn increases the likelihood that some segments won't match a range query at all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Early exit range queries on non-matching segments. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz merged PR #13033:
URL: https://github.com/apache/lucene/pull/13033
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Early exit range queries on non-matching segments. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #13033:
URL: https://github.com/apache/lucene/pull/13033#issuecomment-1911681276
> I am not 100% sure the expensive part we are trying to bypass
Typically it's browsing terms dictionaries and/or building bitsets across the entire term/doc ID space. So multi-term queries, term-in-set queries and a few more.
> we will call scorerSupplier.cost() on that "expensiveScorerSupplier". If this is fine
This is fine, `ScorerSupplier` was introduced in the first place to be able to do a quick estimation of the cost without building a scorer. This is then used in `IndexOrDocValuesQuery` to know whether the query will be used to drive iteration (in which case the index query is used), or just to validate matches (in which case the doc-values query is used).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Early exit range queries on non-matching segments. [lucene]
Posted by "benwtrent (via GitHub)" <gi...@apache.org>.
benwtrent commented on code in PR #13033:
URL: https://github.com/apache/lucene/pull/13033#discussion_r1466767966
##########
lucene/core/src/java/org/apache/lucene/search/PointRangeQuery.java:
##########
@@ -290,6 +290,24 @@ public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOExcepti
return null;
}
+ if (values.getDocCount() == 0) {
+ return null;
+ } else {
Review Comment:
From what I can tell, `ScorerSupplier` is an optional interface, which is probably just fine for most things (I am unsure which queries satisfy it or if we care).
I am not 100% sure the expensive part we are trying to bypass. But in a `BooleanQuery`, if we hit an "expensiveScorerSupplier" first, we will call `scorerSupplier.cost()` on that "expensiveScorerSupplier". If this is fine & still allows for the biggest optimizations (bypassing `scorerSupplier.get(leadCost)`), then this solution is rather nice!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org