[jira] [Commented] (LUCENE-8990) IndexOrDocValuesQuery can take a bad decision for range queries if field has many values per document

["Atri Sharma (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-8990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16938700#comment-16938700 ] 

Atri Sharma commented on LUCENE-8990:
-------------------------------------

+1, I think that is a good heuristic – strangely enough, I was thinking of this limitation for a similar problem.

 

Would it suffice if we just made PointRangeQuery also consider the BKDReader's docCount, in addition to pointCount? e.g. (cost = values.estimatePointCount() / values.estimateDocCount())?

> IndexOrDocValuesQuery can take a bad decision for range queries if field has many values per document
> -----------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-8990
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8990
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Ignacio Vera
>            Priority: Major
>
> Heuristics of IndexOrDocValuesQuery are somewhat inconsistent for range queries . The leadCost that is provided is based on number of documents, meanwhile the cost() of a range query is based on the number of points that potentially match the query. 
> Therefore it might happen that a BKD tree has millions of points but this points correspond to just a few documents. Therefore we can take the decision of executing the query using docValues and in fact we are almost scanning all the points.
> Maybe the cost() function for range queries need to take into account the average number of points per document in the tree and adjust the value accordingly.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org