You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "jpountz (via GitHub)" <gi...@apache.org> on 2025/02/25 21:02:12 UTC

[PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

jpountz opened a new pull request, #14293:
URL: https://github.com/apache/lucene/pull/14293

   Even though there is a single clause, it often needs to be intersected, either with live docs or with the collector's competitive iterator.
   
   This uses `DenseConjunctionBulkScorer` for:
    - `MatchAllDocsQuery`,
    - `TermQuery` when scores are not needed and `docFreq >= maxDoc / 32`,
    - some constant-score queries including `PointRangeQuery` when `cost >= maxDoc / 32`.
   
   In addition,
    - `DenseConjunctionBulkScorer` was improved to stop collecting hits when the min competitive score is greater than the configured constant score,
    - Added test coverage to sorting tests to make sure that their competitive iterators are happy with how `DenseConjunctionBulkScorer` reads ahead.
   
   The downside of this change is that it forces the impacted queries to score at least 4096 (the window size) docs at once. So queries that can start skipping very early (and which are thus very fast) may see a slowdown (e.g. `TermMonthSort`). On the other hand, when it takes time before dynamic pruning becomes effective, there could be a speedup thanks to the more efficient intersection logic (e.g.  `TermDTSort`).
   
   This change should also help in the presence of deleted docs, by taking advantage of the more efficient way how deleted docs are applied in this bulk scorer.
   
   Closes #14283


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

Posted by "gf2121 (via GitHub)" <gi...@apache.org>.
gf2121 commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1972872831


##########
lucene/core/src/java/org/apache/lucene/search/TermQuery.java:
##########
@@ -165,6 +165,17 @@ public Scorer get(long leadCost) throws IOException {
           }
         }
 
+        @Override
+        public BulkScorer bulkScorer() throws IOException {
+          if (scoreMode.needsScores() == false) {
+            DocIdSetIterator iterator = get(Long.MAX_VALUE).iterator();
+            int maxDoc = context.reader().maxDoc();
+            return ConstantScoreScorerSupplier.fromIterator(iterator, 0f, scoreMode, maxDoc)

Review Comment:
   Is it correct to return `MatchAllScorerSupplier#bulkScorer()` when`getTermsEnum().docFreq() == maxDoc`?



##########
lucene/core/src/java/org/apache/lucene/search/PointRangeQuery.java:
##########
@@ -341,11 +341,10 @@ public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOExcepti
 
         if (allDocsMatch) {
           // all docs have a value and all points are within bounds, so everything matches
-          return new ScorerSupplier() {
+          return new ConstantScoreScorerSupplier(score(), scoreMode, reader.maxDoc()) {

Review Comment:
   Can we return `MatchAllScorerSupplier` here as well?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

Posted by "gf2121 (via GitHub)" <gi...@apache.org>.
gf2121 commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1971065012


##########
lucene/core/src/java/org/apache/lucene/document/SortedSetDocValuesRangeQuery.java:
##########
@@ -158,16 +157,15 @@ public Scorer get(long leadCost) throws IOException {
             if (minOrd > maxOrd
                 || (skipper != null
                     && (minOrd > skipper.maxValue() || maxOrd < skipper.minValue()))) {
-              return new ConstantScoreScorer(score(), scoreMode, DocIdSetIterator.empty());
+              return DocIdSetIterator.empty();

Review Comment:
   Not related to this PR: Should we actually should return a null `ScoreSupplier` to short cut following scorer building?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1973122492


##########
lucene/core/src/java/org/apache/lucene/search/TermQuery.java:
##########
@@ -165,6 +165,17 @@ public Scorer get(long leadCost) throws IOException {
           }
         }
 
+        @Override
+        public BulkScorer bulkScorer() throws IOException {
+          if (scoreMode.needsScores() == false) {
+            DocIdSetIterator iterator = get(Long.MAX_VALUE).iterator();
+            int maxDoc = context.reader().maxDoc();
+            return ConstantScoreScorerSupplier.fromIterator(iterator, 0f, scoreMode, maxDoc)

Review Comment:
   I think so, but I plan on having a potentially better way of doing it by reviving #12194 as a follow-up and skipping clauses that fully match a window (which would also remove the need for `MatchAllScorerSupplier`).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz merged PR #14293:
URL: https://github.com/apache/lucene/pull/14293


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #14293:
URL: https://github.com/apache/lucene/pull/14293#issuecomment-2685119226

   > Maybe we can replace all ScorerSuppliers returning ConstantScoreScorer with ConstantScoreScorerSupplier in follow up?
   
   Yes, that would be great. I started looking into it, but there were so many queries producing constant scores and I wanted to avoid making the change too big.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #14293:
URL: https://github.com/apache/lucene/pull/14293#issuecomment-2683288839

   luceneutil on wikibigall:
   
   ```
                               TaskQPS baseline      StdDevQPS my_modified_version      StdDev                Pct diff p-value
                      TermMonthSort     3445.14      (2.0%)     3079.25      (2.1%)  -10.6% ( -14% -   -6%) 0.000
                             IntNRQ      304.81      (3.1%)      293.29      (2.2%)   -3.8% (  -8% -    1%) 0.000
                        OrStopWords       34.42      (6.7%)       33.41      (7.0%)   -2.9% ( -15% -   11%) 0.176
                     FilteredIntNRQ      295.80      (3.0%)      287.79      (2.1%)   -2.7% (  -7% -    2%) 0.001
                       AndStopWords       31.38      (5.0%)       30.88      (5.3%)   -1.6% ( -11% -    9%) 0.325
                 Or2Terms2StopWords      165.37      (3.5%)      162.80      (4.0%)   -1.6% (  -8% -    6%) 0.189
                           Or3Terms      171.63      (3.7%)      169.33      (4.5%)   -1.3% (  -9% -    7%) 0.303
                         OrHighRare      274.93      (6.3%)      271.36      (7.6%)   -1.3% ( -14% -   13%) 0.556
                               Term      476.37      (3.6%)      471.05      (3.5%)   -1.1% (  -7% -    6%) 0.317
                      TermTitleSort      148.19      (1.6%)      146.56      (1.9%)   -1.1% (  -4% -    2%) 0.049
                         OrHighHigh       54.49      (3.5%)       53.90      (3.6%)   -1.1% (  -7% -    6%) 0.336
                          CountTerm     9946.28      (3.6%)     9848.37      (3.1%)   -1.0% (  -7% -    5%) 0.353
                And2Terms2StopWords      163.20      (2.8%)      161.62      (3.0%)   -1.0% (  -6% -    4%) 0.290
                          And3Terms      172.81      (3.0%)      171.20      (3.8%)   -0.9% (  -7% -    6%) 0.394
                          OrHighMed      204.60      (3.1%)      202.76      (3.2%)   -0.9% (  -7% -    5%) 0.369
                             Phrase       15.48      (5.3%)       15.36      (5.0%)   -0.8% ( -10% -   10%) 0.612
                   DismaxOrHighHigh      120.11      (2.4%)      119.14      (3.5%)   -0.8% (  -6% -    5%) 0.394
                       FilteredTerm      162.67      (2.2%)      161.62      (2.5%)   -0.6% (  -5% -    4%) 0.380
                 FilteredAndHighMed      131.15      (2.6%)      130.31      (3.0%)   -0.6% (  -6% -    5%) 0.469
                 CombinedOrHighHigh       19.37      (1.8%)       19.26      (1.4%)   -0.6% (  -3% -    2%) 0.239
                FilteredOrStopWords       47.47      (2.0%)       47.21      (2.0%)   -0.5% (  -4% -    3%) 0.393
                    DismaxOrHighMed      176.09      (1.7%)      175.13      (2.1%)   -0.5% (  -4% -    3%) 0.374
             CountFilteredOrHighMed      119.50      (0.5%)      118.93      (0.5%)   -0.5% (  -1% -    0%) 0.003
                             Fuzzy1       81.40      (3.0%)       81.04      (2.8%)   -0.4% (  -6% -    5%) 0.629
                             OrMany       19.38      (3.7%)       19.29      (4.8%)   -0.4% (  -8% -    8%) 0.747
                 FilteredOrHighHigh       68.81      (1.4%)       68.53      (2.0%)   -0.4% (  -3% -    2%) 0.437
                        AndHighHigh       43.50      (1.8%)       43.32      (1.8%)   -0.4% (  -3% -    3%) 0.488
                         AndHighMed      126.73      (1.7%)      126.26      (2.0%)   -0.4% (  -3% -    3%) 0.524
                CountFilteredPhrase       26.73      (1.9%)       26.65      (1.7%)   -0.3% (  -3% -    3%) 0.593
                             Fuzzy2       76.71      (2.7%)       76.48      (2.4%)   -0.3% (  -5% -    4%) 0.721
                FilteredAndHighHigh       69.24      (1.7%)       69.04      (1.7%)   -0.3% (  -3% -    3%) 0.593
                  FilteredOrHighMed      157.63      (0.8%)      157.21      (1.0%)   -0.3% (  -2% -    1%) 0.366
                  CombinedOrHighMed       74.22      (1.5%)       74.02      (1.4%)   -0.3% (  -3% -    2%) 0.570
            CountFilteredOrHighHigh      108.09      (0.7%)      107.83      (0.7%)   -0.2% (  -1% -    1%) 0.278
                  FilteredAnd3Terms      194.66      (2.0%)      194.26      (2.3%)   -0.2% (  -4% -    4%) 0.761
         FilteredOr2Terms2StopWords      151.10      (0.8%)      150.79      (1.1%)   -0.2% (  -2% -    1%) 0.510
                         DismaxTerm      574.89      (3.5%)      573.80      (4.0%)   -0.2% (  -7% -    7%) 0.874
                   AndMedOrHighHigh       66.19      (1.8%)       66.07      (1.5%)   -0.2% (  -3% -    3%) 0.738
        FilteredAnd2Terms2StopWords      202.29      (1.2%)      201.97      (1.8%)   -0.2% (  -3% -    2%) 0.734
               FilteredAndStopWords       55.33      (2.0%)       55.25      (1.7%)   -0.1% (  -3% -    3%) 0.813
                     FilteredPhrase       33.40      (2.1%)       33.38      (1.8%)   -0.1% (  -3% -    3%) 0.920
                   FilteredOr3Terms      166.05      (0.8%)      166.00      (1.1%)   -0.0% (  -1% -    1%) 0.921
                       CombinedTerm       31.77      (1.6%)       31.76      (1.5%)   -0.0% (  -3% -    3%) 0.952
                    AndHighOrMedMed       46.93      (1.1%)       46.93      (1.2%)   -0.0% (  -2% -    2%) 0.993
                    FilteredPrefix3      130.33      (4.6%)      130.42      (4.3%)    0.1% (  -8% -    9%) 0.962
                     CountOrHighMed      365.64      (1.8%)      365.90      (1.8%)    0.1% (  -3% -    3%) 0.901
                    CountAndHighMed      299.92      (2.6%)      300.28      (2.2%)    0.1% (  -4% -    5%) 0.874
                            Prefix3      135.80      (4.8%)      135.97      (4.5%)    0.1% (  -8% -    9%) 0.931
                        CountPhrase        4.29      (1.8%)        4.30      (1.8%)    0.2% (  -3% -    3%) 0.784
                     FilteredOrMany       17.71      (2.2%)       17.75      (1.8%)    0.2% (  -3% -    4%) 0.736
                CombinedAndHighHigh       15.63      (2.0%)       15.67      (1.6%)    0.3% (  -3% -    3%) 0.607
                           PKLookup      280.18      (1.9%)      281.10      (3.0%)    0.3% (  -4% -    5%) 0.680
                 CombinedAndHighMed       57.53      (1.7%)       57.73      (1.6%)    0.4% (  -2% -    3%) 0.499
                  TermDayOfYearSort      690.57      (2.3%)      694.34      (1.8%)    0.5% (  -3% -    4%) 0.395
                    CountOrHighHigh      298.01      (1.7%)      299.89      (2.2%)    0.6% (  -3% -    4%) 0.315
                           Wildcard       77.99      (4.5%)       78.57      (4.4%)    0.7% (  -7% -   10%) 0.595
                        CountOrMany       29.74      (2.2%)       29.99      (2.2%)    0.8% (  -3% -    5%) 0.220
                CountFilteredOrMany       25.92      (1.8%)       26.22      (1.8%)    1.1% (  -2% -    4%) 0.045
                   CountAndHighHigh      311.04      (3.8%)      316.26      (2.2%)    1.7% (  -4% -    7%) 0.087
                         TermDTSort      281.39      (8.4%)      504.98      (6.6%)   79.5% (  59% -  103%) 0.000
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

Posted by "gf2121 (via GitHub)" <gi...@apache.org>.
gf2121 commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1971058705


##########
lucene/core/src/java/org/apache/lucene/search/PointRangeQuery.java:
##########
@@ -341,11 +341,10 @@ public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOExcepti
 
         if (allDocsMatch) {
           // all docs have a value and all points are within bounds, so everything matches
-          return new ScorerSupplier() {
+          return new ConstantScoreScorerSupplier(score(), scoreMode, reader.maxDoc()) {
             @Override
-            public Scorer get(long leadCost) {
-              return new ConstantScoreScorer(
-                  score(), scoreMode, DocIdSetIterator.all(reader.maxDoc()));
+            public DocIdSetIterator iterator(long leadCost) {
+              return DocIdSetIterator.all(reader.maxDoc());

Review Comment:
   Can we somehow tells the supplier to return a `DenseConjunctionBulkScorer` with empty set of iterators so that we can take advantage of `RangeDocIdStream` ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #14293:
URL: https://github.com/apache/lucene/pull/14293#issuecomment-2683464574

   Same dataset but with 5% docs deleted this time and #14294 applied on both the baseline and the candidate:
   
   ```
                               TaskQPS baseline      StdDevQPS my_modified_version      StdDev                Pct diff p-value
                      TermMonthSort     3275.67      (2.5%)     2974.27      (2.2%)   -9.2% ( -13% -   -4%) 0.000
                             IntNRQ      302.63      (1.5%)      292.99      (1.0%)   -3.2% (  -5% -    0%) 0.000
                     FilteredIntNRQ      294.31      (1.6%)      285.28      (1.2%)   -3.1% (  -5% -    0%) 0.000
                            Prefix3      134.96      (2.8%)      133.43      (3.7%)   -1.1% (  -7% -    5%) 0.438
                    FilteredPrefix3      128.98      (2.9%)      127.64      (3.3%)   -1.0% (  -6% -    5%) 0.449
                CountFilteredPhrase       27.07      (1.9%)       26.87      (2.8%)   -0.7% (  -5% -    4%) 0.504
                    AndHighOrMedMed       45.77      (0.7%)       45.46      (1.5%)   -0.7% (  -2% -    1%) 0.193
                        CountPhrase        4.35      (2.2%)        4.32      (2.8%)   -0.7% (  -5% -    4%) 0.550
                 FilteredOrHighHigh       67.56      (1.9%)       67.17      (2.0%)   -0.6% (  -4% -    3%) 0.509
                     FilteredPhrase       33.43      (1.5%)       33.27      (1.8%)   -0.5% (  -3% -    2%) 0.509
                        AndHighHigh       41.58      (1.3%)       41.39      (2.5%)   -0.4% (  -4% -    3%) 0.622
                FilteredOrStopWords       46.53      (2.4%)       46.35      (2.5%)   -0.4% (  -5% -    4%) 0.717
                         AndHighMed      122.26      (1.0%)      121.85      (2.6%)   -0.3% (  -3% -    3%) 0.705
                             Fuzzy1       80.15      (1.3%)       79.88      (2.9%)   -0.3% (  -4% -    3%) 0.739
                   FilteredOr3Terms      161.01      (0.8%)      160.49      (1.4%)   -0.3% (  -2% -    1%) 0.529
                             Fuzzy2       75.56      (1.4%)       75.35      (2.6%)   -0.3% (  -4% -    3%) 0.764
         FilteredOr2Terms2StopWords      146.81      (1.1%)      146.50      (1.7%)   -0.2% (  -2% -    2%) 0.741
                   CountAndHighHigh      307.68      (1.7%)      307.05      (1.6%)   -0.2% (  -3% -    3%) 0.778
                           Wildcard       74.61      (2.2%)       74.48      (2.8%)   -0.2% (  -5% -    4%) 0.879
                        CountOrMany       29.88      (2.0%)       29.84      (1.9%)   -0.1% (  -3% -    3%) 0.863
                  FilteredOrHighMed      155.01      (0.9%)      154.82      (1.2%)   -0.1% (  -2% -    2%) 0.791
                   AndMedOrHighHigh       64.70      (2.1%)       64.63      (2.1%)   -0.1% (  -4% -    4%) 0.914
                     CountOrHighMed      418.10      (1.6%)      417.85      (1.7%)   -0.1% (  -3% -    3%) 0.935
                    CountAndHighMed      287.90      (1.6%)      287.77      (1.3%)   -0.0% (  -2% -    2%) 0.946
             CountFilteredOrHighMed      118.11      (0.5%)      118.08      (0.5%)   -0.0% (  -1% -    1%) 0.908
                         OrHighHigh       48.49      (2.1%)       48.49      (3.5%)   -0.0% (  -5% -    5%) 0.997
            CountFilteredOrHighHigh      106.97      (0.7%)      106.99      (0.7%)    0.0% (  -1% -    1%) 0.962
                     FilteredOrMany       17.52      (1.6%)       17.54      (1.3%)    0.1% (  -2% -    3%) 0.855
                 CombinedOrHighHigh       19.32      (2.1%)       19.36      (1.2%)    0.2% (  -2% -    3%) 0.764
                CombinedAndHighHigh       15.56      (1.8%)       15.60      (2.2%)    0.3% (  -3% -    4%) 0.774
                       CombinedTerm       31.62      (2.0%)       31.71      (1.0%)    0.3% (  -2% -    3%) 0.681
                    CountOrHighHigh      304.90      (1.7%)      305.77      (2.1%)    0.3% (  -3% -    4%) 0.739
                 CombinedAndHighMed       57.31      (1.6%)       57.48      (2.1%)    0.3% (  -3% -    4%) 0.725
                          OrHighMed      182.65      (2.0%)      183.40      (3.4%)    0.4% (  -4% -    5%) 0.743
                    DismaxOrHighMed      164.06      (1.3%)      164.74      (1.9%)    0.4% (  -2% -    3%) 0.577
                  CombinedOrHighMed       73.24      (1.6%)       73.65      (1.0%)    0.6% (  -1% -    3%) 0.338
                  FilteredAnd3Terms      191.89      (2.5%)      193.07      (1.7%)    0.6% (  -3% -    4%) 0.519
                CountFilteredOrMany       25.99      (1.3%)       26.17      (1.3%)    0.7% (  -1% -    3%) 0.230
                       FilteredTerm      155.92      (2.0%)      157.02      (2.5%)    0.7% (  -3% -    5%) 0.493
                           PKLookup      264.18      (1.3%)      266.20      (2.1%)    0.8% (  -2% -    4%) 0.325
                               Term      442.84      (3.4%)      446.23      (3.5%)    0.8% (  -5% -    7%) 0.619
        FilteredAnd2Terms2StopWords      198.39      (2.1%)      200.23      (0.8%)    0.9% (  -1% -    3%) 0.189
                             OrMany       18.89      (4.1%)       19.08      (3.7%)    1.0% (  -6% -    9%) 0.557
                             Phrase       14.87      (6.4%)       15.05      (6.0%)    1.2% ( -10% -   14%) 0.661
                   DismaxOrHighHigh      112.13      (2.6%)      113.73      (2.0%)    1.4% (  -3% -    6%) 0.165
                FilteredAndHighHigh       68.35      (4.0%)       69.41      (0.7%)    1.5% (  -3% -    6%) 0.231
                 FilteredAndHighMed      127.69      (3.4%)      129.82      (1.6%)    1.7% (  -3% -    6%) 0.165
               FilteredAndStopWords       54.67      (4.4%)       55.60      (1.0%)    1.7% (  -3% -    7%) 0.231
                 Or2Terms2StopWords      153.15      (4.1%)      155.83      (2.0%)    1.8% (  -4% -    8%) 0.222
                And2Terms2StopWords      156.52      (3.4%)      159.34      (1.6%)    1.8% (  -3% -    7%) 0.128
                          And3Terms      168.24      (3.2%)      171.55      (2.1%)    2.0% (  -3% -    7%) 0.104
                           Or3Terms      161.16      (3.5%)      164.43      (2.0%)    2.0% (  -3% -    7%) 0.117
                         DismaxTerm      531.13      (2.2%)      542.04      (3.0%)    2.1% (  -3% -    7%) 0.078
                      TermTitleSort      142.97      (1.5%)      145.96      (1.9%)    2.1% (  -1% -    5%) 0.006
                       AndStopWords       30.17      (5.6%)       30.85      (2.4%)    2.3% (  -5% -   10%) 0.240
                        OrStopWords       30.90      (5.9%)       31.68      (3.1%)    2.5% (  -6% -   12%) 0.237
                         OrHighRare      256.93      (8.3%)      266.71      (7.1%)    3.8% ( -10% -   20%) 0.270
                  TermDayOfYearSort      584.17      (2.2%)      655.16      (1.8%)   12.2% (   8% -   16%) 0.000
                         TermDTSort      263.14      (8.5%)      494.06      (3.5%)   87.8% (  69% -  109%) 0.000
                          CountTerm      156.94      (1.2%)      673.10      (9.2%)  328.9% ( 314% -  343%) 0.000
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #14293:
URL: https://github.com/apache/lucene/pull/14293#issuecomment-2687300793

   Yes, especially with queries that match long ranges of doc IDs by design, such as those that take advantage of sparse indexing.
   
   > For reminding, I think we also need a CHANGES entry :)
   
   Good point, added. :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1971643383


##########
lucene/core/src/java/org/apache/lucene/document/SortedSetDocValuesRangeQuery.java:
##########
@@ -158,16 +157,15 @@ public Scorer get(long leadCost) throws IOException {
             if (minOrd > maxOrd
                 || (skipper != null
                     && (minOrd > skipper.maxValue() || maxOrd < skipper.minValue()))) {
-              return new ConstantScoreScorer(score(), scoreMode, DocIdSetIterator.empty());
+              return DocIdSetIterator.empty();

Review Comment:
   Good question, I'm not sure we can. This would require doing I/O in `scorerSupplier()`, which would break Lucene's I/O parallelism.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

Posted by "gf2121 (via GitHub)" <gi...@apache.org>.
gf2121 commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1971065012


##########
lucene/core/src/java/org/apache/lucene/document/SortedSetDocValuesRangeQuery.java:
##########
@@ -158,16 +157,15 @@ public Scorer get(long leadCost) throws IOException {
             if (minOrd > maxOrd
                 || (skipper != null
                     && (minOrd > skipper.maxValue() || maxOrd < skipper.minValue()))) {
-              return new ConstantScoreScorer(score(), scoreMode, DocIdSetIterator.empty());
+              return DocIdSetIterator.empty();

Review Comment:
   Not related to this PR: Should we actually return a null `ScoreSupplier` to short cut following scorer building?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1971632861


##########
lucene/core/src/java/org/apache/lucene/search/PointRangeQuery.java:
##########
@@ -341,11 +341,10 @@ public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOExcepti
 
         if (allDocsMatch) {
           // all docs have a value and all points are within bounds, so everything matches
-          return new ScorerSupplier() {
+          return new ConstantScoreScorerSupplier(score(), scoreMode, reader.maxDoc()) {
             @Override
-            public Scorer get(long leadCost) {
-              return new ConstantScoreScorer(
-                  score(), scoreMode, DocIdSetIterator.all(reader.maxDoc()));
+            public DocIdSetIterator iterator(long leadCost) {
+              return DocIdSetIterator.all(reader.maxDoc());

Review Comment:
   Absolutely, I had planned to do this, this is a miss. Thanks for catching!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1973119540


##########
lucene/core/src/java/org/apache/lucene/search/PointRangeQuery.java:
##########
@@ -341,11 +341,10 @@ public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOExcepti
 
         if (allDocsMatch) {
           // all docs have a value and all points are within bounds, so everything matches
-          return new ScorerSupplier() {
+          return new ConstantScoreScorerSupplier(score(), scoreMode, reader.maxDoc()) {

Review Comment:
   Oops, yes, fixing.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]

Posted by "gf2121 (via GitHub)" <gi...@apache.org>.
gf2121 commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1971065012


##########
lucene/core/src/java/org/apache/lucene/document/SortedSetDocValuesRangeQuery.java:
##########
@@ -158,16 +157,15 @@ public Scorer get(long leadCost) throws IOException {
             if (minOrd > maxOrd
                 || (skipper != null
                     && (minOrd > skipper.maxValue() || maxOrd < skipper.minValue()))) {
-              return new ConstantScoreScorer(score(), scoreMode, DocIdSetIterator.empty());
+              return DocIdSetIterator.empty();

Review Comment:
   Not related to this PR: Should we return a null `ScoreSupplier` to short cut following scorer building?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org