You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "jpountz (via GitHub)" <gi...@apache.org> on 2025/02/25 21:02:12 UTC
[PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]
jpountz opened a new pull request, #14293:
URL: https://github.com/apache/lucene/pull/14293
Even though there is a single clause, it often needs to be intersected, either with live docs or with the collector's competitive iterator.
This uses `DenseConjunctionBulkScorer` for:
- `MatchAllDocsQuery`,
- `TermQuery` when scores are not needed and `docFreq >= maxDoc / 32`,
- some constant-score queries including `PointRangeQuery` when `cost >= maxDoc / 32`.
In addition,
- `DenseConjunctionBulkScorer` was improved to stop collecting hits when the min competitive score is greater than the configured constant score,
- Added test coverage to sorting tests to make sure that their competitive iterators are happy with how `DenseConjunctionBulkScorer` reads ahead.
The downside of this change is that it forces the impacted queries to score at least 4096 (the window size) docs at once. So queries that can start skipping very early (and which are thus very fast) may see a slowdown (e.g. `TermMonthSort`). On the other hand, when it takes time before dynamic pruning becomes effective, there could be a speedup thanks to the more efficient intersection logic (e.g. `TermDTSort`).
This change should also help in the presence of deleted docs, by taking advantage of the more efficient way how deleted docs are applied in this bulk scorer.
Closes #14283
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]
Posted by "gf2121 (via GitHub)" <gi...@apache.org>.
gf2121 commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1972872831
##########
lucene/core/src/java/org/apache/lucene/search/TermQuery.java:
##########
@@ -165,6 +165,17 @@ public Scorer get(long leadCost) throws IOException {
}
}
+ @Override
+ public BulkScorer bulkScorer() throws IOException {
+ if (scoreMode.needsScores() == false) {
+ DocIdSetIterator iterator = get(Long.MAX_VALUE).iterator();
+ int maxDoc = context.reader().maxDoc();
+ return ConstantScoreScorerSupplier.fromIterator(iterator, 0f, scoreMode, maxDoc)
Review Comment:
Is it correct to return `MatchAllScorerSupplier#bulkScorer()` when`getTermsEnum().docFreq() == maxDoc`?
##########
lucene/core/src/java/org/apache/lucene/search/PointRangeQuery.java:
##########
@@ -341,11 +341,10 @@ public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOExcepti
if (allDocsMatch) {
// all docs have a value and all points are within bounds, so everything matches
- return new ScorerSupplier() {
+ return new ConstantScoreScorerSupplier(score(), scoreMode, reader.maxDoc()) {
Review Comment:
Can we return `MatchAllScorerSupplier` here as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]
Posted by "gf2121 (via GitHub)" <gi...@apache.org>.
gf2121 commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1971065012
##########
lucene/core/src/java/org/apache/lucene/document/SortedSetDocValuesRangeQuery.java:
##########
@@ -158,16 +157,15 @@ public Scorer get(long leadCost) throws IOException {
if (minOrd > maxOrd
|| (skipper != null
&& (minOrd > skipper.maxValue() || maxOrd < skipper.minValue()))) {
- return new ConstantScoreScorer(score(), scoreMode, DocIdSetIterator.empty());
+ return DocIdSetIterator.empty();
Review Comment:
Not related to this PR: Should we actually should return a null `ScoreSupplier` to short cut following scorer building?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1973122492
##########
lucene/core/src/java/org/apache/lucene/search/TermQuery.java:
##########
@@ -165,6 +165,17 @@ public Scorer get(long leadCost) throws IOException {
}
}
+ @Override
+ public BulkScorer bulkScorer() throws IOException {
+ if (scoreMode.needsScores() == false) {
+ DocIdSetIterator iterator = get(Long.MAX_VALUE).iterator();
+ int maxDoc = context.reader().maxDoc();
+ return ConstantScoreScorerSupplier.fromIterator(iterator, 0f, scoreMode, maxDoc)
Review Comment:
I think so, but I plan on having a potentially better way of doing it by reviving #12194 as a follow-up and skipping clauses that fully match a window (which would also remove the need for `MatchAllScorerSupplier`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz merged PR #14293:
URL: https://github.com/apache/lucene/pull/14293
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #14293:
URL: https://github.com/apache/lucene/pull/14293#issuecomment-2685119226
> Maybe we can replace all ScorerSuppliers returning ConstantScoreScorer with ConstantScoreScorerSupplier in follow up?
Yes, that would be great. I started looking into it, but there were so many queries producing constant scores and I wanted to avoid making the change too big.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #14293:
URL: https://github.com/apache/lucene/pull/14293#issuecomment-2683288839
luceneutil on wikibigall:
```
TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value
TermMonthSort 3445.14 (2.0%) 3079.25 (2.1%) -10.6% ( -14% - -6%) 0.000
IntNRQ 304.81 (3.1%) 293.29 (2.2%) -3.8% ( -8% - 1%) 0.000
OrStopWords 34.42 (6.7%) 33.41 (7.0%) -2.9% ( -15% - 11%) 0.176
FilteredIntNRQ 295.80 (3.0%) 287.79 (2.1%) -2.7% ( -7% - 2%) 0.001
AndStopWords 31.38 (5.0%) 30.88 (5.3%) -1.6% ( -11% - 9%) 0.325
Or2Terms2StopWords 165.37 (3.5%) 162.80 (4.0%) -1.6% ( -8% - 6%) 0.189
Or3Terms 171.63 (3.7%) 169.33 (4.5%) -1.3% ( -9% - 7%) 0.303
OrHighRare 274.93 (6.3%) 271.36 (7.6%) -1.3% ( -14% - 13%) 0.556
Term 476.37 (3.6%) 471.05 (3.5%) -1.1% ( -7% - 6%) 0.317
TermTitleSort 148.19 (1.6%) 146.56 (1.9%) -1.1% ( -4% - 2%) 0.049
OrHighHigh 54.49 (3.5%) 53.90 (3.6%) -1.1% ( -7% - 6%) 0.336
CountTerm 9946.28 (3.6%) 9848.37 (3.1%) -1.0% ( -7% - 5%) 0.353
And2Terms2StopWords 163.20 (2.8%) 161.62 (3.0%) -1.0% ( -6% - 4%) 0.290
And3Terms 172.81 (3.0%) 171.20 (3.8%) -0.9% ( -7% - 6%) 0.394
OrHighMed 204.60 (3.1%) 202.76 (3.2%) -0.9% ( -7% - 5%) 0.369
Phrase 15.48 (5.3%) 15.36 (5.0%) -0.8% ( -10% - 10%) 0.612
DismaxOrHighHigh 120.11 (2.4%) 119.14 (3.5%) -0.8% ( -6% - 5%) 0.394
FilteredTerm 162.67 (2.2%) 161.62 (2.5%) -0.6% ( -5% - 4%) 0.380
FilteredAndHighMed 131.15 (2.6%) 130.31 (3.0%) -0.6% ( -6% - 5%) 0.469
CombinedOrHighHigh 19.37 (1.8%) 19.26 (1.4%) -0.6% ( -3% - 2%) 0.239
FilteredOrStopWords 47.47 (2.0%) 47.21 (2.0%) -0.5% ( -4% - 3%) 0.393
DismaxOrHighMed 176.09 (1.7%) 175.13 (2.1%) -0.5% ( -4% - 3%) 0.374
CountFilteredOrHighMed 119.50 (0.5%) 118.93 (0.5%) -0.5% ( -1% - 0%) 0.003
Fuzzy1 81.40 (3.0%) 81.04 (2.8%) -0.4% ( -6% - 5%) 0.629
OrMany 19.38 (3.7%) 19.29 (4.8%) -0.4% ( -8% - 8%) 0.747
FilteredOrHighHigh 68.81 (1.4%) 68.53 (2.0%) -0.4% ( -3% - 2%) 0.437
AndHighHigh 43.50 (1.8%) 43.32 (1.8%) -0.4% ( -3% - 3%) 0.488
AndHighMed 126.73 (1.7%) 126.26 (2.0%) -0.4% ( -3% - 3%) 0.524
CountFilteredPhrase 26.73 (1.9%) 26.65 (1.7%) -0.3% ( -3% - 3%) 0.593
Fuzzy2 76.71 (2.7%) 76.48 (2.4%) -0.3% ( -5% - 4%) 0.721
FilteredAndHighHigh 69.24 (1.7%) 69.04 (1.7%) -0.3% ( -3% - 3%) 0.593
FilteredOrHighMed 157.63 (0.8%) 157.21 (1.0%) -0.3% ( -2% - 1%) 0.366
CombinedOrHighMed 74.22 (1.5%) 74.02 (1.4%) -0.3% ( -3% - 2%) 0.570
CountFilteredOrHighHigh 108.09 (0.7%) 107.83 (0.7%) -0.2% ( -1% - 1%) 0.278
FilteredAnd3Terms 194.66 (2.0%) 194.26 (2.3%) -0.2% ( -4% - 4%) 0.761
FilteredOr2Terms2StopWords 151.10 (0.8%) 150.79 (1.1%) -0.2% ( -2% - 1%) 0.510
DismaxTerm 574.89 (3.5%) 573.80 (4.0%) -0.2% ( -7% - 7%) 0.874
AndMedOrHighHigh 66.19 (1.8%) 66.07 (1.5%) -0.2% ( -3% - 3%) 0.738
FilteredAnd2Terms2StopWords 202.29 (1.2%) 201.97 (1.8%) -0.2% ( -3% - 2%) 0.734
FilteredAndStopWords 55.33 (2.0%) 55.25 (1.7%) -0.1% ( -3% - 3%) 0.813
FilteredPhrase 33.40 (2.1%) 33.38 (1.8%) -0.1% ( -3% - 3%) 0.920
FilteredOr3Terms 166.05 (0.8%) 166.00 (1.1%) -0.0% ( -1% - 1%) 0.921
CombinedTerm 31.77 (1.6%) 31.76 (1.5%) -0.0% ( -3% - 3%) 0.952
AndHighOrMedMed 46.93 (1.1%) 46.93 (1.2%) -0.0% ( -2% - 2%) 0.993
FilteredPrefix3 130.33 (4.6%) 130.42 (4.3%) 0.1% ( -8% - 9%) 0.962
CountOrHighMed 365.64 (1.8%) 365.90 (1.8%) 0.1% ( -3% - 3%) 0.901
CountAndHighMed 299.92 (2.6%) 300.28 (2.2%) 0.1% ( -4% - 5%) 0.874
Prefix3 135.80 (4.8%) 135.97 (4.5%) 0.1% ( -8% - 9%) 0.931
CountPhrase 4.29 (1.8%) 4.30 (1.8%) 0.2% ( -3% - 3%) 0.784
FilteredOrMany 17.71 (2.2%) 17.75 (1.8%) 0.2% ( -3% - 4%) 0.736
CombinedAndHighHigh 15.63 (2.0%) 15.67 (1.6%) 0.3% ( -3% - 3%) 0.607
PKLookup 280.18 (1.9%) 281.10 (3.0%) 0.3% ( -4% - 5%) 0.680
CombinedAndHighMed 57.53 (1.7%) 57.73 (1.6%) 0.4% ( -2% - 3%) 0.499
TermDayOfYearSort 690.57 (2.3%) 694.34 (1.8%) 0.5% ( -3% - 4%) 0.395
CountOrHighHigh 298.01 (1.7%) 299.89 (2.2%) 0.6% ( -3% - 4%) 0.315
Wildcard 77.99 (4.5%) 78.57 (4.4%) 0.7% ( -7% - 10%) 0.595
CountOrMany 29.74 (2.2%) 29.99 (2.2%) 0.8% ( -3% - 5%) 0.220
CountFilteredOrMany 25.92 (1.8%) 26.22 (1.8%) 1.1% ( -2% - 4%) 0.045
CountAndHighHigh 311.04 (3.8%) 316.26 (2.2%) 1.7% ( -4% - 7%) 0.087
TermDTSort 281.39 (8.4%) 504.98 (6.6%) 79.5% ( 59% - 103%) 0.000
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]
Posted by "gf2121 (via GitHub)" <gi...@apache.org>.
gf2121 commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1971058705
##########
lucene/core/src/java/org/apache/lucene/search/PointRangeQuery.java:
##########
@@ -341,11 +341,10 @@ public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOExcepti
if (allDocsMatch) {
// all docs have a value and all points are within bounds, so everything matches
- return new ScorerSupplier() {
+ return new ConstantScoreScorerSupplier(score(), scoreMode, reader.maxDoc()) {
@Override
- public Scorer get(long leadCost) {
- return new ConstantScoreScorer(
- score(), scoreMode, DocIdSetIterator.all(reader.maxDoc()));
+ public DocIdSetIterator iterator(long leadCost) {
+ return DocIdSetIterator.all(reader.maxDoc());
Review Comment:
Can we somehow tells the supplier to return a `DenseConjunctionBulkScorer` with empty set of iterators so that we can take advantage of `RangeDocIdStream` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #14293:
URL: https://github.com/apache/lucene/pull/14293#issuecomment-2683464574
Same dataset but with 5% docs deleted this time and #14294 applied on both the baseline and the candidate:
```
TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value
TermMonthSort 3275.67 (2.5%) 2974.27 (2.2%) -9.2% ( -13% - -4%) 0.000
IntNRQ 302.63 (1.5%) 292.99 (1.0%) -3.2% ( -5% - 0%) 0.000
FilteredIntNRQ 294.31 (1.6%) 285.28 (1.2%) -3.1% ( -5% - 0%) 0.000
Prefix3 134.96 (2.8%) 133.43 (3.7%) -1.1% ( -7% - 5%) 0.438
FilteredPrefix3 128.98 (2.9%) 127.64 (3.3%) -1.0% ( -6% - 5%) 0.449
CountFilteredPhrase 27.07 (1.9%) 26.87 (2.8%) -0.7% ( -5% - 4%) 0.504
AndHighOrMedMed 45.77 (0.7%) 45.46 (1.5%) -0.7% ( -2% - 1%) 0.193
CountPhrase 4.35 (2.2%) 4.32 (2.8%) -0.7% ( -5% - 4%) 0.550
FilteredOrHighHigh 67.56 (1.9%) 67.17 (2.0%) -0.6% ( -4% - 3%) 0.509
FilteredPhrase 33.43 (1.5%) 33.27 (1.8%) -0.5% ( -3% - 2%) 0.509
AndHighHigh 41.58 (1.3%) 41.39 (2.5%) -0.4% ( -4% - 3%) 0.622
FilteredOrStopWords 46.53 (2.4%) 46.35 (2.5%) -0.4% ( -5% - 4%) 0.717
AndHighMed 122.26 (1.0%) 121.85 (2.6%) -0.3% ( -3% - 3%) 0.705
Fuzzy1 80.15 (1.3%) 79.88 (2.9%) -0.3% ( -4% - 3%) 0.739
FilteredOr3Terms 161.01 (0.8%) 160.49 (1.4%) -0.3% ( -2% - 1%) 0.529
Fuzzy2 75.56 (1.4%) 75.35 (2.6%) -0.3% ( -4% - 3%) 0.764
FilteredOr2Terms2StopWords 146.81 (1.1%) 146.50 (1.7%) -0.2% ( -2% - 2%) 0.741
CountAndHighHigh 307.68 (1.7%) 307.05 (1.6%) -0.2% ( -3% - 3%) 0.778
Wildcard 74.61 (2.2%) 74.48 (2.8%) -0.2% ( -5% - 4%) 0.879
CountOrMany 29.88 (2.0%) 29.84 (1.9%) -0.1% ( -3% - 3%) 0.863
FilteredOrHighMed 155.01 (0.9%) 154.82 (1.2%) -0.1% ( -2% - 2%) 0.791
AndMedOrHighHigh 64.70 (2.1%) 64.63 (2.1%) -0.1% ( -4% - 4%) 0.914
CountOrHighMed 418.10 (1.6%) 417.85 (1.7%) -0.1% ( -3% - 3%) 0.935
CountAndHighMed 287.90 (1.6%) 287.77 (1.3%) -0.0% ( -2% - 2%) 0.946
CountFilteredOrHighMed 118.11 (0.5%) 118.08 (0.5%) -0.0% ( -1% - 1%) 0.908
OrHighHigh 48.49 (2.1%) 48.49 (3.5%) -0.0% ( -5% - 5%) 0.997
CountFilteredOrHighHigh 106.97 (0.7%) 106.99 (0.7%) 0.0% ( -1% - 1%) 0.962
FilteredOrMany 17.52 (1.6%) 17.54 (1.3%) 0.1% ( -2% - 3%) 0.855
CombinedOrHighHigh 19.32 (2.1%) 19.36 (1.2%) 0.2% ( -2% - 3%) 0.764
CombinedAndHighHigh 15.56 (1.8%) 15.60 (2.2%) 0.3% ( -3% - 4%) 0.774
CombinedTerm 31.62 (2.0%) 31.71 (1.0%) 0.3% ( -2% - 3%) 0.681
CountOrHighHigh 304.90 (1.7%) 305.77 (2.1%) 0.3% ( -3% - 4%) 0.739
CombinedAndHighMed 57.31 (1.6%) 57.48 (2.1%) 0.3% ( -3% - 4%) 0.725
OrHighMed 182.65 (2.0%) 183.40 (3.4%) 0.4% ( -4% - 5%) 0.743
DismaxOrHighMed 164.06 (1.3%) 164.74 (1.9%) 0.4% ( -2% - 3%) 0.577
CombinedOrHighMed 73.24 (1.6%) 73.65 (1.0%) 0.6% ( -1% - 3%) 0.338
FilteredAnd3Terms 191.89 (2.5%) 193.07 (1.7%) 0.6% ( -3% - 4%) 0.519
CountFilteredOrMany 25.99 (1.3%) 26.17 (1.3%) 0.7% ( -1% - 3%) 0.230
FilteredTerm 155.92 (2.0%) 157.02 (2.5%) 0.7% ( -3% - 5%) 0.493
PKLookup 264.18 (1.3%) 266.20 (2.1%) 0.8% ( -2% - 4%) 0.325
Term 442.84 (3.4%) 446.23 (3.5%) 0.8% ( -5% - 7%) 0.619
FilteredAnd2Terms2StopWords 198.39 (2.1%) 200.23 (0.8%) 0.9% ( -1% - 3%) 0.189
OrMany 18.89 (4.1%) 19.08 (3.7%) 1.0% ( -6% - 9%) 0.557
Phrase 14.87 (6.4%) 15.05 (6.0%) 1.2% ( -10% - 14%) 0.661
DismaxOrHighHigh 112.13 (2.6%) 113.73 (2.0%) 1.4% ( -3% - 6%) 0.165
FilteredAndHighHigh 68.35 (4.0%) 69.41 (0.7%) 1.5% ( -3% - 6%) 0.231
FilteredAndHighMed 127.69 (3.4%) 129.82 (1.6%) 1.7% ( -3% - 6%) 0.165
FilteredAndStopWords 54.67 (4.4%) 55.60 (1.0%) 1.7% ( -3% - 7%) 0.231
Or2Terms2StopWords 153.15 (4.1%) 155.83 (2.0%) 1.8% ( -4% - 8%) 0.222
And2Terms2StopWords 156.52 (3.4%) 159.34 (1.6%) 1.8% ( -3% - 7%) 0.128
And3Terms 168.24 (3.2%) 171.55 (2.1%) 2.0% ( -3% - 7%) 0.104
Or3Terms 161.16 (3.5%) 164.43 (2.0%) 2.0% ( -3% - 7%) 0.117
DismaxTerm 531.13 (2.2%) 542.04 (3.0%) 2.1% ( -3% - 7%) 0.078
TermTitleSort 142.97 (1.5%) 145.96 (1.9%) 2.1% ( -1% - 5%) 0.006
AndStopWords 30.17 (5.6%) 30.85 (2.4%) 2.3% ( -5% - 10%) 0.240
OrStopWords 30.90 (5.9%) 31.68 (3.1%) 2.5% ( -6% - 12%) 0.237
OrHighRare 256.93 (8.3%) 266.71 (7.1%) 3.8% ( -10% - 20%) 0.270
TermDayOfYearSort 584.17 (2.2%) 655.16 (1.8%) 12.2% ( 8% - 16%) 0.000
TermDTSort 263.14 (8.5%) 494.06 (3.5%) 87.8% ( 69% - 109%) 0.000
CountTerm 156.94 (1.2%) 673.10 (9.2%) 328.9% ( 314% - 343%) 0.000
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on PR #14293:
URL: https://github.com/apache/lucene/pull/14293#issuecomment-2687300793
Yes, especially with queries that match long ranges of doc IDs by design, such as those that take advantage of sparse indexing.
> For reminding, I think we also need a CHANGES entry :)
Good point, added. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1971643383
##########
lucene/core/src/java/org/apache/lucene/document/SortedSetDocValuesRangeQuery.java:
##########
@@ -158,16 +157,15 @@ public Scorer get(long leadCost) throws IOException {
if (minOrd > maxOrd
|| (skipper != null
&& (minOrd > skipper.maxValue() || maxOrd < skipper.minValue()))) {
- return new ConstantScoreScorer(score(), scoreMode, DocIdSetIterator.empty());
+ return DocIdSetIterator.empty();
Review Comment:
Good question, I'm not sure we can. This would require doing I/O in `scorerSupplier()`, which would break Lucene's I/O parallelism.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]
Posted by "gf2121 (via GitHub)" <gi...@apache.org>.
gf2121 commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1971065012
##########
lucene/core/src/java/org/apache/lucene/document/SortedSetDocValuesRangeQuery.java:
##########
@@ -158,16 +157,15 @@ public Scorer get(long leadCost) throws IOException {
if (minOrd > maxOrd
|| (skipper != null
&& (minOrd > skipper.maxValue() || maxOrd < skipper.minValue()))) {
- return new ConstantScoreScorer(score(), scoreMode, DocIdSetIterator.empty());
+ return DocIdSetIterator.empty();
Review Comment:
Not related to this PR: Should we actually return a null `ScoreSupplier` to short cut following scorer building?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1971632861
##########
lucene/core/src/java/org/apache/lucene/search/PointRangeQuery.java:
##########
@@ -341,11 +341,10 @@ public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOExcepti
if (allDocsMatch) {
// all docs have a value and all points are within bounds, so everything matches
- return new ScorerSupplier() {
+ return new ConstantScoreScorerSupplier(score(), scoreMode, reader.maxDoc()) {
@Override
- public Scorer get(long leadCost) {
- return new ConstantScoreScorer(
- score(), scoreMode, DocIdSetIterator.all(reader.maxDoc()));
+ public DocIdSetIterator iterator(long leadCost) {
+ return DocIdSetIterator.all(reader.maxDoc());
Review Comment:
Absolutely, I had planned to do this, this is a miss. Thanks for catching!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1973119540
##########
lucene/core/src/java/org/apache/lucene/search/PointRangeQuery.java:
##########
@@ -341,11 +341,10 @@ public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOExcepti
if (allDocsMatch) {
// all docs have a value and all points are within bounds, so everything matches
- return new ScorerSupplier() {
+ return new ConstantScoreScorerSupplier(score(), scoreMode, reader.maxDoc()) {
Review Comment:
Oops, yes, fixing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [PR] Use DenseConjunctionBulkScorer for single queries sometimes. [lucene]
Posted by "gf2121 (via GitHub)" <gi...@apache.org>.
gf2121 commented on code in PR #14293:
URL: https://github.com/apache/lucene/pull/14293#discussion_r1971065012
##########
lucene/core/src/java/org/apache/lucene/document/SortedSetDocValuesRangeQuery.java:
##########
@@ -158,16 +157,15 @@ public Scorer get(long leadCost) throws IOException {
if (minOrd > maxOrd
|| (skipper != null
&& (minOrd > skipper.maxValue() || maxOrd < skipper.minValue()))) {
- return new ConstantScoreScorer(score(), scoreMode, DocIdSetIterator.empty());
+ return DocIdSetIterator.empty();
Review Comment:
Not related to this PR: Should we return a null `ScoreSupplier` to short cut following scorer building?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org