You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by gu...@apache.org on 2025/04/25 15:33:37 UTC
(lucene) branch branch_10_2 updated: Provide better impacts for fields indexed with IndexOptions.DOCS (#14558)
This is an automated email from the ASF dual-hosted git repository.
guofeng pushed a commit to branch branch_10_2
in repository https://gitbox.apache.org/repos/asf/lucene.git
The following commit(s) were added to refs/heads/branch_10_2 by this push:
new 1b2451b5f43 Provide better impacts for fields indexed with IndexOptions.DOCS (#14558)
1b2451b5f43 is described below
commit 1b2451b5f435ad2c41a8b6b0f0d6a59b24ff9ad2
Author: Guo Feng <52...@users.noreply.github.com>
AuthorDate: Fri Apr 25 23:33:32 2025 +0800
Provide better impacts for fields indexed with IndexOptions.DOCS (#14558)
Postings always return impacts with freq=Integer.MAX_VALUE and norm=1 when frequencies are not indexed (IndexOptions.DOCS). This significantly overestimates the score upper bound of term queries, since the similarity scorer is effectively called with freq=1 all the time in this case (and either norm=1 if norms are not indexed, or the number of terms in the field otherwise).
This updates postings to always return impacts with freq=1 and norm=1 when frequencies are not indexed, which helps compute better score upper bounds, and in-turn makes dynamic pruning perform better.
Closes #14445
---
lucene/CHANGES.txt | 2 ++
.../codecs/lucene101/Lucene101PostingsReader.java | 19 ++++++++++++-------
2 files changed, 14 insertions(+), 7 deletions(-)
diff --git a/lucene/CHANGES.txt b/lucene/CHANGES.txt
index decb1de0abe..453b7a34a92 100644
--- a/lucene/CHANGES.txt
+++ b/lucene/CHANGES.txt
@@ -14,6 +14,8 @@ Bug Fixes
* GITHUB#14523, GITHUB#14530: Correct TermOrdValComparator competitive iterator so that it forces sparse
field iteration to be at least scoring window baseline when doing intoBitSet. (Ben Trent, Adrien Grand)
+* GITHUB#14445: Provide better impacts for fields indexed with IndexOptions.DOCS GITHUB#14511 (Aniketh Jain)
+
* GITHUB#14543: Fixed lead cost computations for bulk scorers of conjunctive
queries that mix MUST and FILTER clauses, and disjunctive queries that
configure a minimum number of matching SHOULD clauses.
diff --git a/lucene/core/src/java/org/apache/lucene/codecs/lucene101/Lucene101PostingsReader.java b/lucene/core/src/java/org/apache/lucene/codecs/lucene101/Lucene101PostingsReader.java
index 052e281ca56..9141ad5133d 100644
--- a/lucene/core/src/java/org/apache/lucene/codecs/lucene101/Lucene101PostingsReader.java
+++ b/lucene/core/src/java/org/apache/lucene/codecs/lucene101/Lucene101PostingsReader.java
@@ -73,6 +73,10 @@ public final class Lucene101PostingsReader extends PostingsReaderBase {
private static final List<Impact> DUMMY_IMPACTS =
Collections.singletonList(new Impact(Integer.MAX_VALUE, 1L));
+ // We stopped storing a placeholder impact with freq=1 for fields with DOCS after 9.12.0
+ private static final List<Impact> DUMMY_IMPACTS_NO_FREQS =
+ Collections.singletonList(new Impact(1, 1L));
+
private final IndexInput docIn;
private final IndexInput posIn;
private final IndexInput payIn;
@@ -1328,13 +1332,14 @@ public final class Lucene101PostingsReader extends PostingsReaderBase {
@Override
public List<Impact> getImpacts(int level) {
- if (indexHasFreq) {
- if (level == 0 && level0LastDocID != NO_MORE_DOCS) {
- return readImpacts(level0SerializedImpacts, level0Impacts);
- }
- if (level == 1) {
- return readImpacts(level1SerializedImpacts, level1Impacts);
- }
+ if (indexHasFreq == false) {
+ return DUMMY_IMPACTS_NO_FREQS;
+ }
+ if (level == 0 && level0LastDocID != NO_MORE_DOCS) {
+ return readImpacts(level0SerializedImpacts, level0Impacts);
+ }
+ if (level == 1) {
+ return readImpacts(level1SerializedImpacts, level1Impacts);
}
return DUMMY_IMPACTS;
}