You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by be...@apache.org on 2026/03/27 20:08:20 UTC
(lucene) branch main updated: Add BayesianScoreQuery and LogOddsFusionQuery for probabilistic hybrid search (#15827)
This is an automated email from the ASF dual-hosted git repository.
benwtrent pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/lucene.git
The following commit(s) were added to refs/heads/main by this push:
new 5af15dc4be5 Add BayesianScoreQuery and LogOddsFusionQuery for probabilistic hybrid search (#15827)
5af15dc4be5 is described below
commit 5af15dc4be5b3a5089f1c02e6f11122165d82939
Author: Jaepil Jeong <zg...@gmail.com>
AuthorDate: Sat Mar 28 05:08:14 2026 +0900
Add BayesianScoreQuery and LogOddsFusionQuery for probabilistic hybrid search (#15827)
* Add BayesianBM25Similarity and LogOddsConjunctionQuery to Lucene
BayesianBM25Similarity transforms BM25 scores into calibrated
probabilities via sigmoid likelihood + tf-prior + Bayesian posterior.
The tf-only prior preserves SimScorer's norm monotonicity contract.
LogOddsConjunctionQuery combines probability scores from multiple
sub-queries via mean log-odds with n^alpha confidence scaling,
resolving the shrinkage problem of naive probabilistic AND.
Softplus gating (log(1 + exp(logit))) is applied to matching
sub-scorer logits before aggregation, ensuring that any match
contributes positively -- distinguishing "absence of evidence"
(non-match, contributes 0) from weak evidence (match with low
probability, contributes a small positive value).
* Replace magic numbers in BayesianBM25Similarity with named constants
Extract tfPrior formula constants (TF_PRIOR_BASE, TF_PRIOR_MAX,
TF_SATURATION_FREQ, PRIOR_CLAMP_MIN/MAX) and auto-estimation
constants (AUTO_ALPHA_NUMERATOR, AUTO_BETA_MULTIPLIER, WEIGHT_FLOOR)
into named static fields.
* Add vector+text hybrid search tests and fix posterior monotonicity
Add hybrid search tests combining BayesianBM25 text queries with KNN
cosine vector queries via LogOddsConjunctionQuery: single/multi-field
text+vector, multiple vector fields, three-way hybrid, BooleanQuery
MUST/MUST_NOT integration.
Fix float precision issue in BayesianBM25Scorer that broke SimScorer
monotonicity contract -- posterior computation now uses double
precision internally to prevent rounding errors from inverting the
score ordering between adjacent freq values.
Fix flaky KNN tests by setting the default codec via
TestUtil.getDefaultCodec(), matching the pattern used by Lucene's
own BaseKnnVectorQueryTestCase.
* Add AND/OR/NOT combination tests for vector+text hybrid search
- testBooleanShouldWithHybrid: OR of two independent hybrid queries
- testVectorAndVector: MUST conjunction of two KNN vector queries
- testVectorNot: MUST_NOT exclusion of vector-similar documents
from text search results
* Apply google-java-format
* Add CHANGES.txt entry for BayesianBM25Similarity and LogOddsConjunctionQuery
* Remove unused alpha field from LogOddsConjunctionScorer
* Remove unused fields and apply google-java-format
* Replace BayesianBM25Similarity with BayesianScoreQuery
BayesianBM25Similarity applied the Bayesian transform at the per-term
level inside SimScorer, which broke BM25's IDF-weighted term
combination when used with BooleanQuery -- BEIR benchmarks confirmed
a significant NDCG degradation.
The paper's design operates at the query level: BM25 produces a
query-level score via its standard sum(IDF * tf_norm), then a single
sigmoid calibration maps this to a probability. BayesianScoreQuery
implements this correctly as a Query wrapper that transforms any inner
query's score via sigmoid(alpha * (score - beta)).
This preserves BM25 ranking exactly (sigmoid is monotone) while
producing (0,1) probability scores suitable for LogOddsConjunction.
- Remove BayesianBM25Similarity and TestBayesianBM25Similarity
- Add BayesianScoreQuery with advanceShallow/getMaxScore for WAND
- Add TestBayesianScoreQuery (14 tests)
- Rewrite TestLogOddsConjunctionQuery to use BayesianScoreQuery
* Address PR review: add @lucene.experimental, use Math.clamp, move CHANGES entry to 10.5
- Add @lucene.experimental tag to BayesianScoreQuery, LogOddsConjunctionQuery,
and LogOddsConjunctionScorer Javadoc
- Replace Math.max/Math.min with Math.clamp in clampProbability
- Move CHANGES.txt entry from 11.0.0 to 10.5.0
* Rename LogOddsConjunctionQuery/Scorer to LogOddsFusionQuery/Scorer
* Apply google-java-format
---
lucene/CHANGES.txt | 6 +
.../apache/lucene/search/BayesianScoreQuery.java | 236 +++++++
.../apache/lucene/search/LogOddsFusionQuery.java | 289 +++++++++
.../apache/lucene/search/LogOddsFusionScorer.java | 157 +++++
.../lucene/search/TestBayesianScoreQuery.java | 260 ++++++++
.../lucene/search/TestLogOddsFusionQuery.java | 684 +++++++++++++++++++++
6 files changed, 1632 insertions(+)
diff --git a/lucene/CHANGES.txt b/lucene/CHANGES.txt
index f46334fcaeb..0d756b2087c 100644
--- a/lucene/CHANGES.txt
+++ b/lucene/CHANGES.txt
@@ -246,6 +246,12 @@ New Features
* GITHUB#15722: Add interface to do prefetch on KnnVectorValues and an example implementation to use prefetch
in Scorer (Navneet Verma)
+* GITHUB#15827: Add BayesianScoreQuery and LogOddsFusionQuery for probabilistic hybrid
+ search. BayesianScoreQuery wraps any query and applies sigmoid calibration to transform scores
+ into probabilities. LogOddsFusionQuery combines multiple probability signals (text, vector,
+ or mixed) via log-odds fusion with softplus gating and sqrt(n) confidence scaling.
+ (Jaepil Jeong)
+
Improvements
---------------------
* GITHUB#15711: DirectoryTaxonomyReader.getBulkPath() now uses getPathFromCache. (Shubham Sharma)
diff --git a/lucene/core/src/java/org/apache/lucene/search/BayesianScoreQuery.java b/lucene/core/src/java/org/apache/lucene/search/BayesianScoreQuery.java
new file mode 100644
index 00000000000..fbb7af8fb6c
--- /dev/null
+++ b/lucene/core/src/java/org/apache/lucene/search/BayesianScoreQuery.java
@@ -0,0 +1,236 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.search;
+
+import java.io.IOException;
+import java.util.Objects;
+import org.apache.lucene.index.LeafReaderContext;
+
+/**
+ * A query wrapper that transforms the inner query's score into a calibrated probability via sigmoid
+ * calibration: {@code P = sigmoid(alpha * (score - beta))}.
+ *
+ * <p>This implements the query-level Bayesian transform from "Bayesian BM25": the inner query
+ * (typically a multi-term BooleanQuery with BM25Similarity) produces a raw score, and this wrapper
+ * maps it to a probability in (0, 1) suitable for combination with other probability signals via
+ * {@link LogOddsFusionQuery}.
+ *
+ * <p>The alpha parameter controls the sigmoid steepness (score sensitivity), and beta controls the
+ * midpoint (decision boundary). These can be set manually or estimated from the score distribution.
+ *
+ * @lucene.experimental
+ */
+public final class BayesianScoreQuery extends Query {
+
+ private final Query query;
+ private final float alpha;
+ private final float beta;
+
+ /**
+ * Creates a BayesianScoreQuery.
+ *
+ * @param query the inner query whose scores will be transformed
+ * @param alpha sigmoid steepness (must be positive and finite)
+ * @param beta sigmoid midpoint (must be finite)
+ */
+ public BayesianScoreQuery(Query query, float alpha, float beta) {
+ this.query = Objects.requireNonNull(query);
+ if (Float.isFinite(alpha) == false || alpha <= 0) {
+ throw new IllegalArgumentException("alpha must be a positive finite value, got " + alpha);
+ }
+ if (Float.isFinite(beta) == false) {
+ throw new IllegalArgumentException("beta must be a finite value, got " + beta);
+ }
+ this.alpha = alpha;
+ this.beta = beta;
+ }
+
+ /** Returns the wrapped query. */
+ public Query getQuery() {
+ return query;
+ }
+
+ /** Returns the sigmoid steepness parameter. */
+ public float getAlpha() {
+ return alpha;
+ }
+
+ /** Returns the sigmoid midpoint parameter. */
+ public float getBeta() {
+ return beta;
+ }
+
+ static float sigmoid(float x) {
+ if (x >= 0) {
+ return (float) (1.0 / (1.0 + Math.exp(-x)));
+ } else {
+ double expX = Math.exp(x);
+ return (float) (expX / (1.0 + expX));
+ }
+ }
+
+ @Override
+ public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float boost)
+ throws IOException {
+ Weight innerWeight = query.createWeight(searcher, scoreMode, boost);
+ if (scoreMode.needsScores() == false) {
+ return innerWeight;
+ }
+ return new BayesianScoreWeight(this, innerWeight);
+ }
+
+ @Override
+ public Query rewrite(IndexSearcher indexSearcher) throws IOException {
+ Query rewritten = query.rewrite(indexSearcher);
+ if (rewritten instanceof MatchNoDocsQuery) {
+ return rewritten;
+ }
+ if (rewritten != query) {
+ return new BayesianScoreQuery(rewritten, alpha, beta);
+ }
+ return super.rewrite(indexSearcher);
+ }
+
+ @Override
+ public void visit(QueryVisitor visitor) {
+ query.visit(visitor.getSubVisitor(BooleanClause.Occur.MUST, this));
+ }
+
+ @Override
+ public String toString(String field) {
+ return "BayesianScore(" + query.toString(field) + ", alpha=" + alpha + ", beta=" + beta + ")";
+ }
+
+ @Override
+ public boolean equals(Object other) {
+ return sameClassAs(other) && equalsTo(getClass().cast(other));
+ }
+
+ private boolean equalsTo(BayesianScoreQuery other) {
+ return query.equals(other.query)
+ && Float.floatToIntBits(alpha) == Float.floatToIntBits(other.alpha)
+ && Float.floatToIntBits(beta) == Float.floatToIntBits(other.beta);
+ }
+
+ @Override
+ public int hashCode() {
+ int h = classHash();
+ h = 31 * h + query.hashCode();
+ h = 31 * h + Float.floatToIntBits(alpha);
+ h = 31 * h + Float.floatToIntBits(beta);
+ return h;
+ }
+
+ private class BayesianScoreWeight extends Weight {
+ private final Weight innerWeight;
+
+ BayesianScoreWeight(Query query, Weight innerWeight) {
+ super(query);
+ this.innerWeight = innerWeight;
+ }
+
+ @Override
+ public Matches matches(LeafReaderContext context, int doc) throws IOException {
+ return innerWeight.matches(context, doc);
+ }
+
+ @Override
+ public Explanation explain(LeafReaderContext context, int doc) throws IOException {
+ Explanation innerExpl = innerWeight.explain(context, doc);
+ if (innerExpl.isMatch() == false) {
+ return innerExpl;
+ }
+ float innerScore = innerExpl.getValue().floatValue();
+ float transformed = sigmoid(alpha * (innerScore - beta));
+ return Explanation.match(
+ transformed,
+ "sigmoid calibration, computed as sigmoid(alpha * (score - beta)) from:",
+ innerExpl,
+ Explanation.match(alpha, "alpha, sigmoid steepness"),
+ Explanation.match(beta, "beta, sigmoid midpoint"));
+ }
+
+ @Override
+ public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException {
+ ScorerSupplier innerSupplier = innerWeight.scorerSupplier(context);
+ if (innerSupplier == null) {
+ return null;
+ }
+ return new ScorerSupplier() {
+ @Override
+ public Scorer get(long leadCost) throws IOException {
+ Scorer innerScorer = innerSupplier.get(leadCost);
+ return new BayesianScoreScorer(innerScorer);
+ }
+
+ @Override
+ public long cost() {
+ return innerSupplier.cost();
+ }
+
+ @Override
+ public void setTopLevelScoringClause() {
+ innerSupplier.setTopLevelScoringClause();
+ }
+ };
+ }
+
+ @Override
+ public boolean isCacheable(LeafReaderContext ctx) {
+ return innerWeight.isCacheable(ctx);
+ }
+ }
+
+ private class BayesianScoreScorer extends FilterScorer {
+
+ BayesianScoreScorer(Scorer in) {
+ super(in);
+ }
+
+ @Override
+ public float score() throws IOException {
+ float innerScore = in.score();
+ return sigmoid(alpha * (innerScore - beta));
+ }
+
+ @Override
+ public int advanceShallow(int target) throws IOException {
+ return in.advanceShallow(target);
+ }
+
+ @Override
+ public float getMaxScore(int upTo) throws IOException {
+ float innerMax = in.getMaxScore(upTo);
+ // sigmoid is monotone, so max(sigmoid(f(x))) = sigmoid(max(f(x)))
+ return sigmoid(alpha * (innerMax - beta));
+ }
+
+ @Override
+ public void setMinCompetitiveScore(float minScore) throws IOException {
+ // Invert the sigmoid to get the minimum inner score needed:
+ // minScore = sigmoid(alpha * (innerScore - beta))
+ // => alpha * (innerScore - beta) = logit(minScore)
+ // => innerScore = logit(minScore) / alpha + beta
+ if (minScore > 0f && minScore < 1f) {
+ float clamped = Math.max(1e-7f, Math.min(1f - 1e-7f, minScore));
+ float logitMin = (float) Math.log(clamped / (1f - clamped));
+ float innerMin = logitMin / alpha + beta;
+ in.setMinCompetitiveScore(Math.max(0f, innerMin));
+ }
+ }
+ }
+}
diff --git a/lucene/core/src/java/org/apache/lucene/search/LogOddsFusionQuery.java b/lucene/core/src/java/org/apache/lucene/search/LogOddsFusionQuery.java
new file mode 100644
index 00000000000..ad1d26179cc
--- /dev/null
+++ b/lucene/core/src/java/org/apache/lucene/search/LogOddsFusionQuery.java
@@ -0,0 +1,289 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.search;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
+import org.apache.lucene.index.LeafReaderContext;
+
+/**
+ * A query that combines sub-query probability scores via log-odds fusion. Sub-queries are expected
+ * to produce scores in (0, 1) representing probabilities (e.g., from {@link BayesianScoreQuery}
+ * wrapping a BM25 query, or KNN cosine similarity).
+ *
+ * <p>The combination formula resolves the shrinkage problem of naive probabilistic AND by:
+ *
+ * <ol>
+ * <li>Converting each sub-score to log-odds: logit(p) = log(p / (1 - p))
+ * <li>Computing the mean log-odds across all clauses (non-matching contribute 0 = neutral)
+ * <li>Applying multiplicative confidence scaling: meanLogit * n^alpha
+ * <li>Converting back to probability via sigmoid
+ * </ol>
+ *
+ * <p>The alpha parameter controls the confidence scaling exponent. The default alpha=0.5 implements
+ * the sqrt(n) scaling law from "From Bayesian Inference to Neural Computation".
+ *
+ * @see LogOddsFusionScorer
+ * @lucene.experimental
+ */
+public final class LogOddsFusionQuery extends Query implements Iterable<Query> {
+
+ private final Multiset<Query> clauses = new Multiset<>();
+ private final List<Query> orderedClauses;
+ private final float alpha;
+
+ /**
+ * Creates a new LogOddsFusionQuery.
+ *
+ * @param clauses the sub-queries to combine
+ * @param alpha confidence scaling exponent (0.5 = sqrt(n) law)
+ * @throws IllegalArgumentException if alpha is not in [0, 1]
+ */
+ public LogOddsFusionQuery(Collection<? extends Query> clauses, float alpha) {
+ Objects.requireNonNull(clauses, "Collection of Queries must not be null");
+ if (Float.isNaN(alpha) || alpha < 0 || alpha > 1) {
+ throw new IllegalArgumentException("alpha must be in [0, 1], got " + alpha);
+ }
+ this.alpha = alpha;
+ this.clauses.addAll(clauses);
+ this.orderedClauses = new ArrayList<>(clauses);
+ }
+
+ /**
+ * Creates a new LogOddsFusionQuery with default alpha=0.5 (sqrt(n) scaling law).
+ *
+ * @param clauses the sub-queries to combine
+ */
+ public LogOddsFusionQuery(Collection<? extends Query> clauses) {
+ this(clauses, 0.5f);
+ }
+
+ @Override
+ public Iterator<Query> iterator() {
+ return getClauses().iterator();
+ }
+
+ /** Returns the clauses. */
+ public Collection<Query> getClauses() {
+ return Collections.unmodifiableCollection(orderedClauses);
+ }
+
+ /** Returns the alpha (confidence scaling exponent). */
+ public float getAlpha() {
+ return alpha;
+ }
+
+ /** Weight for LogOddsFusionQuery. */
+ protected class LogOddsFusionWeight extends Weight {
+
+ protected final ArrayList<Weight> weights = new ArrayList<>();
+ private final ScoreMode scoreMode;
+
+ public LogOddsFusionWeight(IndexSearcher searcher, ScoreMode scoreMode, float boost)
+ throws IOException {
+ super(LogOddsFusionQuery.this);
+ for (Query clauseQuery : clauses) {
+ weights.add(searcher.createWeight(clauseQuery, scoreMode, boost));
+ }
+ this.scoreMode = scoreMode;
+ }
+
+ @Override
+ public Matches matches(LeafReaderContext context, int doc) throws IOException {
+ List<Matches> mis = new ArrayList<>();
+ for (Weight weight : weights) {
+ Matches mi = weight.matches(context, doc);
+ if (mi != null) {
+ mis.add(mi);
+ }
+ }
+ return MatchesUtils.fromSubMatches(mis);
+ }
+
+ @Override
+ public ScorerSupplier scorerSupplier(LeafReaderContext context) throws IOException {
+ List<ScorerSupplier> scorerSuppliers = new ArrayList<>();
+ for (Weight w : weights) {
+ ScorerSupplier ss = w.scorerSupplier(context);
+ if (ss != null) {
+ scorerSuppliers.add(ss);
+ }
+ }
+
+ if (scorerSuppliers.isEmpty()) {
+ return null;
+ } else if (scorerSuppliers.size() == 1) {
+ return scorerSuppliers.get(0);
+ } else {
+ final int totalClauses = clauses.size();
+ return new ScorerSupplier() {
+
+ private long cost = -1;
+
+ @Override
+ public Scorer get(long leadCost) throws IOException {
+ List<Scorer> scorers = new ArrayList<>();
+ for (ScorerSupplier ss : scorerSuppliers) {
+ scorers.add(ss.get(leadCost));
+ }
+ return new LogOddsFusionScorer(scorers, totalClauses, alpha, scoreMode, leadCost);
+ }
+
+ @Override
+ public long cost() {
+ if (cost == -1) {
+ long cost = 0;
+ for (ScorerSupplier ss : scorerSuppliers) {
+ cost += ss.cost();
+ }
+ this.cost = cost;
+ }
+ return cost;
+ }
+
+ @Override
+ public void setTopLevelScoringClause() {
+ for (ScorerSupplier ss : scorerSuppliers) {
+ ss.setTopLevelScoringClause();
+ }
+ }
+ };
+ }
+ }
+
+ @Override
+ public boolean isCacheable(LeafReaderContext ctx) {
+ if (weights.size()
+ > AbstractMultiTermQueryConstantScoreWrapper.BOOLEAN_REWRITE_TERM_COUNT_THRESHOLD) {
+ return false;
+ }
+ for (Weight w : weights) {
+ if (w.isCacheable(ctx) == false) return false;
+ }
+ return true;
+ }
+
+ @Override
+ public Explanation explain(LeafReaderContext context, int doc) throws IOException {
+ boolean match = false;
+ List<Explanation> subsOnMatch = new ArrayList<>();
+ List<Explanation> subsOnNoMatch = new ArrayList<>();
+ double logitSum = 0;
+ int totalClauses = weights.size();
+
+ for (Weight wt : weights) {
+ Explanation e = wt.explain(context, doc);
+ if (e.isMatch()) {
+ match = true;
+ subsOnMatch.add(e);
+ float subScore = e.getValue().floatValue();
+ logitSum += LogOddsFusionScorer.softplus(LogOddsFusionScorer.logit(subScore));
+ } else if (match == false) {
+ subsOnNoMatch.add(e);
+ }
+ }
+
+ if (match) {
+ // Non-matching contribute logit(0.5) = 0
+ float meanLogit = (float) (logitSum / totalClauses);
+ float scalingFactor = (float) Math.pow(totalClauses, alpha);
+ float scaledLogit = meanLogit * scalingFactor;
+ float score = LogOddsFusionScorer.sigmoid(scaledLogit);
+
+ return Explanation.match(
+ score, "log-odds fusion, computed as sigmoid(meanLogit * n^alpha) from:", subsOnMatch);
+ } else {
+ return Explanation.noMatch("No matching clause", subsOnNoMatch);
+ }
+ }
+ }
+
+ @Override
+ public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float boost)
+ throws IOException {
+ return new LogOddsFusionWeight(searcher, scoreMode, boost);
+ }
+
+ @Override
+ public Query rewrite(IndexSearcher indexSearcher) throws IOException {
+ if (clauses.isEmpty()) {
+ return new MatchNoDocsQuery("empty LogOddsFusionQuery");
+ }
+
+ if (clauses.size() == 1) {
+ return clauses.iterator().next();
+ }
+
+ boolean actuallyRewritten = false;
+ List<Query> rewrittenClauses = new ArrayList<>();
+ for (Query sub : clauses) {
+ Query rewrittenSub = sub.rewrite(indexSearcher);
+ actuallyRewritten |= rewrittenSub != sub;
+ rewrittenClauses.add(rewrittenSub);
+ }
+
+ if (actuallyRewritten) {
+ return new LogOddsFusionQuery(rewrittenClauses, alpha);
+ }
+
+ return super.rewrite(indexSearcher);
+ }
+
+ @Override
+ public void visit(QueryVisitor visitor) {
+ QueryVisitor v = visitor.getSubVisitor(BooleanClause.Occur.SHOULD, this);
+ for (Query q : clauses) {
+ q.visit(v);
+ }
+ }
+
+ @Override
+ public String toString(String field) {
+ return this.orderedClauses.stream()
+ .map(
+ subquery -> {
+ if (subquery instanceof BooleanQuery) {
+ return "(" + subquery.toString(field) + ")";
+ }
+ return subquery.toString(field);
+ })
+ .collect(Collectors.joining(" & ", "LogOdds(", ")^" + alpha));
+ }
+
+ @Override
+ public boolean equals(Object other) {
+ return sameClassAs(other) && equalsTo(getClass().cast(other));
+ }
+
+ private boolean equalsTo(LogOddsFusionQuery other) {
+ return alpha == other.alpha && Objects.equals(clauses, other.clauses);
+ }
+
+ @Override
+ public int hashCode() {
+ int h = classHash();
+ h = 31 * h + Float.floatToIntBits(alpha);
+ h = 31 * h + Objects.hashCode(clauses);
+ return h;
+ }
+}
diff --git a/lucene/core/src/java/org/apache/lucene/search/LogOddsFusionScorer.java b/lucene/core/src/java/org/apache/lucene/search/LogOddsFusionScorer.java
new file mode 100644
index 00000000000..cb8ee8aa46d
--- /dev/null
+++ b/lucene/core/src/java/org/apache/lucene/search/LogOddsFusionScorer.java
@@ -0,0 +1,157 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.search;
+
+import java.io.IOException;
+import java.util.List;
+
+/**
+ * Scorer for {@link LogOddsFusionQuery}. Combines sub-scorer outputs (assumed to be probabilities
+ * in (0, 1)) via log-odds fusion with multiplicative confidence scaling.
+ *
+ * <p>The scoring formula is:
+ *
+ * <pre>
+ * gatedLogit = softplus(logit(clamp(subScore))) for each matching sub-scorer
+ * logitSum = sum of gatedLogit values
+ * meanLogit = logitSum / n (n = total clause count, not just matching)
+ * scaledLogit = meanLogit * pow(n, alpha)
+ * score = sigmoid(scaledLogit)
+ * </pre>
+ *
+ * <p>Softplus gating ({@code log(1 + exp(x))}) is applied to logit values before aggregation. This
+ * distinguishes "absence of evidence" (non-matching sub-scorer, contributes 0) from "evidence of
+ * absence" (matching sub-scorer with weak probability, contributes a small positive value). A
+ * matching sub-scorer always contributes more than a non-matching one, preserving the ordering
+ * among weak matches while ensuring that no match is ever penalized.
+ *
+ * <p>Non-matching sub-scorers contribute logit(0.5) = 0 (neutral evidence).
+ *
+ * @lucene.experimental
+ */
+final class LogOddsFusionScorer extends DisjunctionScorer {
+ private static final float CLAMP_MIN = 1e-7f;
+ private static final float CLAMP_MAX = 1f - 1e-7f;
+
+ private final List<Scorer> subScorers;
+ private final int totalClauses;
+ private final float scalingFactor;
+
+ private final DisjunctionScoreBlockBoundaryPropagator disjunctionBlockPropagator;
+
+ /**
+ * Creates a new LogOddsFusionScorer.
+ *
+ * @param subScorers the sub scorers to combine
+ * @param totalClauses the total number of clauses (including non-matching)
+ * @param alpha confidence scaling exponent (0.5 = sqrt(n) law)
+ * @param scoreMode the score mode
+ * @param leadCost the lead cost for iteration
+ */
+ LogOddsFusionScorer(
+ List<Scorer> subScorers, int totalClauses, float alpha, ScoreMode scoreMode, long leadCost)
+ throws IOException {
+ super(subScorers, scoreMode, leadCost);
+ this.subScorers = subScorers;
+ this.totalClauses = totalClauses;
+ this.scalingFactor = (float) Math.pow(totalClauses, alpha);
+ if (scoreMode == ScoreMode.TOP_SCORES) {
+ this.disjunctionBlockPropagator = new DisjunctionScoreBlockBoundaryPropagator(subScorers);
+ } else {
+ this.disjunctionBlockPropagator = null;
+ }
+ }
+
+ static float clampProbability(float p) {
+ return Math.clamp(p, CLAMP_MIN, CLAMP_MAX);
+ }
+
+ static float logit(float p) {
+ float clamped = clampProbability(p);
+ return (float) Math.log(clamped / (1f - clamped));
+ }
+
+ static float sigmoid(float x) {
+ if (x >= 0) {
+ return (float) (1.0 / (1.0 + Math.exp(-x)));
+ } else {
+ double expX = Math.exp(x);
+ return (float) (expX / (1.0 + expX));
+ }
+ }
+
+ /**
+ * Softplus function: log(1 + exp(x)). Always positive, smooth approximation of ReLU. For large
+ * positive x, approaches x. For large negative x, approaches 0 from above. At x=0, returns log(2)
+ * ~ 0.693.
+ *
+ * <p>Uses a numerically stable formulation: for x > 20, softplus(x) ~ x.
+ */
+ static float softplus(float x) {
+ if (x > 20f) {
+ return x;
+ }
+ return (float) Math.log1p(Math.exp(x));
+ }
+
+ @Override
+ protected float score(DisiWrapper topList) throws IOException {
+ double logitSum = 0;
+ for (DisiWrapper w = topList; w != null; w = w.next) {
+ float subScore = w.scorable.score();
+ logitSum += softplus(logit(subScore));
+ }
+ // Non-matching sub-scorers contribute logit(0.5) = 0, softplus(0) = log(2) ~ 0.693.
+ // But we do NOT add this for non-matching scorers: they contribute 0, not softplus(0).
+ // This is the key distinction: a match always contributes softplus(logit) > 0,
+ // while a non-match contributes exactly 0.
+ float meanLogit = (float) (logitSum / totalClauses);
+ float scaledLogit = meanLogit * scalingFactor;
+ return sigmoid(scaledLogit);
+ }
+
+ @Override
+ public int advanceShallow(int target) throws IOException {
+ if (disjunctionBlockPropagator != null) {
+ return disjunctionBlockPropagator.advanceShallow(target);
+ }
+ return super.advanceShallow(target);
+ }
+
+ @Override
+ public float getMaxScore(int upTo) throws IOException {
+ double maxLogitSum = 0;
+ for (Scorer scorer : subScorers) {
+ if (scorer.docID() <= upTo) {
+ float maxSubScore = scorer.getMaxScore(upTo);
+ maxLogitSum += softplus(logit(maxSubScore));
+ }
+ }
+ // Safe upper bound: softplus(logit) is monotone in p, sum of upper bounds >= sum of actuals,
+ // sigmoid is monotone
+ float meanLogit = (float) (maxLogitSum / totalClauses);
+ float scaledLogit = meanLogit * scalingFactor;
+ return sigmoid(scaledLogit);
+ }
+
+ @Override
+ public void setMinCompetitiveScore(float minScore) throws IOException {
+ if (disjunctionBlockPropagator != null) {
+ disjunctionBlockPropagator.setMinCompetitiveScore(minScore);
+ }
+ }
+}
diff --git a/lucene/core/src/test/org/apache/lucene/search/TestBayesianScoreQuery.java b/lucene/core/src/test/org/apache/lucene/search/TestBayesianScoreQuery.java
new file mode 100644
index 00000000000..6cc086ceb30
--- /dev/null
+++ b/lucene/core/src/test/org/apache/lucene/search/TestBayesianScoreQuery.java
@@ -0,0 +1,260 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.search;
+
+import org.apache.lucene.document.Document;
+import org.apache.lucene.document.Field;
+import org.apache.lucene.document.TextField;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexReader;
+import org.apache.lucene.index.IndexWriter;
+import org.apache.lucene.index.IndexWriterConfig;
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.index.Term;
+import org.apache.lucene.search.similarities.BM25Similarity;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.tests.search.CheckHits;
+import org.apache.lucene.tests.search.QueryUtils;
+import org.apache.lucene.tests.util.LuceneTestCase;
+
+/** Tests for {@link BayesianScoreQuery}. */
+public class TestBayesianScoreQuery extends LuceneTestCase {
+
+ private Directory dir;
+ private IndexReader reader;
+ private IndexSearcher searcher;
+
+ @Override
+ public void setUp() throws Exception {
+ super.setUp();
+
+ dir = newDirectory();
+ IndexWriter writer = new IndexWriter(dir, new IndexWriterConfig());
+
+ // doc0: high tf for "alpha"
+ Document doc0 = new Document();
+ doc0.add(new TextField("body", "alpha alpha alpha alpha alpha beta", Field.Store.NO));
+ writer.addDocument(doc0);
+
+ // doc1: low tf for "alpha"
+ Document doc1 = new Document();
+ doc1.add(new TextField("body", "alpha gamma delta epsilon", Field.Store.NO));
+ writer.addDocument(doc1);
+
+ // doc2: no "alpha"
+ Document doc2 = new Document();
+ doc2.add(new TextField("body", "beta gamma delta epsilon zeta", Field.Store.NO));
+ writer.addDocument(doc2);
+
+ // Filler docs for meaningful IDF
+ for (int i = 0; i < 50; i++) {
+ Document filler = new Document();
+ filler.add(new TextField("body", "filler content number " + i, Field.Store.NO));
+ writer.addDocument(filler);
+ }
+
+ writer.forceMerge(1);
+ reader = DirectoryReader.open(writer);
+ writer.close();
+ searcher = new IndexSearcher(reader);
+ searcher.setSimilarity(new BM25Similarity());
+ }
+
+ @Override
+ public void tearDown() throws Exception {
+ reader.close();
+ dir.close();
+ super.tearDown();
+ }
+
+ public void testScoresInProbabilityRange() throws Exception {
+ Query inner = new TermQuery(new Term("body", "alpha"));
+ BayesianScoreQuery bsq = new BayesianScoreQuery(inner, 0.5f, 5.0f);
+
+ ScoreDoc[] hits = searcher.search(bsq, 10).scoreDocs;
+ assertTrue("should have hits", hits.length > 0);
+
+ for (ScoreDoc hit : hits) {
+ assertTrue("score should be > 0, got " + hit.score, hit.score > 0f);
+ assertTrue("score should be < 1, got " + hit.score, hit.score < 1f);
+ }
+ }
+
+ public void testRankingPreserved() throws Exception {
+ // BM25 ranking for "alpha": doc0 (high tf) > doc1 (low tf)
+ Query bm25Query = new TermQuery(new Term("body", "alpha"));
+ ScoreDoc[] bm25Hits = searcher.search(bm25Query, 10).scoreDocs;
+ assertEquals("doc0 should rank first in BM25", 0, bm25Hits[0].doc);
+ assertEquals("doc1 should rank second in BM25", 1, bm25Hits[1].doc);
+
+ // BayesianScoreQuery should preserve this ranking (sigmoid is monotone)
+ BayesianScoreQuery bsq = new BayesianScoreQuery(bm25Query, 0.5f, 5.0f);
+ ScoreDoc[] bsqHits = searcher.search(bsq, 10).scoreDocs;
+ assertEquals("doc0 should still rank first", 0, bsqHits[0].doc);
+ assertEquals("doc1 should still rank second", 1, bsqHits[1].doc);
+ assertTrue(
+ "doc0 (" + bsqHits[0].score + ") > doc1 (" + bsqHits[1].score + ")",
+ bsqHits[0].score > bsqHits[1].score);
+ }
+
+ public void testMultiTermQueryRankingPreserved() throws Exception {
+ // Multi-term BooleanQuery
+ BooleanQuery bm25Query =
+ new BooleanQuery.Builder()
+ .add(new TermQuery(new Term("body", "alpha")), BooleanClause.Occur.SHOULD)
+ .add(new TermQuery(new Term("body", "beta")), BooleanClause.Occur.SHOULD)
+ .build();
+
+ ScoreDoc[] bm25Hits = searcher.search(bm25Query, 10).scoreDocs;
+
+ // Wrap with BayesianScoreQuery
+ BayesianScoreQuery bsq = new BayesianScoreQuery(bm25Query, 0.3f, 8.0f);
+ ScoreDoc[] bsqHits = searcher.search(bsq, 10).scoreDocs;
+
+ // Same number of hits
+ assertEquals("same number of hits", bm25Hits.length, bsqHits.length);
+
+ // Same document ordering (sigmoid preserves ranking)
+ for (int i = 0; i < bm25Hits.length; i++) {
+ assertEquals("ranking should be preserved at position " + i, bm25Hits[i].doc, bsqHits[i].doc);
+ }
+
+ // All BayesianScore outputs in (0, 1)
+ for (ScoreDoc hit : bsqHits) {
+ assertTrue("score should be > 0", hit.score > 0f);
+ assertTrue("score should be < 1", hit.score < 1f);
+ }
+ }
+
+ public void testSigmoidProperties() throws Exception {
+ // sigmoid(0) = 0.5
+ assertEquals(0.5f, BayesianScoreQuery.sigmoid(0f), 1e-6f);
+
+ // sigmoid is monotone
+ float prev = BayesianScoreQuery.sigmoid(-50f);
+ for (float x = -49f; x <= 50f; x += 1f) {
+ float curr = BayesianScoreQuery.sigmoid(x);
+ assertTrue("sigmoid should be monotone", curr >= prev);
+ prev = curr;
+ }
+
+ // sigmoid is numerically stable at extremes
+ float large = BayesianScoreQuery.sigmoid(100f);
+ assertTrue("sigmoid(100) near 1", large > 0.99f && large <= 1.0f);
+ float small = BayesianScoreQuery.sigmoid(-100f);
+ assertTrue("sigmoid(-100) near 0", small >= 0f && small < 0.01f);
+ }
+
+ public void testIllegalParameters() {
+ Query q = new TermQuery(new Term("body", "alpha"));
+
+ expectThrows(IllegalArgumentException.class, () -> new BayesianScoreQuery(q, -1f, 5f));
+ expectThrows(IllegalArgumentException.class, () -> new BayesianScoreQuery(q, 0f, 5f));
+ expectThrows(
+ IllegalArgumentException.class,
+ () -> new BayesianScoreQuery(q, Float.POSITIVE_INFINITY, 5f));
+ expectThrows(IllegalArgumentException.class, () -> new BayesianScoreQuery(q, 1f, Float.NaN));
+ }
+
+ public void testRewrite() throws Exception {
+ Query inner = new TermQuery(new Term("body", "alpha"));
+ BayesianScoreQuery bsq = new BayesianScoreQuery(inner, 0.5f, 5.0f);
+
+ Query rewritten = searcher.rewrite(bsq);
+ // Should rewrite to a BayesianScoreQuery wrapping the rewritten inner query
+ assertTrue(
+ "should rewrite to BayesianScoreQuery, got " + rewritten.getClass().getName(),
+ rewritten instanceof BayesianScoreQuery);
+ }
+
+ public void testEqualsAndHashCode() {
+ Query q1 = new TermQuery(new Term("body", "alpha"));
+ Query q2 = new TermQuery(new Term("body", "beta"));
+
+ BayesianScoreQuery a = new BayesianScoreQuery(q1, 0.5f, 5.0f);
+ BayesianScoreQuery b = new BayesianScoreQuery(q1, 0.5f, 5.0f);
+ BayesianScoreQuery c = new BayesianScoreQuery(q1, 0.3f, 5.0f);
+ BayesianScoreQuery d = new BayesianScoreQuery(q2, 0.5f, 5.0f);
+
+ assertEquals(a, b);
+ assertEquals(a.hashCode(), b.hashCode());
+ assertNotEquals(a, c);
+ assertNotEquals(a, d);
+ }
+
+ public void testExplanation() throws Exception {
+ Query inner = new TermQuery(new Term("body", "alpha"));
+ BayesianScoreQuery bsq = new BayesianScoreQuery(inner, 0.5f, 5.0f);
+
+ Weight w = searcher.createWeight(searcher.rewrite(bsq), ScoreMode.COMPLETE, 1);
+ LeafReaderContext context = searcher.getIndexReader().leaves().get(0);
+
+ // doc0 matches
+ Explanation expl = w.explain(context, 0);
+ assertTrue("should match", expl.isMatch());
+ assertTrue("should mention sigmoid", expl.getDescription().contains("sigmoid"));
+ float score = expl.getValue().floatValue();
+ assertTrue("score in (0,1)", score > 0f && score < 1f);
+
+ // doc2 does not match "alpha"
+ Explanation noMatch = w.explain(context, 2);
+ assertFalse("should not match", noMatch.isMatch());
+ }
+
+ public void testQueryUtils() throws Exception {
+ Query inner = new TermQuery(new Term("body", "alpha"));
+ BayesianScoreQuery bsq = new BayesianScoreQuery(inner, 0.5f, 5.0f);
+ QueryUtils.check(random(), bsq, searcher);
+ }
+
+ public void testMaxScoreCorrectness() throws Exception {
+ Query inner = new TermQuery(new Term("body", "alpha"));
+ BayesianScoreQuery bsq = new BayesianScoreQuery(inner, 0.5f, 5.0f);
+ CheckHits.checkTopScores(random(), bsq, searcher);
+ }
+
+ public void testToString() {
+ Query inner = new TermQuery(new Term("body", "alpha"));
+ BayesianScoreQuery bsq = new BayesianScoreQuery(inner, 0.5f, 5.0f);
+ String s = bsq.toString("body");
+ assertTrue("should contain BayesianScore", s.contains("BayesianScore"));
+ assertTrue("should contain alpha", s.contains("alpha"));
+ }
+
+ public void testDifferentAlphaBeta() throws Exception {
+ Query inner = new TermQuery(new Term("body", "alpha"));
+
+ // Gentle calibration: scores close to 0.5
+ BayesianScoreQuery gentle = new BayesianScoreQuery(inner, 0.1f, 5.0f);
+ ScoreDoc[] gentleHits = searcher.search(gentle, 10).scoreDocs;
+
+ // Steep calibration: scores pushed toward 0 or 1
+ BayesianScoreQuery steep = new BayesianScoreQuery(inner, 2.0f, 5.0f);
+ ScoreDoc[] steepHits = searcher.search(steep, 10).scoreDocs;
+
+ // Both should produce valid probabilities
+ for (ScoreDoc hit : gentleHits) {
+ assertTrue("gentle score in (0,1)", hit.score > 0f && hit.score < 1f);
+ }
+ for (ScoreDoc hit : steepHits) {
+ assertTrue("steep score in (0,1)", hit.score > 0f && hit.score < 1f);
+ }
+
+ // Same ranking order (sigmoid is monotone regardless of alpha/beta)
+ assertEquals("same top doc", gentleHits[0].doc, steepHits[0].doc);
+ }
+}
diff --git a/lucene/core/src/test/org/apache/lucene/search/TestLogOddsFusionQuery.java b/lucene/core/src/test/org/apache/lucene/search/TestLogOddsFusionQuery.java
new file mode 100644
index 00000000000..d3f1f9c653e
--- /dev/null
+++ b/lucene/core/src/test/org/apache/lucene/search/TestLogOddsFusionQuery.java
@@ -0,0 +1,684 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.search;
+
+import java.util.Arrays;
+import java.util.Collections;
+import org.apache.lucene.document.Document;
+import org.apache.lucene.document.Field;
+import org.apache.lucene.document.KnnFloatVectorField;
+import org.apache.lucene.document.TextField;
+import org.apache.lucene.index.DirectoryReader;
+import org.apache.lucene.index.IndexReader;
+import org.apache.lucene.index.IndexWriter;
+import org.apache.lucene.index.IndexWriterConfig;
+import org.apache.lucene.index.LeafReaderContext;
+import org.apache.lucene.index.Term;
+import org.apache.lucene.index.VectorSimilarityFunction;
+import org.apache.lucene.search.similarities.BM25Similarity;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.tests.search.CheckHits;
+import org.apache.lucene.tests.search.QueryUtils;
+import org.apache.lucene.tests.util.LuceneTestCase;
+import org.apache.lucene.tests.util.TestUtil;
+
+/**
+ * Tests for {@link LogOddsFusionQuery}. Clauses are wrapped with {@link BayesianScoreQuery} to
+ * produce (0,1) probability scores, matching the paper's query-level calibration design.
+ */
+public class TestLogOddsFusionQuery extends LuceneTestCase {
+
+ private static final float BSQ_ALPHA = 0.5f;
+ private static final float BSQ_BETA = 3.0f;
+
+ /** Wraps a query with BayesianScoreQuery for sigmoid calibration to (0,1). */
+ private static Query bayesian(Query q) {
+ return new BayesianScoreQuery(q, BSQ_ALPHA, BSQ_BETA);
+ }
+
+ private Directory dir;
+ private IndexReader reader;
+ private IndexSearcher searcher;
+
+ @Override
+ public void setUp() throws Exception {
+ super.setUp();
+
+ dir = newDirectory();
+ IndexWriter writer = new IndexWriter(dir, new IndexWriterConfig());
+
+ Document doc0 = new Document();
+ doc0.add(new TextField("body", "alpha beta gamma", Field.Store.YES));
+ writer.addDocument(doc0);
+
+ Document doc1 = new Document();
+ doc1.add(new TextField("body", "alpha gamma delta", Field.Store.YES));
+ writer.addDocument(doc1);
+
+ Document doc2 = new Document();
+ doc2.add(new TextField("body", "beta gamma delta", Field.Store.YES));
+ writer.addDocument(doc2);
+
+ Document doc3 = new Document();
+ doc3.add(new TextField("body", "gamma delta epsilon", Field.Store.YES));
+ writer.addDocument(doc3);
+
+ writer.forceMerge(1);
+ reader = DirectoryReader.open(writer);
+ writer.close();
+ searcher = new IndexSearcher(reader);
+ searcher.setSimilarity(new BM25Similarity());
+ }
+
+ @Override
+ public void tearDown() throws Exception {
+ reader.close();
+ dir.close();
+ super.tearDown();
+ }
+
+ // ---- Core query behavior ----
+
+ public void testBasicFusion() throws Exception {
+ Query q1 = bayesian(new TermQuery(new Term("body", "alpha")));
+ Query q2 = bayesian(new TermQuery(new Term("body", "beta")));
+
+ LogOddsFusionQuery loq = new LogOddsFusionQuery(Arrays.asList(q1, q2), 0.5f);
+ QueryUtils.check(random(), loq, searcher);
+
+ ScoreDoc[] hits = searcher.search(loq, 10).scoreDocs;
+ assertTrue("should have at least 1 hit", hits.length >= 1);
+
+ for (ScoreDoc hit : hits) {
+ assertTrue("score should be > 0, got " + hit.score, hit.score > 0);
+ assertTrue("score should be < 1, got " + hit.score, hit.score < 1);
+ }
+
+ if (hits.length >= 2) {
+ assertTrue(
+ "doc matching both terms should score higher than doc matching one",
+ hits[0].score >= hits[1].score);
+ }
+ }
+
+ public void testSingleClauseRewrite() throws Exception {
+ Query sub = bayesian(new TermQuery(new Term("body", "alpha")));
+ LogOddsFusionQuery loq = new LogOddsFusionQuery(Collections.singletonList(sub));
+
+ Query rewritten = searcher.rewrite(loq);
+ assertTrue(
+ "single clause should rewrite to BayesianScoreQuery, got " + rewritten.getClass().getName(),
+ rewritten instanceof BayesianScoreQuery);
+ }
+
+ public void testEmptyClauseRewrite() throws Exception {
+ LogOddsFusionQuery loq = new LogOddsFusionQuery(Collections.emptyList(), 0.5f);
+ Query rewritten = searcher.rewrite(loq);
+ assertTrue("empty should rewrite to MatchNoDocsQuery", rewritten instanceof MatchNoDocsQuery);
+ }
+
+ public void testAlphaValues() throws Exception {
+ Query q1 = bayesian(new TermQuery(new Term("body", "alpha")));
+ Query q2 = bayesian(new TermQuery(new Term("body", "beta")));
+
+ for (float alpha : new float[] {0.0f, 0.5f, 1.0f}) {
+ LogOddsFusionQuery loq = new LogOddsFusionQuery(Arrays.asList(q1, q2), alpha);
+ ScoreDoc[] hits = searcher.search(loq, 10).scoreDocs;
+ assertTrue("alpha=" + alpha + " should have hits", hits.length > 0);
+ for (ScoreDoc hit : hits) {
+ assertTrue("score should be finite and > 0", Float.isFinite(hit.score) && hit.score > 0);
+ assertTrue("score should be < 1", hit.score < 1);
+ }
+ }
+ }
+
+ public void testIllegalAlpha() {
+ expectThrows(
+ IllegalArgumentException.class,
+ () -> new LogOddsFusionQuery(Collections.emptyList(), -0.1f));
+ expectThrows(
+ IllegalArgumentException.class,
+ () -> new LogOddsFusionQuery(Collections.emptyList(), 1.1f));
+ expectThrows(
+ IllegalArgumentException.class,
+ () -> new LogOddsFusionQuery(Collections.emptyList(), Float.NaN));
+ }
+
+ public void testNonMatchingSubQueries() throws Exception {
+ Query q1 = bayesian(new TermQuery(new Term("body", "nonexistent1")));
+ Query q2 = bayesian(new TermQuery(new Term("body", "nonexistent2")));
+ LogOddsFusionQuery loq = new LogOddsFusionQuery(Arrays.asList(q1, q2), 0.5f);
+ assertEquals("no docs should match", 0, searcher.search(loq, 10).scoreDocs.length);
+ }
+
+ public void testExplanation() throws Exception {
+ Query q1 = bayesian(new TermQuery(new Term("body", "alpha")));
+ Query q2 = bayesian(new TermQuery(new Term("body", "beta")));
+ LogOddsFusionQuery loq = new LogOddsFusionQuery(Arrays.asList(q1, q2), 0.5f);
+
+ Weight w = searcher.createWeight(searcher.rewrite(loq), ScoreMode.COMPLETE, 1);
+ LeafReaderContext context = searcher.getIndexReader().leaves().get(0);
+
+ Explanation expl = w.explain(context, 0);
+ assertTrue("doc0 should match", expl.isMatch());
+ assertTrue("should mention log-odds", expl.getDescription().contains("log-odds"));
+
+ Explanation noMatch = w.explain(context, 3);
+ assertFalse("doc3 should not match", noMatch.isMatch());
+ }
+
+ public void testEqualsAndHashCode() {
+ Query q1 = bayesian(new TermQuery(new Term("body", "alpha")));
+ Query q2 = bayesian(new TermQuery(new Term("body", "beta")));
+
+ LogOddsFusionQuery a = new LogOddsFusionQuery(Arrays.asList(q1, q2), 0.5f);
+ LogOddsFusionQuery b = new LogOddsFusionQuery(Arrays.asList(q1, q2), 0.5f);
+ LogOddsFusionQuery c = new LogOddsFusionQuery(Arrays.asList(q1, q2), 0.3f);
+ LogOddsFusionQuery d = new LogOddsFusionQuery(Arrays.asList(q2, q1), 0.5f);
+
+ assertEquals(a, b);
+ assertEquals(a.hashCode(), b.hashCode());
+ assertNotEquals(a, c);
+ assertEquals(a, d);
+ }
+
+ public void testQueryUtils() throws Exception {
+ Query q1 = bayesian(new TermQuery(new Term("body", "alpha")));
+ Query q2 = bayesian(new TermQuery(new Term("body", "beta")));
+ LogOddsFusionQuery loq = new LogOddsFusionQuery(Arrays.asList(q1, q2), 0.5f);
+ QueryUtils.check(random(), loq, searcher);
+ }
+
+ public void testMaxScoreCorrectness() throws Exception {
+ Query q1 = bayesian(new TermQuery(new Term("body", "alpha")));
+ Query q2 = bayesian(new TermQuery(new Term("body", "beta")));
+ LogOddsFusionQuery loq = new LogOddsFusionQuery(Arrays.asList(q1, q2), 0.5f);
+ CheckHits.checkTopScores(random(), loq, searcher);
+ }
+
+ public void testToString() {
+ Query q1 = new TermQuery(new Term("body", "alpha"));
+ Query q2 = new TermQuery(new Term("body", "beta"));
+ LogOddsFusionQuery loq = new LogOddsFusionQuery(Arrays.asList(q1, q2), 0.5f);
+ String str = loq.toString("body");
+ assertTrue("should contain LogOdds", str.contains("LogOdds"));
+ assertTrue("should contain alpha", str.contains("0.5"));
+ }
+
+ public void testThreeWayCombination() throws Exception {
+ Query q1 = bayesian(new TermQuery(new Term("body", "alpha")));
+ Query q2 = bayesian(new TermQuery(new Term("body", "beta")));
+ Query q3 = bayesian(new TermQuery(new Term("body", "gamma")));
+ LogOddsFusionQuery loq = new LogOddsFusionQuery(Arrays.asList(q1, q2, q3), 0.5f);
+ QueryUtils.check(random(), loq, searcher);
+
+ ScoreDoc[] hits = searcher.search(loq, 10).scoreDocs;
+ assertTrue("should have hits", hits.length > 0);
+ for (ScoreDoc hit : hits) {
+ assertTrue("score in (0,1)", hit.score > 0 && hit.score < 1);
+ }
+ }
+
+ // ---- Multi-field hybrid text tests ----
+
+ public void testMultiFieldHybridSearch() throws Exception {
+ Directory hybridDir = newDirectory();
+ IndexWriter writer = new IndexWriter(hybridDir, new IndexWriterConfig());
+
+ Document doc0 = new Document();
+ doc0.add(new TextField("title", "lucene search engine", Field.Store.YES));
+ doc0.add(new TextField("body", "lucene is a full-text search library", Field.Store.YES));
+ writer.addDocument(doc0);
+
+ Document doc1 = new Document();
+ doc1.add(new TextField("title", "lucene search engine", Field.Store.YES));
+ doc1.add(new TextField("body", "this is about information retrieval", Field.Store.YES));
+ writer.addDocument(doc1);
+
+ Document doc2 = new Document();
+ doc2.add(new TextField("title", "database systems overview", Field.Store.YES));
+ doc2.add(new TextField("body", "relational databases use SQL", Field.Store.YES));
+ writer.addDocument(doc2);
+
+ writer.forceMerge(1);
+ writer.close();
+ IndexReader hybridReader = DirectoryReader.open(hybridDir);
+ IndexSearcher hybridSearcher = new IndexSearcher(hybridReader);
+ hybridSearcher.setSimilarity(new BM25Similarity());
+
+ Query titleQuery = bayesian(new TermQuery(new Term("title", "lucene")));
+ Query bodyQuery = bayesian(new TermQuery(new Term("body", "lucene")));
+ LogOddsFusionQuery hybridQuery =
+ new LogOddsFusionQuery(Arrays.asList(titleQuery, bodyQuery), 0.5f);
+
+ ScoreDoc[] hits = hybridSearcher.search(hybridQuery, 10).scoreDocs;
+ assertEquals("should have 2 hits", 2, hits.length);
+
+ for (ScoreDoc hit : hits) {
+ assertTrue("score in (0,1)", hit.score > 0f && hit.score < 1f);
+ }
+
+ // doc0 matches both fields, doc1 matches title only
+ assertEquals("doc0 should rank first", 0, hits[0].doc);
+ assertTrue(
+ "doc0 (" + hits[0].score + ") > doc1 (" + hits[1].score + ")",
+ hits[0].score > hits[1].score);
+
+ hybridReader.close();
+ hybridDir.close();
+ }
+
+ public void testMoreSignalsIncreaseScore() throws Exception {
+ Directory signalDir = newDirectory();
+ IndexWriter writer = new IndexWriter(signalDir, new IndexWriterConfig());
+
+ Document doc = new Document();
+ doc.add(new TextField("f1", "search engine optimization", Field.Store.NO));
+ doc.add(new TextField("f2", "search algorithms and data structures", Field.Store.NO));
+ doc.add(new TextField("f3", "search relevance tuning", Field.Store.NO));
+ writer.addDocument(doc);
+
+ writer.forceMerge(1);
+ writer.close();
+ IndexReader signalReader = DirectoryReader.open(signalDir);
+ IndexSearcher signalSearcher = new IndexSearcher(signalReader);
+ signalSearcher.setSimilarity(new BM25Similarity());
+
+ Query q1 = bayesian(new TermQuery(new Term("f1", "search")));
+ Query q2 = bayesian(new TermQuery(new Term("f2", "search")));
+ Query q3 = bayesian(new TermQuery(new Term("f3", "search")));
+
+ float score1 = signalSearcher.search(q1, 1).scoreDocs[0].score;
+
+ LogOddsFusionQuery twoSignals = new LogOddsFusionQuery(Arrays.asList(q1, q2), 0.5f);
+ float score2 = signalSearcher.search(twoSignals, 1).scoreDocs[0].score;
+
+ LogOddsFusionQuery threeSignals = new LogOddsFusionQuery(Arrays.asList(q1, q2, q3), 0.5f);
+ float score3 = signalSearcher.search(threeSignals, 1).scoreDocs[0].score;
+
+ // With softplus gating, more matching signals always produce >= score.
+ assertTrue("3 >= 2 signals", score3 >= score2);
+ assertTrue("2 >= 1 signal", score2 >= score1);
+ assertTrue("all in (0,1)", score1 > 0 && score1 < 1 && score2 > 0 && score3 < 1);
+
+ signalReader.close();
+ signalDir.close();
+ }
+
+ // ---- Vector + Text hybrid search tests ----
+
+ private static final int HYBRID_NUM_FILLER = 196;
+ private static final float[] QUERY_A = normalize(new float[] {1.0f, 0.0f, 0.0f});
+
+ private Directory hybridDir;
+ private IndexReader hybridReader;
+ private IndexSearcher hybridSearcher;
+
+ private void buildHybridIndex() throws Exception {
+ hybridDir = newDirectory();
+ IndexWriterConfig config = new IndexWriterConfig();
+ config.setCodec(TestUtil.getDefaultCodec());
+ IndexWriter writer = new IndexWriter(hybridDir, config);
+
+ // doc0: text match + vector near queryA
+ Document doc0 = new Document();
+ doc0.add(new TextField("body", "lucene search engine optimization library", Field.Store.NO));
+ doc0.add(
+ new KnnFloatVectorField(
+ "embedding",
+ normalize(new float[] {0.95f, 0.05f, 0.0f}),
+ VectorSimilarityFunction.COSINE));
+ writer.addDocument(doc0);
+
+ // doc1: text match + vector far from queryA
+ Document doc1 = new Document();
+ doc1.add(new TextField("body", "lucene full text search library index", Field.Store.NO));
+ doc1.add(
+ new KnnFloatVectorField(
+ "embedding",
+ normalize(new float[] {0.0f, 0.0f, 1.0f}),
+ VectorSimilarityFunction.COSINE));
+ writer.addDocument(doc1);
+
+ // doc2: no text match + vector near queryA
+ Document doc2 = new Document();
+ doc2.add(new TextField("body", "unrelated topic about cooking recipes", Field.Store.NO));
+ doc2.add(
+ new KnnFloatVectorField(
+ "embedding",
+ normalize(new float[] {0.90f, 0.10f, 0.0f}),
+ VectorSimilarityFunction.COSINE));
+ writer.addDocument(doc2);
+
+ // doc3: neither
+ Document doc3 = new Document();
+ doc3.add(new TextField("body", "gardening tips for spring planting", Field.Store.NO));
+ doc3.add(
+ new KnnFloatVectorField(
+ "embedding",
+ normalize(new float[] {0.0f, 1.0f, 0.0f}),
+ VectorSimilarityFunction.COSINE));
+ writer.addDocument(doc3);
+
+ // Filler docs with vectors away from queryA
+ for (int i = 0; i < HYBRID_NUM_FILLER; i++) {
+ Document filler = new Document();
+ filler.add(new TextField("body", "filler document about topic " + i, Field.Store.NO));
+ float y = (float) Math.sin(i * 0.1);
+ float z = (float) Math.cos(i * 0.1);
+ filler.add(
+ new KnnFloatVectorField(
+ "embedding", normalize(new float[] {0.0f, y, z}), VectorSimilarityFunction.COSINE));
+ writer.addDocument(filler);
+ }
+
+ writer.forceMerge(1);
+ writer.close();
+ hybridReader = DirectoryReader.open(hybridDir);
+ hybridSearcher = new IndexSearcher(hybridReader);
+ hybridSearcher.setSimilarity(new BM25Similarity());
+ }
+
+ private void closeHybridIndex() throws Exception {
+ hybridReader.close();
+ hybridDir.close();
+ }
+
+ public void testTextAndVectorHybridSearch() throws Exception {
+ buildHybridIndex();
+ try {
+ // Text query wrapped with BayesianScoreQuery -> probability in (0,1)
+ Query textQuery = bayesian(new TermQuery(new Term("body", "lucene")));
+ // KNN COSINE already produces (0,1) scores via (1+cos)/2
+ Query vectorQuery = new KnnFloatVectorQuery("embedding", QUERY_A, 10);
+
+ LogOddsFusionQuery hybridQuery =
+ new LogOddsFusionQuery(Arrays.asList(textQuery, vectorQuery), 0.5f);
+
+ ScoreDoc[] hits = hybridSearcher.search(hybridQuery, 10).scoreDocs;
+ assertTrue("should have hits", hits.length >= 3);
+
+ for (ScoreDoc hit : hits) {
+ assertTrue("score > 0", hit.score > 0f);
+ assertTrue("score < 1", hit.score < 1f);
+ }
+
+ // doc0 matches both text and vector
+ assertEquals("doc0 (text+vector) should rank first", 0, hits[0].doc);
+ assertTrue("doc0 > second", hits[0].score > hits[1].score);
+ } finally {
+ closeHybridIndex();
+ }
+ }
+
+ public void testVectorAndTextBothContribute() throws Exception {
+ buildHybridIndex();
+ try {
+ Query textQuery = bayesian(new TermQuery(new Term("body", "lucene")));
+ Query vectorQuery = new KnnFloatVectorQuery("embedding", QUERY_A, 10);
+
+ // Verify KNN returns doc0 and doc2
+ ScoreDoc[] knnHits = hybridSearcher.search(vectorQuery, 10).scoreDocs;
+ boolean knnFoundDoc0 = false;
+ boolean knnFoundDoc2 = false;
+ for (ScoreDoc hit : knnHits) {
+ if (hit.doc == 0) knnFoundDoc0 = true;
+ if (hit.doc == 2) knnFoundDoc2 = true;
+ }
+ assertTrue("KNN should return doc0", knnFoundDoc0);
+ assertTrue("KNN should return doc2", knnFoundDoc2);
+
+ LogOddsFusionQuery hybridQuery =
+ new LogOddsFusionQuery(Arrays.asList(textQuery, vectorQuery), 0.5f);
+ ScoreDoc[] hits = hybridSearcher.search(hybridQuery, 20).scoreDocs;
+
+ boolean foundTextOnly = false;
+ boolean foundVectorOnly = false;
+ for (ScoreDoc hit : hits) {
+ if (hit.doc == 1) foundTextOnly = true;
+ if (hit.doc == 2) foundVectorOnly = true;
+ }
+ assertTrue("text-only doc1 should appear", foundTextOnly);
+ assertTrue("vector-only doc2 should appear", foundVectorOnly);
+ } finally {
+ closeHybridIndex();
+ }
+ }
+
+ public void testMultipleVectorFields() throws Exception {
+ float[] queryA = normalize(new float[] {1.0f, 0.0f});
+ float[] queryB = normalize(new float[] {0.0f, 1.0f});
+
+ Directory testDir = newDirectory();
+ IndexWriterConfig config = new IndexWriterConfig();
+ config.setCodec(TestUtil.getDefaultCodec());
+ IndexWriter writer = new IndexWriter(testDir, config);
+
+ Document doc0 = new Document();
+ doc0.add(
+ new KnnFloatVectorField(
+ "emb_a", normalize(new float[] {0.95f, 0.05f}), VectorSimilarityFunction.COSINE));
+ doc0.add(
+ new KnnFloatVectorField(
+ "emb_b", normalize(new float[] {0.05f, 0.95f}), VectorSimilarityFunction.COSINE));
+ writer.addDocument(doc0);
+
+ Document doc1 = new Document();
+ doc1.add(
+ new KnnFloatVectorField(
+ "emb_a", normalize(new float[] {0.90f, 0.10f}), VectorSimilarityFunction.COSINE));
+ doc1.add(
+ new KnnFloatVectorField(
+ "emb_b", normalize(new float[] {1.0f, 0.0f}), VectorSimilarityFunction.COSINE));
+ writer.addDocument(doc1);
+
+ Document doc2 = new Document();
+ doc2.add(
+ new KnnFloatVectorField(
+ "emb_a", normalize(new float[] {0.0f, 1.0f}), VectorSimilarityFunction.COSINE));
+ doc2.add(
+ new KnnFloatVectorField(
+ "emb_b", normalize(new float[] {0.10f, 0.90f}), VectorSimilarityFunction.COSINE));
+ writer.addDocument(doc2);
+
+ writer.forceMerge(1);
+ writer.close();
+ IndexReader testReader = DirectoryReader.open(testDir);
+ IndexSearcher testSearcher = newSearcher(testReader);
+
+ Query vecQueryA = new KnnFloatVectorQuery("emb_a", queryA, 3);
+ Query vecQueryB = new KnnFloatVectorQuery("emb_b", queryB, 3);
+
+ LogOddsFusionQuery multiVecQuery =
+ new LogOddsFusionQuery(Arrays.asList(vecQueryA, vecQueryB), 0.5f);
+
+ ScoreDoc[] hits = testSearcher.search(multiVecQuery, 3).scoreDocs;
+ assertEquals("all 3 docs should match", 3, hits.length);
+ assertEquals("doc0 (close to both) should rank first", 0, hits[0].doc);
+
+ testReader.close();
+ testDir.close();
+ }
+
+ // ---- Boolean logic combinations ----
+
+ public void testBooleanMustWithHybrid() throws Exception {
+ buildHybridIndex();
+ try {
+ Query textQuery = bayesian(new TermQuery(new Term("body", "lucene")));
+ Query vectorQuery = new KnnFloatVectorQuery("embedding", QUERY_A, 10);
+
+ LogOddsFusionQuery hybridQuery =
+ new LogOddsFusionQuery(Arrays.asList(textQuery, vectorQuery), 0.5f);
+
+ BooleanQuery filtered =
+ new BooleanQuery.Builder()
+ .add(new TermQuery(new Term("body", "lucene")), BooleanClause.Occur.FILTER)
+ .add(hybridQuery, BooleanClause.Occur.MUST)
+ .build();
+
+ ScoreDoc[] hits = hybridSearcher.search(filtered, 10).scoreDocs;
+ for (ScoreDoc hit : hits) {
+ assertTrue("only docs 0 and 1 match text", hit.doc == 0 || hit.doc == 1);
+ }
+ assertTrue("should have at least 1 hit", hits.length >= 1);
+ } finally {
+ closeHybridIndex();
+ }
+ }
+
+ public void testBooleanMustNotWithHybrid() throws Exception {
+ buildHybridIndex();
+ try {
+ Query textQuery = bayesian(new TermQuery(new Term("body", "lucene")));
+ Query vectorQuery = new KnnFloatVectorQuery("embedding", QUERY_A, 10);
+
+ LogOddsFusionQuery hybridQuery =
+ new LogOddsFusionQuery(Arrays.asList(textQuery, vectorQuery), 0.5f);
+
+ Query excludeFilter = new TermQuery(new Term("body", "cooking"));
+ BooleanQuery excludeQuery =
+ new BooleanQuery.Builder()
+ .add(hybridQuery, BooleanClause.Occur.MUST)
+ .add(excludeFilter, BooleanClause.Occur.MUST_NOT)
+ .build();
+
+ ScoreDoc[] hits = hybridSearcher.search(excludeQuery, 10).scoreDocs;
+ for (ScoreDoc hit : hits) {
+ assertNotEquals("doc2 (cooking) should be excluded", 2, hit.doc);
+ }
+ } finally {
+ closeHybridIndex();
+ }
+ }
+
+ public void testBooleanShouldWithHybrid() throws Exception {
+ buildHybridIndex();
+ try {
+ Query textLucene = bayesian(new TermQuery(new Term("body", "lucene")));
+ Query textCooking = bayesian(new TermQuery(new Term("body", "cooking")));
+ Query vecA = new KnnFloatVectorQuery("embedding", QUERY_A, 10);
+
+ LogOddsFusionQuery hybrid1 = new LogOddsFusionQuery(Arrays.asList(textLucene, vecA), 0.5f);
+ LogOddsFusionQuery hybrid2 = new LogOddsFusionQuery(Arrays.asList(textCooking, vecA), 0.5f);
+
+ BooleanQuery orQuery =
+ new BooleanQuery.Builder()
+ .add(hybrid1, BooleanClause.Occur.SHOULD)
+ .add(hybrid2, BooleanClause.Occur.SHOULD)
+ .build();
+
+ ScoreDoc[] hits = hybridSearcher.search(orQuery, 20).scoreDocs;
+ assertTrue("should have hits", hits.length > 0);
+
+ boolean foundDoc0 = false;
+ boolean foundDoc2 = false;
+ for (ScoreDoc hit : hits) {
+ if (hit.doc == 0) foundDoc0 = true;
+ if (hit.doc == 2) foundDoc2 = true;
+ }
+ assertTrue("doc0 (lucene + near vector) should appear", foundDoc0);
+ assertTrue("doc2 (cooking + near vector) should appear", foundDoc2);
+ } finally {
+ closeHybridIndex();
+ }
+ }
+
+ public void testVectorAndVector() throws Exception {
+ float[] queryX = normalize(new float[] {1.0f, 0.0f});
+ float[] queryY = normalize(new float[] {0.0f, 1.0f});
+
+ Directory testDir = newDirectory();
+ IndexWriterConfig config = new IndexWriterConfig();
+ config.setCodec(TestUtil.getDefaultCodec());
+ IndexWriter writer = new IndexWriter(testDir, config);
+
+ Document doc0 = new Document();
+ doc0.add(
+ new KnnFloatVectorField(
+ "emb", normalize(new float[] {0.7f, 0.7f}), VectorSimilarityFunction.COSINE));
+ writer.addDocument(doc0);
+
+ Document doc1 = new Document();
+ doc1.add(
+ new KnnFloatVectorField(
+ "emb", normalize(new float[] {0.95f, 0.05f}), VectorSimilarityFunction.COSINE));
+ writer.addDocument(doc1);
+
+ Document doc2 = new Document();
+ doc2.add(
+ new KnnFloatVectorField(
+ "emb", normalize(new float[] {0.05f, 0.95f}), VectorSimilarityFunction.COSINE));
+ writer.addDocument(doc2);
+
+ writer.forceMerge(1);
+ writer.close();
+ IndexReader testReader = DirectoryReader.open(testDir);
+ IndexSearcher testSearcher = newSearcher(testReader);
+
+ Query vecX = new KnnFloatVectorQuery("emb", queryX, 3);
+ Query vecY = new KnnFloatVectorQuery("emb", queryY, 3);
+
+ BooleanQuery vecAnd =
+ new BooleanQuery.Builder()
+ .add(vecX, BooleanClause.Occur.MUST)
+ .add(vecY, BooleanClause.Occur.MUST)
+ .build();
+
+ ScoreDoc[] hits = testSearcher.search(vecAnd, 3).scoreDocs;
+ assertEquals("all 3 docs should match", 3, hits.length);
+ assertEquals("doc0 (near both) should rank first", 0, hits[0].doc);
+
+ testReader.close();
+ testDir.close();
+ }
+
+ public void testVectorNot() throws Exception {
+ buildHybridIndex();
+ try {
+ Query textQuery = new TermQuery(new Term("body", "lucene"));
+ Query vecNear = new KnnFloatVectorQuery("embedding", QUERY_A, 10);
+
+ BooleanQuery textNotVector =
+ new BooleanQuery.Builder()
+ .add(textQuery, BooleanClause.Occur.MUST)
+ .add(vecNear, BooleanClause.Occur.MUST_NOT)
+ .build();
+
+ ScoreDoc[] hits = hybridSearcher.search(textNotVector, 10).scoreDocs;
+ for (ScoreDoc hit : hits) {
+ assertNotEquals("doc0 should be excluded (near QUERY_A)", 0, hit.doc);
+ }
+ } finally {
+ closeHybridIndex();
+ }
+ }
+
+ /** L2-normalize a float vector. */
+ private static float[] normalize(float[] v) {
+ double norm = 0;
+ for (float x : v) {
+ norm += x * x;
+ }
+ norm = Math.sqrt(norm);
+ float[] result = new float[v.length];
+ for (int i = 0; i < v.length; i++) {
+ result[i] = (float) (v[i] / norm);
+ }
+ return result;
+ }
+}