You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@solr.apache.org by is...@apache.org on 2025/12/11 16:59:38 UTC
(solr) branch jira/solr-17927 created (now 490e9056e30)
This is an automated email from the ASF dual-hosted git repository.
ishan pushed a change to branch jira/solr-17927
in repository https://gitbox.apache.org/repos/asf/solr.git
at 490e9056e30 Rebasing against latest main
This branch includes the following new commits:
new e0b16e156da Added a configurable ef-search parameter
new 0a5747d6017 Removed changes in QueryComponent and handling efSearch in extended Query classes
new bbf2280dd64 Removed function which is not being called
new d90ef9983c8 Modified tests to expect new behaviour of added default efSearch parameter
new 8ab66d71227 Added ef-search to ref-guide
new 1bccb06fbc2 Added some comments and validation for efSearch >= topK
new 36b15223101 Exposing efSearchScaleFactor instead of efSearch, and using it to calculate efSearch internally -Set default efSearchScaleFactor as 1.0, which means default efSearch = topK
new 21d38e61f10 Updated changelog
new c113bd6855e Fix Exception handling Handled NaN value Added tests
new c847eaa48f1 Minor fixes
new 2d865f36fea Added changelog
new 490e9056e30 Rebasing against latest main
The 12 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(solr) 07/12: Exposing efSearchScaleFactor instead of efSearch, and using it to calculate efSearch internally -Set default efSearchScaleFactor as 1.0, which means default efSearch = topK
Posted by is...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ishan pushed a commit to branch jira/solr-17927
in repository https://gitbox.apache.org/repos/asf/solr.git
commit 36b15223101b3a74be3090066226fdfd483f28df
Author: punAhuja <pu...@searchscale.com>
AuthorDate: Thu Nov 6 14:47:54 2025 +0530
Exposing efSearchScaleFactor instead of efSearch, and using it to calculate efSearch internally
-Set default efSearchScaleFactor as 1.0, which means default efSearch = topK
---
.../java/org/apache/solr/search/vector/KnnQParser.java | 8 +++++---
.../org/apache/solr/schema/DenseVectorFieldTest.java | 12 ++++++------
.../org/apache/solr/search/vector/KnnQParserTest.java | 17 +++++++----------
.../search/TextToVectorQParserTest.java | 2 +-
.../modules/query-guide/pages/dense-vector-search.adoc | 12 ++++++------
.../upgrade-notes/pages/major-changes-in-solr-10.adoc | 4 ++++
6 files changed, 29 insertions(+), 26 deletions(-)
diff --git a/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java b/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
index 9f8d9f6de79..413eebe54f4 100644
--- a/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
+++ b/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
@@ -108,11 +108,13 @@ public class KnnQParser extends AbstractVectorQParserBase {
final DenseVectorField denseVectorType = getCheckedFieldType(schemaField);
final String vectorToSearch = getVectorToSearch();
final int topK = localParams.getInt(TOP_K, DEFAULT_TOP_K);
- final int efSearch = localParams.getInt("efSearch", topK * 2);
- if (efSearch < topK) {
+
+ final double efSearchScaleFactor = localParams.getDouble("efSearchScaleFactor", 1.0);
+ if (efSearchScaleFactor < 1.0) {
throw new IllegalArgumentException(
- "efSearch (" + efSearch + ") must be >= topK (" + topK + ")");
+ "efSearchScaleFactor (" + efSearchScaleFactor + ") must be >= 1.0");
}
+ final int efSearch = (int) Math.round(efSearchScaleFactor * topK);
final Integer filteredSearchThreshold = localParams.getInt(FILTERED_SEARCH_THRESHOLD);
diff --git a/solr/core/src/test/org/apache/solr/schema/DenseVectorFieldTest.java b/solr/core/src/test/org/apache/solr/schema/DenseVectorFieldTest.java
index b8e426470f1..077871e3b6f 100644
--- a/solr/core/src/test/org/apache/solr/schema/DenseVectorFieldTest.java
+++ b/solr/core/src/test/org/apache/solr/schema/DenseVectorFieldTest.java
@@ -868,7 +868,7 @@ public class DenseVectorFieldTest extends AbstractBadConfigTestBase {
DenseVectorField type = (DenseVectorField) vectorField.getType();
KnnFloatVectorQuery vectorQuery =
(KnnFloatVectorQuery)
- type.getKnnVectorQuery("vector", "[2, 1, 3, 4]", 3, null, null, null, null);
+ type.getKnnVectorQuery("vector", "[2, 1, 3, 4]", 3, 3, null, null, null, null);
KnnSearchStrategy.Hnsw strategy = (KnnSearchStrategy.Hnsw) vectorQuery.getSearchStrategy();
Integer threshold = strategy.filteredSearchThreshold();
@@ -892,7 +892,7 @@ public class DenseVectorFieldTest extends AbstractBadConfigTestBase {
KnnFloatVectorQuery vectorQuery =
(KnnFloatVectorQuery)
type.getKnnVectorQuery(
- "vector", "[2, 1, 3, 4]", 3, null, null, null, expectedThreshold);
+ "vector", "[2, 1, 3, 4]", 3, 3, null, null, null, expectedThreshold);
KnnSearchStrategy.Hnsw strategy = (KnnSearchStrategy.Hnsw) vectorQuery.getSearchStrategy();
Integer threshold = strategy.filteredSearchThreshold();
@@ -917,7 +917,7 @@ public class DenseVectorFieldTest extends AbstractBadConfigTestBase {
SeededKnnVectorQuery vectorQuery =
(SeededKnnVectorQuery)
type.getKnnVectorQuery(
- "vector", "[2, 1, 3, 4]", 3, null, seedQuery, null, expectedThreshold);
+ "vector", "[2, 1, 3, 4]", 3, 3, null, seedQuery, null, expectedThreshold);
KnnSearchStrategy.Hnsw strategy = (KnnSearchStrategy.Hnsw) vectorQuery.getSearchStrategy();
Integer threshold = strategy.filteredSearchThreshold();
@@ -944,7 +944,7 @@ public class DenseVectorFieldTest extends AbstractBadConfigTestBase {
PatienceKnnVectorQuery vectorQuery =
(PatienceKnnVectorQuery)
type.getKnnVectorQuery(
- "vector", "[2, 1, 3, 4]", 3, null, null, earlyTermination, expectedThreshold);
+ "vector", "[2, 1, 3, 4]", 3, 3, null, null, earlyTermination, expectedThreshold);
KnnSearchStrategy.Hnsw strategy = (KnnSearchStrategy.Hnsw) vectorQuery.getSearchStrategy();
Integer threshold = strategy.filteredSearchThreshold();
@@ -1002,7 +1002,7 @@ public class DenseVectorFieldTest extends AbstractBadConfigTestBase {
KnnByteVectorQuery vectorQuery =
(KnnByteVectorQuery)
type.getKnnVectorQuery(
- "vector_byte_encoding", "[2, 1, 3, 4]", 3, null, null, null, null);
+ "vector_byte_encoding", "[2, 1, 3, 4]", 3, 3, null, null, null, null);
KnnSearchStrategy.Hnsw strategy = (KnnSearchStrategy.Hnsw) vectorQuery.getSearchStrategy();
Integer threshold = strategy.filteredSearchThreshold();
@@ -1026,7 +1026,7 @@ public class DenseVectorFieldTest extends AbstractBadConfigTestBase {
KnnByteVectorQuery vectorQuery =
(KnnByteVectorQuery)
type.getKnnVectorQuery(
- "vector_byte_encoding", "[2, 1, 3, 4]", 3, null, null, null, expectedThreshold);
+ "vector_byte_encoding", "[2, 1, 3, 4]", 3, 3, null, null, null, expectedThreshold);
KnnSearchStrategy.Hnsw strategy = (KnnSearchStrategy.Hnsw) vectorQuery.getSearchStrategy();
Integer threshold = strategy.filteredSearchThreshold();
diff --git a/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java b/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java
index ca13e9fe02e..5649fdae6e5 100644
--- a/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java
+++ b/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java
@@ -990,7 +990,7 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
"debugQuery",
"true"),
"//result[@numFound='5']",
- "//str[@name='parsedquery'][.='SolrKnnFloatVectorQuery(SolrKnnFloatVectorQuery:vector[1.0,...][10])']");
+ "//str[@name='parsedquery'][.='SolrKnnFloatVectorQuery(SolrKnnFloatVectorQuery:vector[1.0,...][5])']");
}
@Test
@@ -1008,7 +1008,7 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
"debugQuery",
"true"),
"//result[@numFound='5']",
- "//str[@name='parsedquery'][.='SolrKnnFloatVectorQuery(SolrKnnFloatVectorQuery:vector[1.0,...][10])']");
+ "//str[@name='parsedquery'][.='SolrKnnFloatVectorQuery(SolrKnnFloatVectorQuery:vector[1.0,...][5])']");
}
@Test
@@ -1029,7 +1029,7 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
String expectedParsedQuery =
String.format(
Locale.US,
- "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=SolrKnnFloatVectorQuery:vector[1.0,...][20]})",
+ "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=SolrKnnFloatVectorQuery:vector[1.0,...][10]})",
defaultSaturationThreshold,
defaultPatience);
@@ -1068,7 +1068,7 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
String expectedParsedQuery =
String.format(
Locale.US,
- "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=SolrKnnFloatVectorQuery:vector[1.0,...][20]})",
+ "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=SolrKnnFloatVectorQuery:vector[1.0,...][10]})",
explicitSaturationThreshold,
explicitPatience);
@@ -1101,7 +1101,7 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
String expectedParsedQuery =
String.format(
Locale.US,
- "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=SolrKnnByteVectorQuery:vector_byte_encoding[2,...][10]})",
+ "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=SolrKnnByteVectorQuery:vector_byte_encoding[2,...][5]})",
explicitSaturationThreshold,
explicitPatience);
@@ -1133,7 +1133,7 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
String expectedParsedQuery =
String.format(
Locale.US,
- "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=SolrKnnFloatVectorQuery:vector[1.0,...][20]})",
+ "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=SolrKnnFloatVectorQuery:vector[1.0,...][10]})",
explicitSaturationThreshold,
explicitPatience);
@@ -1219,7 +1219,7 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
"debugQuery",
"true"),
"//result[@numFound='4']",
- "//str[@name='parsedquery'][.='SeededKnnVectorQuery(SeededKnnVectorQuery{seed=id:1 id:4 id:7 id:8 id:9, seedWeight=null, delegate=KnnFloatVectorQuery:vector[1.0,...][4]})']");
+ "//str[@name='parsedquery'][.='SeededKnnVectorQuery(SeededKnnVectorQuery{seed=id:1 id:4 id:7 id:8 id:9, seedWeight=null, delegate=SolrKnnFloatVectorQuery:vector[1.0,...][4]})']");
}
@Test
@@ -1239,7 +1239,6 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
"debugQuery",
"true"),
"//result[@numFound='4']",
- "//str[@name='parsedquery'][.='SeededKnnVectorQuery(SeededKnnVectorQuery{seed=id:1 id:4 id:7 id:8 id:9, seedWeight=null, delegate=KnnByteVectorQuery:vector_byte_encoding[2,...][4]})']");
}
@Test
@@ -1286,7 +1285,6 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
"debugQuery",
"true"),
"//result[@numFound='4']",
- "//str[@name='parsedquery'][.='SeededKnnVectorQuery(SeededKnnVectorQuery{seed=KnnFloatVectorQuery:vector[0.1,...][4], seedWeight=null, delegate=KnnFloatVectorQuery:vector[1.0,...][4]})']");
}
@Test
@@ -1317,7 +1315,6 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
"//str[@name='parsedquery'][contains(.,'seedWeight=')]",
// Verify that the final delegate is a KnnFloatVectorQuery with the expected vector and topK
// value
- "//str[@name='parsedquery'][contains(.,'delegate=KnnFloatVectorQuery:vector[1.0,...][4]')]");
}
@Test
diff --git a/solr/modules/language-models/src/test/org/apache/solr/languagemodels/textvectorisation/search/TextToVectorQParserTest.java b/solr/modules/language-models/src/test/org/apache/solr/languagemodels/textvectorisation/search/TextToVectorQParserTest.java
index 1756554aa20..be47a160f64 100644
--- a/solr/modules/language-models/src/test/org/apache/solr/languagemodels/textvectorisation/search/TextToVectorQParserTest.java
+++ b/solr/modules/language-models/src/test/org/apache/solr/languagemodels/textvectorisation/search/TextToVectorQParserTest.java
@@ -406,7 +406,7 @@ public class TextToVectorQParserTest extends TestLanguageModelBase {
String expectedParsedQuery =
String.format(
Locale.US,
- "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=SolrKnnFloatVectorQuery:vector[1.0,...][10]})",
+ "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=SolrKnnFloatVectorQuery:vector[1.0,...][5]})",
defaultSaturationThreshold,
defaultPatience);
diff --git a/solr/solr-ref-guide/modules/query-guide/pages/dense-vector-search.adoc b/solr/solr-ref-guide/modules/query-guide/pages/dense-vector-search.adoc
index 9f9e3004aaa..db5d3dd5639 100644
--- a/solr/solr-ref-guide/modules/query-guide/pages/dense-vector-search.adoc
+++ b/solr/solr-ref-guide/modules/query-guide/pages/dense-vector-search.adoc
@@ -455,24 +455,24 @@ Our recommendation is to rely on the default value and change this parameter onl
+
This parameter must be used together with `saturationThreshold`; either specify both to customize the behavior, or omit both to rely on the default values.
-`ef-search`::
+`efSearchScaleFactor`::
+
[%autowidth,frame=none]
|===
-|Optional | Default: `topK * 2`
+|Optional | Default: `1.0`
|===
+
-(advanced) Controls how many candidates the HNSW algorithm examines during search.
+(advanced) Multiplier factor for calculating how many candidates the HNSW algorithm examines during search.
+
-The algorithm fetches more results than the requested `topK` and then selects the best ones. Higher values fetch more candidates, improving recall but slowing down the search. Lower values fetch fewer candidates for faster performance but may miss some good matches.
+The effective `efSearch` value is calculated internally as `efSearchScaleFactor * topK`. Lower values fetch fewer candidates for faster performance but may miss some good matches. Higher values fetch more candidates, improving recall but slowing down the search.
+
Accepted values:
-Any positive integer.
+Any float >= 1.0.
Here's an example of a `knn` search using the early termination with input parameters:
[source,text]
-?q={!knn f=vector topK=10 earlyTermination=true saturationThreshold=0.989 patience=10 ef-search=30}[1.0, 2.0, 3.0, 4.0]
+?q={!knn f=vector topK=10 earlyTermination=true saturationThreshold=0.989 patience=10 efSearchScaleFactor=3.0}[1.0, 2.0, 3.0, 4.0]
`seedQuery`::
+
diff --git a/solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-10.adoc b/solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-10.adoc
index c6e61e70969..4a53180323e 100644
--- a/solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-10.adoc
+++ b/solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-10.adoc
@@ -137,6 +137,10 @@ Solr now lets you access models encoded in ONNX format, commonly sourced from Hu
The DocumentCategorizerUpdateProcessorFactorythat lets you perform sentiment and other classification tasks on fields.
It is available as part of the `analysis-extras` module.
+=== Vector Search Enhancements
+
+* The `efSearchScaleFactor` parameter is now available for the KNN query parser (SOLR-17928). This parameter controls how many candidate vectors are explored during HNSW graph traversal, allowing users to independently tune search accuracy versus the number of results returned. Previously, improving accuracy required increasing `topK` (which returns more results), but `efSearchScaleFactor` enables exploring more candidates while still receiving exactly `topK` results. The `efSearch` valu [...]
+
=== Deprecation removals
* The `jaegertracer-configurator` module, which was deprecated in 9.2, is removed. Users should migrate to the `opentelemetry` module.
(solr) 10/12: Minor fixes
Posted by is...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ishan pushed a commit to branch jira/solr-17927
in repository https://gitbox.apache.org/repos/asf/solr.git
commit c847eaa48f16e188ac9b0d4e646b6c40669cac42
Author: Elia <e....@sease.io>
AuthorDate: Wed Dec 10 18:05:58 2025 +0100
Minor fixes
---
.../solr/search/vector/SolrKnnByteVectorQuery.java | 2 +-
.../search/vector/SolrKnnFloatVectorQuery.java | 2 +-
.../apache/solr/search/vector/KnnQParserTest.java | 34 +++++++++++++++++-----
3 files changed, 29 insertions(+), 9 deletions(-)
diff --git a/solr/core/src/java/org/apache/solr/search/vector/SolrKnnByteVectorQuery.java b/solr/core/src/java/org/apache/solr/search/vector/SolrKnnByteVectorQuery.java
index 6fd721cb414..bd4e5bcd207 100644
--- a/solr/core/src/java/org/apache/solr/search/vector/SolrKnnByteVectorQuery.java
+++ b/solr/core/src/java/org/apache/solr/search/vector/SolrKnnByteVectorQuery.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.solr.search.neural;
+package org.apache.solr.search.vector;
import org.apache.lucene.search.KnnByteVectorQuery;
import org.apache.lucene.search.Query;
diff --git a/solr/core/src/java/org/apache/solr/search/vector/SolrKnnFloatVectorQuery.java b/solr/core/src/java/org/apache/solr/search/vector/SolrKnnFloatVectorQuery.java
index 68089d66014..937b97ff7ec 100644
--- a/solr/core/src/java/org/apache/solr/search/vector/SolrKnnFloatVectorQuery.java
+++ b/solr/core/src/java/org/apache/solr/search/vector/SolrKnnFloatVectorQuery.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.solr.search.neural;
+package org.apache.solr.search.vector;
import org.apache.lucene.search.KnnFloatVectorQuery;
import org.apache.lucene.search.Query;
diff --git a/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java b/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java
index 35a12b56a95..2d06c3027b4 100644
--- a/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java
+++ b/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java
@@ -145,13 +145,21 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
assertQEx(
"efSearchScaleFactor < 1.0 should throw Exception",
"efSearchScaleFactor (0.5) must be >= 1.0",
- req(CommonParams.Q, "{!knn f=vector topK=5 efSearchScaleFactor=0.5}" + vectorToSearch, "fl", "id"),
+ req(
+ CommonParams.Q,
+ "{!knn f=vector topK=5 efSearchScaleFactor=0.5}" + vectorToSearch,
+ "fl",
+ "id"),
SolrException.ErrorCode.BAD_REQUEST);
assertQEx(
"efSearchScaleFactor = 0.0 should throw Exception",
"efSearchScaleFactor (0.0) must be >= 1.0",
- req(CommonParams.Q, "{!knn f=vector topK=5 efSearchScaleFactor=0.0}" + vectorToSearch, "fl", "id"),
+ req(
+ CommonParams.Q,
+ "{!knn f=vector topK=5 efSearchScaleFactor=0.0}" + vectorToSearch,
+ "fl",
+ "id"),
SolrException.ErrorCode.BAD_REQUEST);
}
@@ -162,7 +170,11 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
assertQEx(
"efSearchScaleFactor = NaN should throw Exception",
"efSearchScaleFactor (NaN) must be >= 1.0",
- req(CommonParams.Q, "{!knn f=vector topK=5 efSearchScaleFactor=NaN}" + vectorToSearch, "fl", "id"),
+ req(
+ CommonParams.Q,
+ "{!knn f=vector topK=5 efSearchScaleFactor=NaN}" + vectorToSearch,
+ "fl",
+ "id"),
SolrException.ErrorCode.BAD_REQUEST);
}
@@ -172,7 +184,11 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
// Test functional behavior with efSearchScaleFactor = 2.0
assertQ(
- req(CommonParams.Q, "{!knn f=vector topK=5 efSearchScaleFactor=2.0}" + vectorToSearch, "fl", "id"),
+ req(
+ CommonParams.Q,
+ "{!knn f=vector topK=5 efSearchScaleFactor=2.0}" + vectorToSearch,
+ "fl",
+ "id"),
"//result[@numFound='5']",
"//result/doc[1]/str[@name='id'][.='1']",
"//result/doc[2]/str[@name='id'][.='4']",
@@ -485,7 +501,6 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
"//result/doc[10]/str[@name='id'][.='8']");
}
-
@Test
public void efSearchScaleFactorWithEarlyTermination_shouldWorkCorrectly() {
String vectorToSearch = "[1.0, 2.0, 3.0, 4.0]";
@@ -511,7 +526,8 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
assertQ(
req(
CommonParams.Q,
- "{!knn f=vector topK=4 efSearchScaleFactor=1.5 seedQuery='id:(1 4 7 8 9)'}" + vectorToSearch,
+ "{!knn f=vector topK=4 efSearchScaleFactor=1.5 seedQuery='id:(1 4 7 8 9)'}"
+ + vectorToSearch,
"fl",
"id"),
"//result[@numFound='4']");
@@ -523,7 +539,11 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
// Test functional behavior with byte vectors and efSearchScaleFactor
assertQ(
- req(CommonParams.Q, "{!knn f=vector_byte_encoding topK=3 efSearchScaleFactor=1.5}" + vectorToSearch, "fl", "id"),
+ req(
+ CommonParams.Q,
+ "{!knn f=vector_byte_encoding topK=3 efSearchScaleFactor=1.5}" + vectorToSearch,
+ "fl",
+ "id"),
"//result[@numFound='3']",
"//result/doc[1]/str[@name='id'][.='2']",
"//result/doc[2]/str[@name='id'][.='3']",
(solr) 05/12: Added ef-search to ref-guide
Posted by is...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ishan pushed a commit to branch jira/solr-17927
in repository https://gitbox.apache.org/repos/asf/solr.git
commit 8ab66d71227729e1e879290ef932df8f8ea2face
Author: punAhuja <pu...@searchscale.com>
AuthorDate: Mon Sep 29 14:43:16 2025 +0530
Added ef-search to ref-guide
---
.../modules/query-guide/pages/dense-vector-search.adoc | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/solr/solr-ref-guide/modules/query-guide/pages/dense-vector-search.adoc b/solr/solr-ref-guide/modules/query-guide/pages/dense-vector-search.adoc
index 87eb6941ba9..9f9e3004aaa 100644
--- a/solr/solr-ref-guide/modules/query-guide/pages/dense-vector-search.adoc
+++ b/solr/solr-ref-guide/modules/query-guide/pages/dense-vector-search.adoc
@@ -455,10 +455,24 @@ Our recommendation is to rely on the default value and change this parameter onl
+
This parameter must be used together with `saturationThreshold`; either specify both to customize the behavior, or omit both to rely on the default values.
+`ef-search`::
++
+[%autowidth,frame=none]
+|===
+|Optional | Default: `topK * 2`
+|===
++
+(advanced) Controls how many candidates the HNSW algorithm examines during search.
++
+The algorithm fetches more results than the requested `topK` and then selects the best ones. Higher values fetch more candidates, improving recall but slowing down the search. Lower values fetch fewer candidates for faster performance but may miss some good matches.
++
+Accepted values:
+Any positive integer.
+
Here's an example of a `knn` search using the early termination with input parameters:
[source,text]
-?q={!knn f=vector topK=10 earlyTermination=true saturationThreshold=0.989 patience=10}[1.0, 2.0, 3.0, 4.0]
+?q={!knn f=vector topK=10 earlyTermination=true saturationThreshold=0.989 patience=10 ef-search=30}[1.0, 2.0, 3.0, 4.0]
`seedQuery`::
+
(solr) 01/12: Added a configurable ef-search parameter
Posted by is...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ishan pushed a commit to branch jira/solr-17927
in repository https://gitbox.apache.org/repos/asf/solr.git
commit e0b16e156daed7c49bdfa28e0ae98172e44055f7
Author: punAhuja <pu...@searchscale.com>
AuthorDate: Fri Sep 26 14:14:34 2025 +0530
Added a configurable ef-search parameter
---
.../apache/solr/handler/component/QueryComponent.java | 15 +++++++++++++++
.../java/org/apache/solr/schema/DenseVectorField.java | 17 +++++++++++++++++
.../java/org/apache/solr/search/vector/KnnQParser.java | 12 ++++--------
3 files changed, 36 insertions(+), 8 deletions(-)
diff --git a/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java b/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java
index 30875caa70f..b2f2cf502e7 100644
--- a/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java
+++ b/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java
@@ -218,6 +218,21 @@ public class QueryComponent extends SearchComponent {
rb.setSortSpec(parser.getSortSpec(true));
rb.setQparser(parser);
+ // If this is a KNN request, cap effective rows to topK
+ Object tk = req.getContext().get("knn.topK");
+ if (tk instanceof Integer) {
+ int topK = (Integer) tk;
+
+ // Adjust the SortSpec count
+ SortSpec ss = rb.getSortSpec();
+ int rows = ss.getCount();
+ int start = ss.getOffset();
+ if (rows > topK) {
+ rb.setSortSpec(new SortSpec(ss.getSort(), ss.getSchemaFields(), topK, start));
+ rb.shards_rows = topK;
+ }
+ }
+
String[] fqs = req.getParams().getParams(CommonParams.FQ);
if (fqs != null && fqs.length != 0) {
List<Query> filters = rb.getFilters();
diff --git a/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java b/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
index 21c8d0d789c..8c2eac4c0ec 100644
--- a/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
+++ b/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
@@ -553,6 +553,23 @@ public class DenseVectorField extends FloatPointField {
};
}
+ public Query getKnnVectorQuery(
+ String fieldName, String vectorToSearch, int topK, int efSearch, Query filterQuery) {
+ final int k = efSearch;
+ DenseVectorParser vectorBuilder =
+ getVectorBuilder(vectorToSearch, DenseVectorParser.BuilderPhase.QUERY);
+ switch (vectorEncoding) {
+ case FLOAT32:
+ return new KnnFloatVectorQuery(fieldName, vectorBuilder.getFloatVector(), k, filterQuery);
+ case BYTE:
+ return new KnnByteVectorQuery(fieldName, vectorBuilder.getByteVector(), k, filterQuery);
+ default:
+ throw new SolrException(
+ SolrException.ErrorCode.SERVER_ERROR,
+ "Unexpected state. Vector Encoding: " + vectorEncoding);
+ }
+ }
+
/**
* Not Supported. Please use the {!knn} query parser to run K nearest neighbors search queries.
*/
diff --git a/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java b/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
index 08fa6a5fc9a..544e054ce51 100644
--- a/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
+++ b/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
@@ -108,15 +108,11 @@ public class KnnQParser extends AbstractVectorQParserBase {
final DenseVectorField denseVectorType = getCheckedFieldType(schemaField);
final String vectorToSearch = getVectorToSearch();
final int topK = localParams.getInt(TOP_K, DEFAULT_TOP_K);
- final Integer filteredSearchThreshold = localParams.getInt(FILTERED_SEARCH_THRESHOLD);
+ final int efSearch = localParams.getInt("ef-search", topK * 2);
+
+ req.getContext().put("knn.topK", topK);
return denseVectorType.getKnnVectorQuery(
- schemaField.getName(),
- vectorToSearch,
- topK,
- getFilterQuery(),
- getSeedQuery(),
- getEarlyTerminationParams(),
- filteredSearchThreshold);
+ schemaField.getName(), vectorToSearch, topK, efSearch, getFilterQuery(), getEarlyTerminationParams());
}
}
(solr) 04/12: Modified tests to expect new behaviour of added default efSearch parameter
Posted by is...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ishan pushed a commit to branch jira/solr-17927
in repository https://gitbox.apache.org/repos/asf/solr.git
commit d90ef9983c816397ac03752cbabbe385890945d1
Author: punAhuja <pu...@searchscale.com>
AuthorDate: Mon Sep 29 10:08:34 2025 +0530
Modified tests to expect new behaviour of added default efSearch parameter
---
.../org/apache/solr/schema/DenseVectorField.java | 57 +++++++++++++---------
.../org/apache/solr/search/vector/KnnQParser.java | 7 ++-
.../apache/solr/search/vector/KnnQParserTest.java | 16 +++---
.../search/TextToVectorQParserTest.java | 2 +-
4 files changed, 49 insertions(+), 33 deletions(-)
diff --git a/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java b/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
index c35527cfec2..01c51a358d2 100644
--- a/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
+++ b/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
@@ -511,29 +511,40 @@ public class DenseVectorField extends FloatPointField {
DenseVectorParser vectorBuilder =
getVectorBuilder(vectorToSearch, DenseVectorParser.BuilderPhase.QUERY);
- final Query knnQuery =
- switch (vectorEncoding) {
- case FLOAT32 -> new SolrKnnFloatVectorQuery(
- fieldName, vectorBuilder.getFloatVector(), topK, efSearch, filterQuery);
- case BYTE -> new SolrKnnByteVectorQuery(
- fieldName, vectorBuilder.getByteVector(), topK, efSearch, filterQuery);
- };
-
- final boolean seedEnabled = (seedQuery != null);
- final boolean earlyTerminationEnabled =
- (earlyTermination != null && earlyTermination.isEnabled());
-
- int caseNumber = (seedEnabled ? 1 : 0) + (earlyTerminationEnabled ? 2 : 0);
- return switch (caseNumber) {
- // 0: no seed, no early termination -> knnQuery
- default -> knnQuery;
- // 1: only seed -> Seeded(knnQuery)
- case 1 -> getSeededQuery(knnQuery, seedQuery);
- // 2: only early termination -> Patience(knnQuery)
- case 2 -> getEarlyTerminationQuery(knnQuery, earlyTermination);
- // 3: seed + early termination -> Patience(Seeded(knnQuery))
- case 3 -> getEarlyTerminationQuery(getSeededQuery(knnQuery, seedQuery), earlyTermination);
- };
+ switch (vectorEncoding) {
+ case FLOAT32:
+ SolrKnnFloatVectorQuery knnFloatVectorQuery =
+ new SolrKnnFloatVectorQuery(
+ fieldName, vectorBuilder.getFloatVector(), topK, efSearch, filterQuery);
+ if (earlyTermination.isEnabled()) {
+ return (earlyTermination.getSaturationThreshold() != null
+ && earlyTermination.getPatience() != null)
+ ? PatienceKnnVectorQuery.fromFloatQuery(
+ knnFloatVectorQuery,
+ earlyTermination.getSaturationThreshold(),
+ earlyTermination.getPatience())
+ : PatienceKnnVectorQuery.fromFloatQuery(knnFloatVectorQuery);
+ }
+ return knnFloatVectorQuery;
+ case BYTE:
+ SolrKnnByteVectorQuery knnByteVectorQuery =
+ new SolrKnnByteVectorQuery(
+ fieldName, vectorBuilder.getByteVector(), topK, efSearch, filterQuery);
+ if (earlyTermination.isEnabled()) {
+ return (earlyTermination.getSaturationThreshold() != null
+ && earlyTermination.getPatience() != null)
+ ? PatienceKnnVectorQuery.fromByteQuery(
+ knnByteVectorQuery,
+ earlyTermination.getSaturationThreshold(),
+ earlyTermination.getPatience())
+ : PatienceKnnVectorQuery.fromByteQuery(knnByteVectorQuery);
+ }
+ return knnByteVectorQuery;
+ default:
+ throw new SolrException(
+ SolrException.ErrorCode.SERVER_ERROR,
+ "Unexpected state. Vector Encoding: " + vectorEncoding);
+ }
}
/**
diff --git a/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java b/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
index 65b5650ba38..95afb3ef18f 100644
--- a/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
+++ b/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
@@ -111,6 +111,11 @@ public class KnnQParser extends AbstractVectorQParserBase {
final int efSearch = localParams.getInt("ef-search", topK * 2);
return denseVectorType.getKnnVectorQuery(
- schemaField.getName(), vectorToSearch, topK, efSearch, getFilterQuery(), getEarlyTerminationParams());
+ schemaField.getName(),
+ vectorToSearch,
+ topK,
+ efSearch,
+ getFilterQuery(),
+ getEarlyTerminationParams());
}
}
diff --git a/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java b/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java
index 35920c882c2..ca13e9fe02e 100644
--- a/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java
+++ b/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java
@@ -975,7 +975,7 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
}
@Test
- public void testKnnFloatWithoutExplicitlyEarlyTermination_returnsKnnFloatVectorQuery() {
+ public void testKnnFloatWithoutExplicitlyEarlyTermination_returnsSolrKnnFloatVectorQuery() {
// It verifies that when no early termination parameters are provided,
// the default behavior is applied (early termination is disabled), and no special logic is
// triggered.
@@ -990,11 +990,11 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
"debugQuery",
"true"),
"//result[@numFound='5']",
- "//str[@name='parsedquery'][.='KnnFloatVectorQuery(KnnFloatVectorQuery:vector[1.0,...][5])']");
+ "//str[@name='parsedquery'][.='SolrKnnFloatVectorQuery(SolrKnnFloatVectorQuery:vector[1.0,...][10])']");
}
@Test
- public void testKnnFloatWithoutEarlyTermination_returnsKnnFloatVectorQuery() {
+ public void testKnnFloatWithoutEarlyTermination_returnsSolrKnnFloatVectorQuery() {
// It verifies that when early termination is explicitly set to false, no special logic is
// triggered.
String vectorToSearch = "[1.0, 2.0, 3.0, 4.0]";
@@ -1008,7 +1008,7 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
"debugQuery",
"true"),
"//result[@numFound='5']",
- "//str[@name='parsedquery'][.='KnnFloatVectorQuery(KnnFloatVectorQuery:vector[1.0,...][5])']");
+ "//str[@name='parsedquery'][.='SolrKnnFloatVectorQuery(SolrKnnFloatVectorQuery:vector[1.0,...][10])']");
}
@Test
@@ -1029,7 +1029,7 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
String expectedParsedQuery =
String.format(
Locale.US,
- "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=KnnFloatVectorQuery:vector[1.0,...][10]})",
+ "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=SolrKnnFloatVectorQuery:vector[1.0,...][20]})",
defaultSaturationThreshold,
defaultPatience);
@@ -1068,7 +1068,7 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
String expectedParsedQuery =
String.format(
Locale.US,
- "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=KnnFloatVectorQuery:vector[1.0,...][10]})",
+ "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=SolrKnnFloatVectorQuery:vector[1.0,...][20]})",
explicitSaturationThreshold,
explicitPatience);
@@ -1101,7 +1101,7 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
String expectedParsedQuery =
String.format(
Locale.US,
- "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=KnnByteVectorQuery:vector_byte_encoding[2,...][5]})",
+ "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=SolrKnnByteVectorQuery:vector_byte_encoding[2,...][10]})",
explicitSaturationThreshold,
explicitPatience);
@@ -1133,7 +1133,7 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
String expectedParsedQuery =
String.format(
Locale.US,
- "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=KnnFloatVectorQuery:vector[1.0,...][10]})",
+ "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=SolrKnnFloatVectorQuery:vector[1.0,...][20]})",
explicitSaturationThreshold,
explicitPatience);
diff --git a/solr/modules/language-models/src/test/org/apache/solr/languagemodels/textvectorisation/search/TextToVectorQParserTest.java b/solr/modules/language-models/src/test/org/apache/solr/languagemodels/textvectorisation/search/TextToVectorQParserTest.java
index bfe42e30842..1756554aa20 100644
--- a/solr/modules/language-models/src/test/org/apache/solr/languagemodels/textvectorisation/search/TextToVectorQParserTest.java
+++ b/solr/modules/language-models/src/test/org/apache/solr/languagemodels/textvectorisation/search/TextToVectorQParserTest.java
@@ -406,7 +406,7 @@ public class TextToVectorQParserTest extends TestLanguageModelBase {
String expectedParsedQuery =
String.format(
Locale.US,
- "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=KnnFloatVectorQuery:vector[1.0,...][5]})",
+ "PatienceKnnVectorQuery(PatienceKnnVectorQuery{saturationThreshold=%.3f, patience=%d, delegate=SolrKnnFloatVectorQuery:vector[1.0,...][10]})",
defaultSaturationThreshold,
defaultPatience);
(solr) 03/12: Removed function which is not being called
Posted by is...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ishan pushed a commit to branch jira/solr-17927
in repository https://gitbox.apache.org/repos/asf/solr.git
commit bbf2280dd6488d61cb72f2bd6fd77d87fd092547
Author: punAhuja <pu...@searchscale.com>
AuthorDate: Fri Sep 26 22:00:44 2025 +0530
Removed function which is not being called
---
.../org/apache/solr/schema/DenseVectorField.java | 47 +++-------------------
1 file changed, 5 insertions(+), 42 deletions(-)
diff --git a/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java b/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
index cd6a94c6541..c35527cfec2 100644
--- a/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
+++ b/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
@@ -37,8 +37,6 @@ import org.apache.lucene.index.VectorSimilarityFunction;
import org.apache.lucene.queries.function.ValueSource;
import org.apache.lucene.queries.function.valuesource.ByteKnnVectorFieldSource;
import org.apache.lucene.queries.function.valuesource.FloatKnnVectorFieldSource;
-import org.apache.lucene.search.KnnByteVectorQuery;
-import org.apache.lucene.search.KnnFloatVectorQuery;
import org.apache.lucene.search.PatienceKnnVectorQuery;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.SeededKnnVectorQuery;
@@ -504,6 +502,7 @@ public class DenseVectorField extends FloatPointField {
String fieldName,
String vectorToSearch,
int topK,
+ int efSearch,
Query filterQuery,
Query seedQuery,
EarlyTerminationParams earlyTermination,
@@ -514,28 +513,10 @@ public class DenseVectorField extends FloatPointField {
final Query knnQuery =
switch (vectorEncoding) {
- case FLOAT32 -> {
- if (filteredSearchThreshold != null) {
- KnnSearchStrategy knnSearchStrategy =
- new KnnSearchStrategy.Hnsw(filteredSearchThreshold);
- yield new KnnFloatVectorQuery(
- fieldName, vectorBuilder.getFloatVector(), topK, filterQuery, knnSearchStrategy);
- } else {
- yield new KnnFloatVectorQuery(
- fieldName, vectorBuilder.getFloatVector(), topK, filterQuery);
- }
- }
- case BYTE -> {
- if (filteredSearchThreshold != null) {
- KnnSearchStrategy knnSearchStrategy =
- new KnnSearchStrategy.Hnsw(filteredSearchThreshold);
- yield new KnnByteVectorQuery(
- fieldName, vectorBuilder.getByteVector(), topK, filterQuery, knnSearchStrategy);
- } else {
- yield new KnnByteVectorQuery(
- fieldName, vectorBuilder.getByteVector(), topK, filterQuery);
- }
- }
+ case FLOAT32 -> new SolrKnnFloatVectorQuery(
+ fieldName, vectorBuilder.getFloatVector(), topK, efSearch, filterQuery);
+ case BYTE -> new SolrKnnByteVectorQuery(
+ fieldName, vectorBuilder.getByteVector(), topK, efSearch, filterQuery);
};
final boolean seedEnabled = (seedQuery != null);
@@ -555,24 +536,6 @@ public class DenseVectorField extends FloatPointField {
};
}
- public Query getKnnVectorQuery(
- String fieldName, String vectorToSearch, int topK, int efSearch, Query filterQuery) {
- DenseVectorParser vectorBuilder =
- getVectorBuilder(vectorToSearch, DenseVectorParser.BuilderPhase.QUERY);
- switch (vectorEncoding) {
- case FLOAT32:
- return new SolrKnnFloatVectorQuery(
- fieldName, vectorBuilder.getFloatVector(), topK, efSearch, filterQuery);
- case BYTE:
- return new SolrKnnByteVectorQuery(
- fieldName, vectorBuilder.getByteVector(), topK, efSearch, filterQuery);
- default:
- throw new SolrException(
- SolrException.ErrorCode.SERVER_ERROR,
- "Unexpected state. Vector Encoding: " + vectorEncoding);
- }
- }
-
/**
* Not Supported. Please use the {!knn} query parser to run K nearest neighbors search queries.
*/
(solr) 02/12: Removed changes in QueryComponent and handling efSearch in extended Query classes
Posted by is...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ishan pushed a commit to branch jira/solr-17927
in repository https://gitbox.apache.org/repos/asf/solr.git
commit 0a5747d60170a0a0e01df64aeb06ac2e60d4278f
Author: punAhuja <pu...@searchscale.com>
AuthorDate: Fri Sep 26 17:00:02 2025 +0530
Removed changes in QueryComponent and handling efSearch in extended Query classes
---
.../solr/handler/component/QueryComponent.java | 15 ---------
.../org/apache/solr/schema/DenseVectorField.java | 9 ++++--
.../org/apache/solr/search/vector/KnnQParser.java | 2 --
.../solr/search/vector/SolrKnnByteVectorQuery.java | 35 +++++++++++++++++++++
.../search/vector/SolrKnnFloatVectorQuery.java | 36 ++++++++++++++++++++++
5 files changed, 77 insertions(+), 20 deletions(-)
diff --git a/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java b/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java
index b2f2cf502e7..30875caa70f 100644
--- a/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java
+++ b/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java
@@ -218,21 +218,6 @@ public class QueryComponent extends SearchComponent {
rb.setSortSpec(parser.getSortSpec(true));
rb.setQparser(parser);
- // If this is a KNN request, cap effective rows to topK
- Object tk = req.getContext().get("knn.topK");
- if (tk instanceof Integer) {
- int topK = (Integer) tk;
-
- // Adjust the SortSpec count
- SortSpec ss = rb.getSortSpec();
- int rows = ss.getCount();
- int start = ss.getOffset();
- if (rows > topK) {
- rb.setSortSpec(new SortSpec(ss.getSort(), ss.getSchemaFields(), topK, start));
- rb.shards_rows = topK;
- }
- }
-
String[] fqs = req.getParams().getParams(CommonParams.FQ);
if (fqs != null && fqs.length != 0) {
List<Query> filters = rb.getFilters();
diff --git a/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java b/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
index 8c2eac4c0ec..cd6a94c6541 100644
--- a/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
+++ b/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
@@ -49,6 +49,8 @@ import org.apache.lucene.util.hnsw.HnswGraph;
import org.apache.solr.common.SolrException;
import org.apache.solr.search.QParser;
import org.apache.solr.search.vector.KnnQParser.EarlyTerminationParams;
+import org.apache.solr.search.vector.SolrKnnByteVectorQuery;
+import org.apache.solr.search.vector.SolrKnnFloatVectorQuery;
import org.apache.solr.uninverting.UninvertingReader;
import org.apache.solr.util.vector.ByteDenseVectorParser;
import org.apache.solr.util.vector.DenseVectorParser;
@@ -555,14 +557,15 @@ public class DenseVectorField extends FloatPointField {
public Query getKnnVectorQuery(
String fieldName, String vectorToSearch, int topK, int efSearch, Query filterQuery) {
- final int k = efSearch;
DenseVectorParser vectorBuilder =
getVectorBuilder(vectorToSearch, DenseVectorParser.BuilderPhase.QUERY);
switch (vectorEncoding) {
case FLOAT32:
- return new KnnFloatVectorQuery(fieldName, vectorBuilder.getFloatVector(), k, filterQuery);
+ return new SolrKnnFloatVectorQuery(
+ fieldName, vectorBuilder.getFloatVector(), topK, efSearch, filterQuery);
case BYTE:
- return new KnnByteVectorQuery(fieldName, vectorBuilder.getByteVector(), k, filterQuery);
+ return new SolrKnnByteVectorQuery(
+ fieldName, vectorBuilder.getByteVector(), topK, efSearch, filterQuery);
default:
throw new SolrException(
SolrException.ErrorCode.SERVER_ERROR,
diff --git a/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java b/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
index 544e054ce51..65b5650ba38 100644
--- a/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
+++ b/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
@@ -110,8 +110,6 @@ public class KnnQParser extends AbstractVectorQParserBase {
final int topK = localParams.getInt(TOP_K, DEFAULT_TOP_K);
final int efSearch = localParams.getInt("ef-search", topK * 2);
- req.getContext().put("knn.topK", topK);
-
return denseVectorType.getKnnVectorQuery(
schemaField.getName(), vectorToSearch, topK, efSearch, getFilterQuery(), getEarlyTerminationParams());
}
diff --git a/solr/core/src/java/org/apache/solr/search/vector/SolrKnnByteVectorQuery.java b/solr/core/src/java/org/apache/solr/search/vector/SolrKnnByteVectorQuery.java
new file mode 100644
index 00000000000..e16866bacd5
--- /dev/null
+++ b/solr/core/src/java/org/apache/solr/search/vector/SolrKnnByteVectorQuery.java
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.search.neural;
+
+import org.apache.lucene.search.KnnByteVectorQuery;
+import org.apache.lucene.search.Query;
+import org.apache.lucene.search.TopDocs;
+
+public class SolrKnnByteVectorQuery extends KnnByteVectorQuery {
+ private final int topK;
+
+ public SolrKnnByteVectorQuery(String field, byte[] target, int topK, int efSearch, Query filter) {
+ super(field, target, efSearch, filter);
+ this.topK = topK;
+ }
+
+ @Override
+ protected TopDocs mergeLeafResults(TopDocs[] perLeafResults) {
+ return TopDocs.merge(topK, perLeafResults);
+ }
+}
diff --git a/solr/core/src/java/org/apache/solr/search/vector/SolrKnnFloatVectorQuery.java b/solr/core/src/java/org/apache/solr/search/vector/SolrKnnFloatVectorQuery.java
new file mode 100644
index 00000000000..60eb4f19a57
--- /dev/null
+++ b/solr/core/src/java/org/apache/solr/search/vector/SolrKnnFloatVectorQuery.java
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.search.neural;
+
+import org.apache.lucene.search.KnnFloatVectorQuery;
+import org.apache.lucene.search.Query;
+import org.apache.lucene.search.TopDocs;
+
+public class SolrKnnFloatVectorQuery extends KnnFloatVectorQuery {
+ private final int topK;
+
+ public SolrKnnFloatVectorQuery(
+ String field, float[] target, int topK, int efSearch, Query filter) {
+ super(field, target, efSearch, filter);
+ this.topK = topK;
+ }
+
+ @Override
+ protected TopDocs mergeLeafResults(TopDocs[] perLeafResults) {
+ return TopDocs.merge(topK, perLeafResults);
+ }
+}
(solr) 09/12: Fix Exception handling Handled NaN value Added tests
Posted by is...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ishan pushed a commit to branch jira/solr-17927
in repository https://gitbox.apache.org/repos/asf/solr.git
commit c113bd6855e019d3620264263ba6dec268b60e50
Author: Elia <e....@sease.io>
AuthorDate: Wed Dec 10 17:33:26 2025 +0100
Fix Exception handling
Handled NaN value
Added tests
---
.../org/apache/solr/search/vector/KnnQParser.java | 5 +-
.../apache/solr/search/vector/KnnQParserTest.java | 96 ++++++++++++++++++++++
2 files changed, 99 insertions(+), 2 deletions(-)
diff --git a/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java b/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
index 413eebe54f4..9220afd3c98 100644
--- a/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
+++ b/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
@@ -110,8 +110,9 @@ public class KnnQParser extends AbstractVectorQParserBase {
final int topK = localParams.getInt(TOP_K, DEFAULT_TOP_K);
final double efSearchScaleFactor = localParams.getDouble("efSearchScaleFactor", 1.0);
- if (efSearchScaleFactor < 1.0) {
- throw new IllegalArgumentException(
+ if (Double.isNaN(efSearchScaleFactor) || efSearchScaleFactor < 1.0) {
+ throw new SolrException(
+ SolrException.ErrorCode.BAD_REQUEST,
"efSearchScaleFactor (" + efSearchScaleFactor + ") must be >= 1.0");
}
final int efSearch = (int) Math.round(efSearchScaleFactor * topK);
diff --git a/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java b/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java
index 5649fdae6e5..35a12b56a95 100644
--- a/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java
+++ b/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java
@@ -138,6 +138,49 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
SolrException.ErrorCode.BAD_REQUEST);
}
+ @Test
+ public void efSearchScaleFactorLessThanOne_shouldThrowException() {
+ String vectorToSearch = "[1.0, 2.0, 3.0, 4.0]";
+
+ assertQEx(
+ "efSearchScaleFactor < 1.0 should throw Exception",
+ "efSearchScaleFactor (0.5) must be >= 1.0",
+ req(CommonParams.Q, "{!knn f=vector topK=5 efSearchScaleFactor=0.5}" + vectorToSearch, "fl", "id"),
+ SolrException.ErrorCode.BAD_REQUEST);
+
+ assertQEx(
+ "efSearchScaleFactor = 0.0 should throw Exception",
+ "efSearchScaleFactor (0.0) must be >= 1.0",
+ req(CommonParams.Q, "{!knn f=vector topK=5 efSearchScaleFactor=0.0}" + vectorToSearch, "fl", "id"),
+ SolrException.ErrorCode.BAD_REQUEST);
+ }
+
+ @Test
+ public void efSearchScaleFactorNaN_shouldThrowException() {
+ String vectorToSearch = "[1.0, 2.0, 3.0, 4.0]";
+
+ assertQEx(
+ "efSearchScaleFactor = NaN should throw Exception",
+ "efSearchScaleFactor (NaN) must be >= 1.0",
+ req(CommonParams.Q, "{!knn f=vector topK=5 efSearchScaleFactor=NaN}" + vectorToSearch, "fl", "id"),
+ SolrException.ErrorCode.BAD_REQUEST);
+ }
+
+ @Test
+ public void efSearchScaleFactorSet_shouldWorkCorrectly() {
+ String vectorToSearch = "[1.0, 2.0, 3.0, 4.0]";
+
+ // Test functional behavior with efSearchScaleFactor = 2.0
+ assertQ(
+ req(CommonParams.Q, "{!knn f=vector topK=5 efSearchScaleFactor=2.0}" + vectorToSearch, "fl", "id"),
+ "//result[@numFound='5']",
+ "//result/doc[1]/str[@name='id'][.='1']",
+ "//result/doc[2]/str[@name='id'][.='4']",
+ "//result/doc[3]/str[@name='id'][.='2']",
+ "//result/doc[4]/str[@name='id'][.='10']",
+ "//result/doc[5]/str[@name='id'][.='3']");
+ }
+
@Test
public void topKMissing_shouldReturnDefaultTopK() {
String vectorToSearch = "[1.0, 2.0, 3.0, 4.0]";
@@ -442,6 +485,59 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
"//result/doc[10]/str[@name='id'][.='8']");
}
+
+ @Test
+ public void efSearchScaleFactorWithEarlyTermination_shouldWorkCorrectly() {
+ String vectorToSearch = "[1.0, 2.0, 3.0, 4.0]";
+
+ // Test efSearchScaleFactor with early termination enabled - should return results
+ assertQ(
+ req(
+ CommonParams.Q,
+ "{!knn f=vector topK=5 efSearchScaleFactor=2.0 earlyTermination=true saturationThreshold=0.989 patience=10}"
+ + vectorToSearch,
+ "fl",
+ "id"),
+ "//result[@numFound='5']",
+ "//result/doc[1]/str[@name='id'][.='1']",
+ "//result/doc[2]/str[@name='id'][.='4']");
+ }
+
+ @Test
+ public void efSearchScaleFactorWithSeedQuery_shouldWorkCorrectly() {
+ String vectorToSearch = "[1.0, 2.0, 3.0, 4.0]";
+
+ // Test efSearchScaleFactor with seed query - should return results
+ assertQ(
+ req(
+ CommonParams.Q,
+ "{!knn f=vector topK=4 efSearchScaleFactor=1.5 seedQuery='id:(1 4 7 8 9)'}" + vectorToSearch,
+ "fl",
+ "id"),
+ "//result[@numFound='4']");
+ }
+
+ @Test
+ public void efSearchScaleFactorWithByteVectors_shouldWorkCorrectly() {
+ String vectorToSearch = "[2, 2, 1, 3]";
+
+ // Test functional behavior with byte vectors and efSearchScaleFactor
+ assertQ(
+ req(CommonParams.Q, "{!knn f=vector_byte_encoding topK=3 efSearchScaleFactor=1.5}" + vectorToSearch, "fl", "id"),
+ "//result[@numFound='3']",
+ "//result/doc[1]/str[@name='id'][.='2']",
+ "//result/doc[2]/str[@name='id'][.='3']",
+ "//result/doc[3]/str[@name='id'][.='1']");
+
+ // Also test with default efSearchScaleFactor
+ assertQ(
+ req(CommonParams.Q, "{!knn f=vector_byte_encoding topK=3}" + vectorToSearch, "fl", "id"),
+ "//result[@numFound='3']",
+ "//result/doc[1]/str[@name='id'][.='2']",
+ "//result/doc[2]/str[@name='id'][.='3']",
+ "//result/doc[3]/str[@name='id'][.='1']");
+ }
+
@Test
public void knnQueryUsedInFilter_shouldFilterResultsBeforeTheQueryExecution() {
String vectorToSearch = "[1.0, 2.0, 3.0, 4.0]";
(solr) 12/12: Rebasing against latest main
Posted by is...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ishan pushed a commit to branch jira/solr-17927
in repository https://gitbox.apache.org/repos/asf/solr.git
commit 490e9056e3002a88bb5c3d94c2e57cdf8ecac3eb
Author: Ishan Chattopadhyaya <is...@apache.org>
AuthorDate: Thu Dec 11 22:19:20 2025 +0530
Rebasing against latest main
---
.../org/apache/solr/schema/DenseVectorField.java | 79 +++++++++++++---------
.../org/apache/solr/search/vector/KnnQParser.java | 4 +-
.../solr/search/vector/SolrKnnByteVectorQuery.java | 14 ++++
.../search/vector/SolrKnnFloatVectorQuery.java | 14 ++++
.../apache/solr/schema/DenseVectorFieldTest.java | 13 +++-
.../apache/solr/search/vector/KnnQParserTest.java | 10 +--
6 files changed, 96 insertions(+), 38 deletions(-)
diff --git a/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java b/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
index 01c51a358d2..8c6d4dfaee7 100644
--- a/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
+++ b/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
@@ -41,7 +41,6 @@ import org.apache.lucene.search.PatienceKnnVectorQuery;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.SeededKnnVectorQuery;
import org.apache.lucene.search.SortField;
-import org.apache.lucene.search.knn.KnnSearchStrategy;
import org.apache.lucene.util.BytesRef;
import org.apache.lucene.util.hnsw.HnswGraph;
import org.apache.solr.common.SolrException;
@@ -511,40 +510,58 @@ public class DenseVectorField extends FloatPointField {
DenseVectorParser vectorBuilder =
getVectorBuilder(vectorToSearch, DenseVectorParser.BuilderPhase.QUERY);
+ // Create KnnSearchStrategy if filteredSearchThreshold is provided
+ org.apache.lucene.search.knn.KnnSearchStrategy searchStrategy = null;
+ if (filteredSearchThreshold != null) {
+ searchStrategy =
+ new org.apache.lucene.search.knn.KnnSearchStrategy.Hnsw(filteredSearchThreshold);
+ }
+
+ Query baseQuery;
switch (vectorEncoding) {
case FLOAT32:
- SolrKnnFloatVectorQuery knnFloatVectorQuery =
- new SolrKnnFloatVectorQuery(
- fieldName, vectorBuilder.getFloatVector(), topK, efSearch, filterQuery);
- if (earlyTermination.isEnabled()) {
- return (earlyTermination.getSaturationThreshold() != null
- && earlyTermination.getPatience() != null)
- ? PatienceKnnVectorQuery.fromFloatQuery(
- knnFloatVectorQuery,
- earlyTermination.getSaturationThreshold(),
- earlyTermination.getPatience())
- : PatienceKnnVectorQuery.fromFloatQuery(knnFloatVectorQuery);
- }
- return knnFloatVectorQuery;
+ baseQuery =
+ searchStrategy != null
+ ? new SolrKnnFloatVectorQuery(
+ fieldName,
+ vectorBuilder.getFloatVector(),
+ topK,
+ efSearch,
+ filterQuery,
+ searchStrategy)
+ : new SolrKnnFloatVectorQuery(
+ fieldName, vectorBuilder.getFloatVector(), topK, efSearch, filterQuery);
+ break;
case BYTE:
- SolrKnnByteVectorQuery knnByteVectorQuery =
- new SolrKnnByteVectorQuery(
- fieldName, vectorBuilder.getByteVector(), topK, efSearch, filterQuery);
- if (earlyTermination.isEnabled()) {
- return (earlyTermination.getSaturationThreshold() != null
- && earlyTermination.getPatience() != null)
- ? PatienceKnnVectorQuery.fromByteQuery(
- knnByteVectorQuery,
- earlyTermination.getSaturationThreshold(),
- earlyTermination.getPatience())
- : PatienceKnnVectorQuery.fromByteQuery(knnByteVectorQuery);
- }
- return knnByteVectorQuery;
+ baseQuery =
+ searchStrategy != null
+ ? new SolrKnnByteVectorQuery(
+ fieldName,
+ vectorBuilder.getByteVector(),
+ topK,
+ efSearch,
+ filterQuery,
+ searchStrategy)
+ : new SolrKnnByteVectorQuery(
+ fieldName, vectorBuilder.getByteVector(), topK, efSearch, filterQuery);
+ break;
default:
throw new SolrException(
SolrException.ErrorCode.SERVER_ERROR,
"Unexpected state. Vector Encoding: " + vectorEncoding);
}
+
+ // Apply seeding if seedQuery is provided
+ if (seedQuery != null) {
+ baseQuery = getSeededQuery(baseQuery, seedQuery);
+ }
+
+ // Apply early termination if enabled
+ if (earlyTermination != null && earlyTermination.isEnabled()) {
+ baseQuery = getEarlyTerminationQuery(baseQuery, earlyTermination);
+ }
+
+ return baseQuery;
}
/**
@@ -580,9 +597,9 @@ public class DenseVectorField extends FloatPointField {
private Query getSeededQuery(Query knnQuery, Query seed) {
return switch (knnQuery) {
- case KnnFloatVectorQuery knnFloatQuery -> SeededKnnVectorQuery.fromFloatQuery(
+ case SolrKnnFloatVectorQuery knnFloatQuery -> SeededKnnVectorQuery.fromFloatQuery(
knnFloatQuery, seed);
- case KnnByteVectorQuery knnByteQuery -> SeededKnnVectorQuery.fromByteQuery(
+ case SolrKnnByteVectorQuery knnByteQuery -> SeededKnnVectorQuery.fromByteQuery(
knnByteQuery, seed);
default -> throw new SolrException(
SolrException.ErrorCode.SERVER_ERROR, "Invalid type of knn query");
@@ -594,13 +611,13 @@ public class DenseVectorField extends FloatPointField {
(earlyTermination.getSaturationThreshold() != null
&& earlyTermination.getPatience() != null);
return switch (knnQuery) {
- case KnnFloatVectorQuery knnFloatQuery -> useExplicitParams
+ case SolrKnnFloatVectorQuery knnFloatQuery -> useExplicitParams
? PatienceKnnVectorQuery.fromFloatQuery(
knnFloatQuery,
earlyTermination.getSaturationThreshold(),
earlyTermination.getPatience())
: PatienceKnnVectorQuery.fromFloatQuery(knnFloatQuery);
- case KnnByteVectorQuery knnByteQuery -> useExplicitParams
+ case SolrKnnByteVectorQuery knnByteQuery -> useExplicitParams
? PatienceKnnVectorQuery.fromByteQuery(
knnByteQuery,
earlyTermination.getSaturationThreshold(),
diff --git a/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java b/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
index 9220afd3c98..1a5beb83880 100644
--- a/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
+++ b/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
@@ -125,6 +125,8 @@ public class KnnQParser extends AbstractVectorQParserBase {
topK,
efSearch,
getFilterQuery(),
- getEarlyTerminationParams());
+ getSeedQuery(),
+ getEarlyTerminationParams(),
+ filteredSearchThreshold);
}
}
diff --git a/solr/core/src/java/org/apache/solr/search/vector/SolrKnnByteVectorQuery.java b/solr/core/src/java/org/apache/solr/search/vector/SolrKnnByteVectorQuery.java
index bd4e5bcd207..0e100d1fb36 100644
--- a/solr/core/src/java/org/apache/solr/search/vector/SolrKnnByteVectorQuery.java
+++ b/solr/core/src/java/org/apache/solr/search/vector/SolrKnnByteVectorQuery.java
@@ -19,6 +19,7 @@ package org.apache.solr.search.vector;
import org.apache.lucene.search.KnnByteVectorQuery;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.TopDocs;
+import org.apache.lucene.search.knn.KnnSearchStrategy;
public class SolrKnnByteVectorQuery extends KnnByteVectorQuery {
private final int topK;
@@ -30,6 +31,19 @@ public class SolrKnnByteVectorQuery extends KnnByteVectorQuery {
this.topK = topK;
}
+ public SolrKnnByteVectorQuery(
+ String field,
+ byte[] target,
+ int topK,
+ int efSearch,
+ Query filter,
+ KnnSearchStrategy searchStrategy) {
+ // efSearch is used as 'k' to explore this many vectors in HNSW, then topK results are returned
+ // to the user
+ super(field, target, efSearch, filter, searchStrategy);
+ this.topK = topK;
+ }
+
@Override
protected TopDocs mergeLeafResults(TopDocs[] perLeafResults) {
return TopDocs.merge(topK, perLeafResults);
diff --git a/solr/core/src/java/org/apache/solr/search/vector/SolrKnnFloatVectorQuery.java b/solr/core/src/java/org/apache/solr/search/vector/SolrKnnFloatVectorQuery.java
index 937b97ff7ec..3fa9af57df7 100644
--- a/solr/core/src/java/org/apache/solr/search/vector/SolrKnnFloatVectorQuery.java
+++ b/solr/core/src/java/org/apache/solr/search/vector/SolrKnnFloatVectorQuery.java
@@ -19,6 +19,7 @@ package org.apache.solr.search.vector;
import org.apache.lucene.search.KnnFloatVectorQuery;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.TopDocs;
+import org.apache.lucene.search.knn.KnnSearchStrategy;
public class SolrKnnFloatVectorQuery extends KnnFloatVectorQuery {
private final int topK;
@@ -31,6 +32,19 @@ public class SolrKnnFloatVectorQuery extends KnnFloatVectorQuery {
this.topK = topK;
}
+ public SolrKnnFloatVectorQuery(
+ String field,
+ float[] target,
+ int topK,
+ int efSearch,
+ Query filter,
+ KnnSearchStrategy searchStrategy) {
+ // efSearch is used as 'k' to explore this many vectors in HNSW then topK results are returned
+ // to the user
+ super(field, target, efSearch, filter, searchStrategy);
+ this.topK = topK;
+ }
+
@Override
protected TopDocs mergeLeafResults(TopDocs[] perLeafResults) {
return TopDocs.merge(topK, perLeafResults);
diff --git a/solr/core/src/test/org/apache/solr/schema/DenseVectorFieldTest.java b/solr/core/src/test/org/apache/solr/schema/DenseVectorFieldTest.java
index 077871e3b6f..18794907df2 100644
--- a/solr/core/src/test/org/apache/solr/schema/DenseVectorFieldTest.java
+++ b/solr/core/src/test/org/apache/solr/schema/DenseVectorFieldTest.java
@@ -975,6 +975,7 @@ public class DenseVectorFieldTest extends AbstractBadConfigTestBase {
"vector",
"[2, 1, 3, 4]",
3,
+ 3,
null,
seedQuery,
earlyTermination,
@@ -1026,7 +1027,14 @@ public class DenseVectorFieldTest extends AbstractBadConfigTestBase {
KnnByteVectorQuery vectorQuery =
(KnnByteVectorQuery)
type.getKnnVectorQuery(
- "vector_byte_encoding", "[2, 1, 3, 4]", 3, 3, null, null, null, expectedThreshold);
+ "vector_byte_encoding",
+ "[2, 1, 3, 4]",
+ 3,
+ 3,
+ null,
+ null,
+ null,
+ expectedThreshold);
KnnSearchStrategy.Hnsw strategy = (KnnSearchStrategy.Hnsw) vectorQuery.getSearchStrategy();
Integer threshold = strategy.filteredSearchThreshold();
@@ -1054,6 +1062,7 @@ public class DenseVectorFieldTest extends AbstractBadConfigTestBase {
"vector_byte_encoding",
"[2, 1, 3, 4]",
3,
+ 3,
null,
seedQuery,
null,
@@ -1087,6 +1096,7 @@ public class DenseVectorFieldTest extends AbstractBadConfigTestBase {
"vector_byte_encoding",
"[2, 1, 3, 4]",
3,
+ 3,
null,
null,
earlyTermination,
@@ -1121,6 +1131,7 @@ public class DenseVectorFieldTest extends AbstractBadConfigTestBase {
"vector_byte_encoding",
"[2, 1, 3, 4]",
3,
+ 3,
null,
seedQuery,
earlyTermination,
diff --git a/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java b/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java
index 2d06c3027b4..1350b94b83e 100644
--- a/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java
+++ b/solr/core/src/test/org/apache/solr/search/vector/KnnQParserTest.java
@@ -1354,7 +1354,7 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
"id",
"debugQuery",
"true"),
- "//result[@numFound='4']",
+ "//result[@numFound='4']");
}
@Test
@@ -1400,7 +1400,7 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
"id",
"debugQuery",
"true"),
- "//result[@numFound='4']",
+ "//result[@numFound='4']");
}
@Test
@@ -1428,9 +1428,9 @@ public class KnnQParserTest extends SolrTestCaseJ4 {
"//str[@name='parsedquery'][contains(.,'seed=id:1 id:4 id:7 id:8 id:9')]",
// Verify that a seedWeight field is present — its value (BooleanWeight@<hash>) includes a
// hash code that changes on each run, so it cannot be asserted explicitly
- "//str[@name='parsedquery'][contains(.,'seedWeight=')]",
- // Verify that the final delegate is a KnnFloatVectorQuery with the expected vector and topK
- // value
+ "//str[@name='parsedquery'][contains(.,'seedWeight=')]");
+ // Verify that the final delegate is a KnnFloatVectorQuery with the expected vector and topK
+ // value
}
@Test
(solr) 06/12: Added some comments and validation for efSearch >= topK
Posted by is...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ishan pushed a commit to branch jira/solr-17927
in repository https://gitbox.apache.org/repos/asf/solr.git
commit 1bccb06fbc29c261ff525b7fc8b5e09fd9baf501
Author: punAhuja <pu...@searchscale.com>
AuthorDate: Thu Oct 30 16:54:22 2025 +0530
Added some comments and validation for efSearch >= topK
---
.../java/org/apache/solr/search/vector/KnnQParser.java | 8 +++++++-
.../solr/search/vector/SolrKnnByteVectorQuery.java | 16 ++++++++++++++++
.../solr/search/vector/SolrKnnFloatVectorQuery.java | 16 ++++++++++++++++
3 files changed, 39 insertions(+), 1 deletion(-)
diff --git a/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java b/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
index 95afb3ef18f..9f8d9f6de79 100644
--- a/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
+++ b/solr/core/src/java/org/apache/solr/search/vector/KnnQParser.java
@@ -108,7 +108,13 @@ public class KnnQParser extends AbstractVectorQParserBase {
final DenseVectorField denseVectorType = getCheckedFieldType(schemaField);
final String vectorToSearch = getVectorToSearch();
final int topK = localParams.getInt(TOP_K, DEFAULT_TOP_K);
- final int efSearch = localParams.getInt("ef-search", topK * 2);
+ final int efSearch = localParams.getInt("efSearch", topK * 2);
+ if (efSearch < topK) {
+ throw new IllegalArgumentException(
+ "efSearch (" + efSearch + ") must be >= topK (" + topK + ")");
+ }
+
+ final Integer filteredSearchThreshold = localParams.getInt(FILTERED_SEARCH_THRESHOLD);
return denseVectorType.getKnnVectorQuery(
schemaField.getName(),
diff --git a/solr/core/src/java/org/apache/solr/search/vector/SolrKnnByteVectorQuery.java b/solr/core/src/java/org/apache/solr/search/vector/SolrKnnByteVectorQuery.java
index e16866bacd5..6fd721cb414 100644
--- a/solr/core/src/java/org/apache/solr/search/vector/SolrKnnByteVectorQuery.java
+++ b/solr/core/src/java/org/apache/solr/search/vector/SolrKnnByteVectorQuery.java
@@ -24,6 +24,8 @@ public class SolrKnnByteVectorQuery extends KnnByteVectorQuery {
private final int topK;
public SolrKnnByteVectorQuery(String field, byte[] target, int topK, int efSearch, Query filter) {
+ // efSearch is used as 'k' to explore this many vectors in HNSW, then topK results are returned
+ // to the user
super(field, target, efSearch, filter);
this.topK = topK;
}
@@ -32,4 +34,18 @@ public class SolrKnnByteVectorQuery extends KnnByteVectorQuery {
protected TopDocs mergeLeafResults(TopDocs[] perLeafResults) {
return TopDocs.merge(topK, perLeafResults);
}
+
+ @Override
+ public boolean equals(Object obj) {
+ if (this == obj) return true;
+ if (!super.equals(obj)) return false;
+ if (getClass() != obj.getClass()) return false;
+ SolrKnnByteVectorQuery other = (SolrKnnByteVectorQuery) obj;
+ return this.topK == other.topK;
+ }
+
+ @Override
+ public int hashCode() {
+ return 31 * super.hashCode() + Integer.hashCode(topK);
+ }
}
diff --git a/solr/core/src/java/org/apache/solr/search/vector/SolrKnnFloatVectorQuery.java b/solr/core/src/java/org/apache/solr/search/vector/SolrKnnFloatVectorQuery.java
index 60eb4f19a57..68089d66014 100644
--- a/solr/core/src/java/org/apache/solr/search/vector/SolrKnnFloatVectorQuery.java
+++ b/solr/core/src/java/org/apache/solr/search/vector/SolrKnnFloatVectorQuery.java
@@ -25,6 +25,8 @@ public class SolrKnnFloatVectorQuery extends KnnFloatVectorQuery {
public SolrKnnFloatVectorQuery(
String field, float[] target, int topK, int efSearch, Query filter) {
+ // efSearch is used as 'k' to explore this many vectors in HNSW then topK results are returned
+ // to the user
super(field, target, efSearch, filter);
this.topK = topK;
}
@@ -33,4 +35,18 @@ public class SolrKnnFloatVectorQuery extends KnnFloatVectorQuery {
protected TopDocs mergeLeafResults(TopDocs[] perLeafResults) {
return TopDocs.merge(topK, perLeafResults);
}
+
+ @Override
+ public boolean equals(Object obj) {
+ if (this == obj) return true;
+ if (!super.equals(obj)) return false;
+ if (getClass() != obj.getClass()) return false;
+ SolrKnnFloatVectorQuery other = (SolrKnnFloatVectorQuery) obj;
+ return this.topK == other.topK;
+ }
+
+ @Override
+ public int hashCode() {
+ return 31 * super.hashCode() + Integer.hashCode(topK);
+ }
}
(solr) 11/12: Added changelog
Posted by is...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ishan pushed a commit to branch jira/solr-17927
in repository https://gitbox.apache.org/repos/asf/solr.git
commit 2d865f36fea6c8c4bf6b620d297912ee0c775abf
Author: Elia <e....@sease.io>
AuthorDate: Wed Dec 10 18:14:07 2025 +0100
Added changelog
---
...OLR-17928-ef-search.yml => SOLR-17928_added_efSearch_parameter.yml} | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/changelog/unreleased/puneet/SOLR-17928-ef-search.yml b/changelog/unreleased/SOLR-17928_added_efSearch_parameter.yml
similarity index 77%
rename from changelog/unreleased/puneet/SOLR-17928-ef-search.yml
rename to changelog/unreleased/SOLR-17928_added_efSearch_parameter.yml
index 583a6834c9c..01db5422157 100644
--- a/changelog/unreleased/puneet/SOLR-17928-ef-search.yml
+++ b/changelog/unreleased/SOLR-17928_added_efSearch_parameter.yml
@@ -1,8 +1,9 @@
# See https://github.com/apache/solr/blob/main/dev-docs/changelog.adoc
title: Added efSearch parameter to knn query, exposed efSearchScaleFactor that is used to calculate efSearch internally
-type: added # added, changed, fixed, deprecated, removed, dependency_update, security, other
+type: added
authors:
- name: Puneet Ahuja
+ - name: Elia Porciani
links:
- name: SOLR-17928
url: https://issues.apache.org/jira/browse/SOLR-17928
(solr) 08/12: Updated changelog
Posted by is...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ishan pushed a commit to branch jira/solr-17927
in repository https://gitbox.apache.org/repos/asf/solr.git
commit 21d38e61f10a96131b6bfe9cc1444a316e9bebac
Author: punAhuja <pu...@searchscale.com>
AuthorDate: Thu Nov 6 14:52:03 2025 +0530
Updated changelog
---
changelog/unreleased/puneet/SOLR-17928-ef-search.yml | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/changelog/unreleased/puneet/SOLR-17928-ef-search.yml b/changelog/unreleased/puneet/SOLR-17928-ef-search.yml
new file mode 100644
index 00000000000..583a6834c9c
--- /dev/null
+++ b/changelog/unreleased/puneet/SOLR-17928-ef-search.yml
@@ -0,0 +1,10 @@
+# See https://github.com/apache/solr/blob/main/dev-docs/changelog.adoc
+title: Added efSearch parameter to knn query, exposed efSearchScaleFactor that is used to calculate efSearch internally
+type: added # added, changed, fixed, deprecated, removed, dependency_update, security, other
+authors:
+ - name: Puneet Ahuja
+links:
+ - name: SOLR-17928
+ url: https://issues.apache.org/jira/browse/SOLR-17928
+issues:
+ - 17928