You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@solr.apache.org by ds...@apache.org on 2026/08/10 04:20:45 UTC
(solr) branch branch_9x updated (6164cd31b18 -> 21e6542287e)
This is an automated email from the ASF dual-hosted git repository.
dsmiley pushed a change to branch branch_9x
in repository https://gitbox.apache.org/repos/asf/solr.git
from 6164cd31b18 Changelog attribution for #4607
new 051b3883f95 SOLR-18327: Balance replicas API does not balance duplicate replicas on same node (#2503)
new 90aac33c33c PKIAuthenticationPlugin: handle non-base64 SolrAuthV2 signature (#4553)
new ed8fa346349 Develocity: publishing opt-in locally
new 7aaf2e8d3a9 SOLR-18329: DenseVectorField support existence queries (#4700)
new 612c3c34079 RAT: exclude dev-docs and AGENTS.md from license header check (follow-up to #4609)
new 21e6542287e Remove ASF License headers when possible. (#4609)
The 6 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
AGENTS.md | 16 -----
.../PR#4553-pki-v2-header-base64-fix.yml | 7 ++
.../SOLR-18329-densevector-existence-queries.yml | 8 +++
.../balance-duplicate-replicas-SOLR-18327.yml | 9 +++
dev-docs/README.adoc | 16 -----
dev-docs/gradle-help/README.md | 17 -----
dev-docs/lucene-upgrade.md | 17 -----
dev-docs/pmc-chair.adoc | 16 -----
dev-docs/ref-guide/antora.adoc | 16 -----
dev-docs/ref-guide/asciidoc-syntax.adoc | 16 -----
dev-docs/working-between-major-versions.adoc | 16 -----
gradle/develocity.gradle | 4 +-
gradle/validation/rat-sources.gradle | 4 ++
.../src/java/org/apache/solr/cluster/Shard.java | 9 +++
.../plugins/OrderedNodePlacementPlugin.java | 71 ++++++++++++++++++-
.../org/apache/solr/schema/DenseVectorField.java | 14 +++-
.../solr/security/PKIAuthenticationPlugin.java | 8 ++-
.../plugins/AffinityPlacementFactoryTest.java | 81 +++++++++++++++++++++-
.../plugins/MinimizeCoresPlacementFactoryTest.java | 8 ++-
.../apache/solr/schema/DenseVectorFieldTest.java | 34 ++++++---
.../solr/security/TestPKIAuthenticationPlugin.java | 18 +++++
.../query-guide/pages/dense-vector-search.adoc | 4 +-
22 files changed, 260 insertions(+), 149 deletions(-)
create mode 100644 changelog/unreleased/PR#4553-pki-v2-header-base64-fix.yml
create mode 100644 changelog/unreleased/SOLR-18329-densevector-existence-queries.yml
create mode 100644 changelog/unreleased/balance-duplicate-replicas-SOLR-18327.yml
(solr) 05/06: RAT: exclude dev-docs and AGENTS.md from license header check (follow-up to #4609)
Posted by ds...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dsmiley pushed a commit to branch branch_9x
in repository https://gitbox.apache.org/repos/asf/solr.git
commit 612c3c34079aa7426a68edcc2475965743ed6d4c
Author: David Smiley <ds...@apache.org>
AuthorDate: Mon Aug 10 00:15:43 2026 -0400
RAT: exclude dev-docs and AGENTS.md from license header check (follow-up to #4609)
These files are not "released" (not in the source or any other release),
mirroring the exclusion main gained via #4569's broader RAT overhaul.
Co-Authored-By: Claude Sonnet 5 <no...@anthropic.com>
---
gradle/validation/rat-sources.gradle | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/gradle/validation/rat-sources.gradle b/gradle/validation/rat-sources.gradle
index abd0db489cd..b813cf53b08 100644
--- a/gradle/validation/rat-sources.gradle
+++ b/gradle/validation/rat-sources.gradle
@@ -91,6 +91,10 @@ allprojects {
// Exclude github stuff (templates, workflows).
exclude ".github"
+ // Not "released" (not in the source or any other release), thus no header needed
+ exclude "dev-docs"
+ exclude "AGENTS.md"
+
exclude "dev-tools/scripts/cloud.sh"
exclude "dev-tools/scripts/README.md"
exclude "dev-tools/scripts/create_line_file_docs.py"
(solr) 02/06: PKIAuthenticationPlugin: handle non-base64 SolrAuthV2 signature (#4553)
Posted by ds...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dsmiley pushed a commit to branch branch_9x
in repository https://gitbox.apache.org/repos/asf/solr.git
commit 90aac33c33cddc71e34bfbeded6f1070be87fc12
Author: Jan Høydahl <ja...@apache.org>
AuthorDate: Wed Aug 5 11:43:13 2026 +0200
PKIAuthenticationPlugin: handle non-base64 SolrAuthV2 signature (#4553)
(cherry picked from commit d9e1468dda4d00f20e211e51a69c095bc3be5457)
---
.../unreleased/PR#4553-pki-v2-header-base64-fix.yml | 7 +++++++
.../apache/solr/security/PKIAuthenticationPlugin.java | 8 +++++++-
.../solr/security/TestPKIAuthenticationPlugin.java | 18 ++++++++++++++++++
3 files changed, 32 insertions(+), 1 deletion(-)
diff --git a/changelog/unreleased/PR#4553-pki-v2-header-base64-fix.yml b/changelog/unreleased/PR#4553-pki-v2-header-base64-fix.yml
new file mode 100644
index 00000000000..9e503d01e70
--- /dev/null
+++ b/changelog/unreleased/PR#4553-pki-v2-header-base64-fix.yml
@@ -0,0 +1,7 @@
+title: PKIAuthenticationPlugin now rejects a SolrAuthV2 header with a malformed signature using a 401 response, instead of returning a 500
+type: fixed
+authors:
+ - name: Jan Høydahl
+links:
+ - name: PR#4553
+ url: https://github.com/apache/solr/pull/4553
diff --git a/solr/core/src/java/org/apache/solr/security/PKIAuthenticationPlugin.java b/solr/core/src/java/org/apache/solr/security/PKIAuthenticationPlugin.java
index ce1c09a8742..c41b00ec66d 100644
--- a/solr/core/src/java/org/apache/solr/security/PKIAuthenticationPlugin.java
+++ b/solr/core/src/java/org/apache/solr/security/PKIAuthenticationPlugin.java
@@ -266,7 +266,13 @@ public class PKIAuthenticationPlugin extends AuthenticationPlugin
int sigStart = header.lastIndexOf(' ');
String data = header.substring(0, sigStart);
- byte[] sig = Base64.getDecoder().decode(header.substring(sigStart + 1));
+ byte[] sig;
+ try {
+ sig = Base64.getDecoder().decode(header.substring(sigStart + 1));
+ } catch (IllegalArgumentException e) {
+ log.warn("Could not parse signature in SolrAuthV2 header as base64");
+ return null;
+ }
PKIHeaderData rv = validateSignature(data, sig, key, false);
if (rv == null) {
log.warn("Failed to verify signature, trying after refreshing the key ");
diff --git a/solr/core/src/test/org/apache/solr/security/TestPKIAuthenticationPlugin.java b/solr/core/src/test/org/apache/solr/security/TestPKIAuthenticationPlugin.java
index 5c2cdd39c92..0f5a3f501e9 100644
--- a/solr/core/src/test/org/apache/solr/security/TestPKIAuthenticationPlugin.java
+++ b/solr/core/src/test/org/apache/solr/security/TestPKIAuthenticationPlugin.java
@@ -36,6 +36,7 @@ import javax.servlet.http.HttpServletResponse;
import org.apache.http.Header;
import org.apache.http.HttpHeaders;
import org.apache.http.auth.BasicUserPrincipal;
+import org.apache.http.message.BasicHeader;
import org.apache.http.message.BasicHttpRequest;
import org.apache.solr.SolrTestCaseJ4;
import org.apache.solr.common.params.ModifiableSolrParams;
@@ -204,6 +205,23 @@ public class TestPKIAuthenticationPlugin extends SolrTestCaseJ4 {
"Should not have proceeded after authentication failure", wrappedRequestByFilter.get());
}
+ @Test
+ public void testMalformedV2HeaderSignatureRejected() throws Exception {
+ headerKey = PKIAuthenticationPlugin.HEADER_V2;
+ header.set(new BasicHeader(headerKey, nodeName + " someuser 1234567890 not_base64!!!"));
+
+ HttpServletResponse response = mock(HttpServletResponse.class);
+ assertFalse(
+ "Should have rejected request with a non-base64 signature",
+ mock.authenticate(mockReq, response, filterChain));
+
+ verify(response).setHeader(HttpHeaders.WWW_AUTHENTICATE, PKIAuthenticationPlugin.HEADER_V2);
+ verify(response).sendError(ArgumentMatchers.eq(401), anyString());
+
+ assertNull(
+ "Should not have proceeded after authentication failure", wrappedRequestByFilter.get());
+ }
+
public void testParseCipher() {
for (String validUser : new String[] {"user1", "$", "some user", "some 123"}) {
for (long validTimestamp :
(solr) 01/06: SOLR-18327: Balance replicas API does not balance duplicate replicas on same node (#2503)
Posted by ds...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dsmiley pushed a commit to branch branch_9x
in repository https://gitbox.apache.org/repos/asf/solr.git
commit 051b3883f952a496c9b5a6dc258328aa85988d6a
Author: Jan Høydahl <ja...@apache.org>
AuthorDate: Mon Aug 3 09:50:18 2026 +0200
SOLR-18327: Balance replicas API does not balance duplicate replicas on same node (#2503)
(cherry picked from commit 5e58e05f0f34fe2d1c878fd78a3c556297359179)
---
.../balance-duplicate-replicas-SOLR-18327.yml | 9 +++
.../src/java/org/apache/solr/cluster/Shard.java | 9 +++
.../plugins/OrderedNodePlacementPlugin.java | 71 ++++++++++++++++++-
.../plugins/AffinityPlacementFactoryTest.java | 81 +++++++++++++++++++++-
.../plugins/MinimizeCoresPlacementFactoryTest.java | 8 ++-
5 files changed, 171 insertions(+), 7 deletions(-)
diff --git a/changelog/unreleased/balance-duplicate-replicas-SOLR-18327.yml b/changelog/unreleased/balance-duplicate-replicas-SOLR-18327.yml
new file mode 100644
index 00000000000..260b8c51249
--- /dev/null
+++ b/changelog/unreleased/balance-duplicate-replicas-SOLR-18327.yml
@@ -0,0 +1,9 @@
+# See https://github.com/apache/solr/blob/main/dev-docs/changelog.adoc
+
+title: Balance replicas API now enforces the rule of at most one replica per shard on the same node
+type: fixed
+authors:
+ - name: Jan Høydahl
+links:
+ - name: SOLR-18327
+ url: https://issues.apache.org/jira/browse/SOLR-18327
diff --git a/solr/core/src/java/org/apache/solr/cluster/Shard.java b/solr/core/src/java/org/apache/solr/cluster/Shard.java
index f62443e897b..cb89e68ad6d 100644
--- a/solr/core/src/java/org/apache/solr/cluster/Shard.java
+++ b/solr/core/src/java/org/apache/solr/cluster/Shard.java
@@ -28,6 +28,15 @@ public interface Shard {
*/
SolrCollection getCollection();
+ /**
+ * @return an id for this shard that is unique across the cluster, combining the collection name
+ * and the shard name. Useful e.g. as a map key when grouping replicas of the same shard,
+ * possibly from different collections.
+ */
+ default String getUniqueShardId() {
+ return getCollection().getName() + "%" + getShardName();
+ }
+
/**
* Returns the {@link Replica} of the given name for that shard, if such a replica exists.
*
diff --git a/solr/core/src/java/org/apache/solr/cluster/placement/plugins/OrderedNodePlacementPlugin.java b/solr/core/src/java/org/apache/solr/cluster/placement/plugins/OrderedNodePlacementPlugin.java
index 86ca79526b3..b78bdb3fe1c 100644
--- a/solr/core/src/java/org/apache/solr/cluster/placement/plugins/OrderedNodePlacementPlugin.java
+++ b/solr/core/src/java/org/apache/solr/cluster/placement/plugins/OrderedNodePlacementPlugin.java
@@ -202,14 +202,21 @@ public abstract class OrderedNodePlacementPlugin implements PlacementPlugin {
public BalancePlan computeBalancing(
BalanceRequest balanceRequest, PlacementContext placementContext) throws PlacementException {
Map<Replica, Node> replicaMovements = new HashMap<>();
- TreeSet<WeightedNode> orderedNodes = new TreeSet<>();
- orderedNodes.addAll(
+ Collection<WeightedNode> weightedNodes =
getWeightedNodes(
placementContext,
balanceRequest.getNodes(),
placementContext.getCluster().collections(),
true)
- .values());
+ .values();
+
+ // First move replicas that share a node with another replica of the same shard, a state the
+ // weight-based balancing below cannot necessarily detect since such duplicates do not have to
+ // make their node weigh more than its peers.
+ moveDuplicateShardReplicas(weightedNodes, replicaMovements);
+
+ TreeSet<WeightedNode> orderedNodes = new TreeSet<>();
+ orderedNodes.addAll(weightedNodes);
// While the node with the lowest weight still has room to take a replica from the node with the
// highest weight, loop
@@ -306,6 +313,64 @@ public abstract class OrderedNodePlacementPlugin implements PlacementPlugin {
.createBalancePlan(balanceRequest, replicaMovements);
}
+ /**
+ * Move replicas that share a node with another replica of the same shard to other nodes, since
+ * multiple replicas of the same shard on one node give neither availability nor capacity
+ * benefits. Placement will never create such a state, per the default {@link
+ * WeightedNode#canAddReplica(Replica)}, but users can, e.g. by adding a replica to an explicit
+ * node. Each duplicate replica is moved to the accepting node with the lowest projected weight
+ * with the replica added, like {@link #computePlacements(Collection, PlacementContext)} does.
+ */
+ private static void moveDuplicateShardReplicas(
+ Collection<WeightedNode> weightedNodes, Map<Replica, Node> replicaMovements) {
+ List<WeightedNode> sourceNodes = new ArrayList<>(weightedNodes);
+ sourceNodes.sort(Comparator.comparing(node -> node.getNode().getName()));
+ for (WeightedNode sourceNode : sourceNodes) {
+ Map<String, List<Replica>> replicasPerShard =
+ sourceNode.getAllReplicasOnNode().stream()
+ .collect(Collectors.groupingBy(replica -> replica.getShard().getUniqueShardId()));
+ for (List<Replica> shardReplicas : replicasPerShard.values()) {
+ if (shardReplicas.size() < 2) {
+ continue;
+ }
+ // Move extra replicas of this shard away until only one remains on the node, in replica
+ // name order like the weight-based balancing below. A replica that cannot be removed or has
+ // no eligible target is left in place and does not stop us from moving the others, so a
+ // single stuck replica no longer leaves the duplicate unresolved.
+ shardReplicas.sort(Comparator.comparing(Replica::getReplicaName));
+ int remainingOnNode = shardReplicas.size();
+ for (Replica replica : shardReplicas) {
+ if (remainingOnNode < 2) {
+ break;
+ }
+ if (!sourceNode.canRemoveReplicas(Set.of(replica)).isEmpty()) {
+ continue;
+ }
+ Optional<WeightedNode> targetNode =
+ weightedNodes.stream()
+ .filter(node -> !node.equals(sourceNode))
+ .filter(node -> node.canAddReplica(replica))
+ .min(
+ Comparator.<WeightedNode>comparingInt(
+ node -> node.calcRelevantWeightWithReplica(replica))
+ .thenComparing(Comparator.naturalOrder()));
+ if (targetNode.isPresent()) {
+ WeightedNode target = targetNode.get();
+ log.debug(
+ "Duplicate replica movement chosen. From: {}, To: {}, Replica: {}",
+ sourceNode,
+ target,
+ replica);
+ target.addReplica(replica);
+ sourceNode.removeReplica(replica);
+ replicaMovements.put(replica, target.getNode());
+ remainingOnNode--;
+ }
+ }
+ }
+ }
+ }
+
protected Map<Node, WeightedNode> getWeightedNodes(
PlacementContext placementContext,
Set<Node> nodes,
diff --git a/solr/core/src/test/org/apache/solr/cluster/placement/plugins/AffinityPlacementFactoryTest.java b/solr/core/src/test/org/apache/solr/cluster/placement/plugins/AffinityPlacementFactoryTest.java
index 424161dafa4..bfa5e91634f 100644
--- a/solr/core/src/test/org/apache/solr/cluster/placement/plugins/AffinityPlacementFactoryTest.java
+++ b/solr/core/src/test/org/apache/solr/cluster/placement/plugins/AffinityPlacementFactoryTest.java
@@ -1629,7 +1629,82 @@ public class AffinityPlacementFactoryTest extends AbstractPlacementFactoryTest {
// Each expected placement is represented as a string "col shard replica-type fromNode ->
// toNode"
- Set<String> expectedPlacements = Set.of("b 1 TLOG 0 -> 2", "b 1 NRT 3 -> 4");
+ // The duplicate replica of shard 1 of collection "a" is first moved off of node 0, then the
+ // regular weight-based balancing evens out the remaining core counts
+ Set<String> expectedPlacements = Set.of("a 1 NRT 0 -> 4", "a 2 TLOG 3 -> 2");
+ verifyBalancing(
+ expectedPlacements,
+ balancePlan,
+ collectionBuilder.getShardBuilders(),
+ clusterBuilder.buildLiveNodes());
+ }
+
+ @Test
+ public void testBalancingAvoidMultiReplicaOnNode() throws Exception {
+ // Cluster nodes and their attributes
+ Builders.ClusterBuilder clusterBuilder = Builders.newClusterBuilder().initializeLiveNodes(4);
+ List<Builders.NodeBuilder> nodeBuilders = clusterBuilder.getLiveNodeBuilders();
+
+ // The collection already exists with shards and replicas
+ Builders.CollectionBuilder collectionBuilder = Builders.newCollectionBuilder("a");
+ // Note that the collection as defined below is in a state that would NOT be returned by the
+ // placement plugin: shard 1 has two replicas on node 0 and two replicas on node 2.
+ // The plugin should still be able to place additional replicas as long as they don't break the
+ // rules.
+ List<List<String>> shardsReplicas = List.of(List.of("NRT 0", "NRT 0", "NRT 2", "NRT 2"));
+ collectionBuilder.customCollectionSetup(shardsReplicas, nodeBuilders);
+ clusterBuilder.addCollection(collectionBuilder);
+
+ // Add another collection. Note that this is also unbalanced with two replicas on node 1 and two
+ // replicas on node 3.
+ // The intent of this test is to demonstrate that the plugin can move replicas so that
+ // a given node does not have more than one replica of a given shard.
+ collectionBuilder = Builders.newCollectionBuilder("b");
+ shardsReplicas = List.of(List.of("NRT 1", "NRT 1", "NRT 3", "NRT 3"));
+ collectionBuilder.customCollectionSetup(shardsReplicas, nodeBuilders);
+ clusterBuilder.addCollection(collectionBuilder);
+
+ BalanceRequestImpl balanceRequest =
+ new BalanceRequestImpl(new HashSet<>(clusterBuilder.buildLiveNodes()));
+ BalancePlan balancePlan =
+ plugin.computeBalancing(balanceRequest, clusterBuilder.buildPlacementContext());
+
+ // Each expected placement is represented as a string "col shard replica-type fromNode ->
+ // toNode"
+ Set<String> expectedPlacements =
+ Set.of("a 1 NRT 0 -> 1", "a 1 NRT 2 -> 3", "b 1 NRT 3 -> 2", "b 1 NRT 1 -> 0");
+ verifyBalancing(
+ expectedPlacements,
+ balancePlan,
+ collectionBuilder.getShardBuilders(),
+ clusterBuilder.buildLiveNodes());
+ }
+
+ @Test
+ public void testBalancingAvoidMultiReplicaOnNodeAcrossAZs() throws Exception {
+ // Cluster with two nodes in each of two availability zones
+ Builders.ClusterBuilder clusterBuilder = Builders.newClusterBuilder().initializeLiveNodes(4);
+ List<Builders.NodeBuilder> nodeBuilders = clusterBuilder.getLiveNodeBuilders();
+ for (int i = 0; i < nodeBuilders.size(); i++) {
+ nodeBuilders
+ .get(i)
+ .setSysprop(AffinityPlacementConfig.AVAILABILITY_ZONE_SYSPROP, i < 2 ? "az1" : "az2");
+ }
+
+ // Shard 1 has both its replicas on node 0 in az1
+ Builders.CollectionBuilder collectionBuilder = Builders.newCollectionBuilder("a");
+ List<List<String>> shardsReplicas = List.of(List.of("NRT 0", "NRT 0"));
+ collectionBuilder.customCollectionSetup(shardsReplicas, nodeBuilders);
+ clusterBuilder.addCollection(collectionBuilder);
+
+ BalanceRequestImpl balanceRequest =
+ new BalanceRequestImpl(new HashSet<>(clusterBuilder.buildLiveNodes()));
+ BalancePlan balancePlan =
+ plugin.computeBalancing(balanceRequest, clusterBuilder.buildPlacementContext());
+
+ // The duplicate replica must move to a node in az2 to keep the availability zones balanced,
+ // not to the empty az1 node 1
+ Set<String> expectedPlacements = Set.of("a 1 NRT 0 -> 2");
verifyBalancing(
expectedPlacements,
balancePlan,
@@ -1668,7 +1743,9 @@ public class AffinityPlacementFactoryTest extends AbstractPlacementFactoryTest {
// Each expected placement is represented as a string "col shard replica-type fromNode ->
// toNode"
- Set<String> expectedPlacements = Set.of("a 1 NRT 3 -> 1", "a 2 NRT 3 -> 0");
+ // The duplicate replica of shard 1 is first moved off of node 0, then the regular weight-based
+ // balancing evens out the remaining core counts
+ Set<String> expectedPlacements = Set.of("a 1 NRT 0 -> 1", "a 1 NRT 3 -> 2", "a 2 NRT 3 -> 0");
verifyBalancing(
expectedPlacements,
balancePlan,
diff --git a/solr/core/src/test/org/apache/solr/cluster/placement/plugins/MinimizeCoresPlacementFactoryTest.java b/solr/core/src/test/org/apache/solr/cluster/placement/plugins/MinimizeCoresPlacementFactoryTest.java
index e4cb6e71a94..a8cbeddda94 100644
--- a/solr/core/src/test/org/apache/solr/cluster/placement/plugins/MinimizeCoresPlacementFactoryTest.java
+++ b/solr/core/src/test/org/apache/solr/cluster/placement/plugins/MinimizeCoresPlacementFactoryTest.java
@@ -257,7 +257,9 @@ public class MinimizeCoresPlacementFactoryTest extends AbstractPlacementFactoryT
// Each expected placement is represented as a string "col shard replica-type fromNode ->
// toNode"
- Set<String> expectedPlacements = Set.of("b 1 TLOG 0 -> 2", "b 1 NRT 3 -> 4");
+ // The duplicate replica of shard 1 of collection "a" is first moved off of node 0, then the
+ // regular weight-based balancing evens out the remaining core counts
+ Set<String> expectedPlacements = Set.of("a 1 NRT 0 -> 4", "a 2 TLOG 3 -> 2");
verifyBalancing(
expectedPlacements,
balancePlan,
@@ -296,7 +298,9 @@ public class MinimizeCoresPlacementFactoryTest extends AbstractPlacementFactoryT
// Each expected placement is represented as a string "col shard replica-type fromNode ->
// toNode"
- Set<String> expectedPlacements = Set.of("a 1 NRT 3 -> 1", "a 2 NRT 3 -> 0");
+ // The duplicate replica of shard 1 is first moved off of node 0, then the regular weight-based
+ // balancing evens out the remaining core counts
+ Set<String> expectedPlacements = Set.of("a 1 NRT 0 -> 1", "a 1 NRT 3 -> 2", "a 2 NRT 3 -> 0");
verifyBalancing(
expectedPlacements,
balancePlan,
(solr) 04/06: SOLR-18329: DenseVectorField support existence queries (#4700)
Posted by ds...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dsmiley pushed a commit to branch branch_9x
in repository https://gitbox.apache.org/repos/asf/solr.git
commit 7aaf2e8d3a9d77330a0a3f1621f5f57289fc6cc4
Author: linslee75 <li...@gmail.com>
AuthorDate: Fri Aug 7 14:11:15 2026 +0800
SOLR-18329: DenseVectorField support existence queries (#4700)
(cherry picked from commit 3c9fcc1ebf33ed337fcd100e125532bb8eff6a54)
---
.../SOLR-18329-densevector-existence-queries.yml | 8 +++++
.../org/apache/solr/schema/DenseVectorField.java | 14 ++++++++-
.../apache/solr/schema/DenseVectorFieldTest.java | 34 +++++++++++++++++-----
.../query-guide/pages/dense-vector-search.adoc | 4 ++-
4 files changed, 50 insertions(+), 10 deletions(-)
diff --git a/changelog/unreleased/SOLR-18329-densevector-existence-queries.yml b/changelog/unreleased/SOLR-18329-densevector-existence-queries.yml
new file mode 100644
index 00000000000..e4ad24480d1
--- /dev/null
+++ b/changelog/unreleased/SOLR-18329-densevector-existence-queries.yml
@@ -0,0 +1,8 @@
+title: DenseVectorField now supports existence queries (field:* / [* TO *])
+type: added
+authors:
+ - name: Li Enlai
+ nick: linslee75
+links:
+ - name: SOLR-18329
+ url: https://issues.apache.org/jira/browse/SOLR-18329
diff --git a/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java b/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
index f331d391cdf..c6607d0556f 100644
--- a/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
+++ b/solr/core/src/java/org/apache/solr/schema/DenseVectorField.java
@@ -36,6 +36,7 @@ import org.apache.lucene.index.VectorSimilarityFunction;
import org.apache.lucene.queries.function.ValueSource;
import org.apache.lucene.queries.function.valuesource.ByteKnnVectorFieldSource;
import org.apache.lucene.queries.function.valuesource.FloatKnnVectorFieldSource;
+import org.apache.lucene.search.FieldExistsQuery;
import org.apache.lucene.search.KnnByteVectorQuery;
import org.apache.lucene.search.KnnFloatVectorQuery;
import org.apache.lucene.search.Query;
@@ -379,6 +380,11 @@ public class DenseVectorField extends FloatPointField {
}
}
+ @Override
+ public Query getExistenceQuery(QParser parser, SchemaField field) {
+ return new FieldExistsQuery(field.getName());
+ }
+
/**
* Not Supported. Please use the {!knn} query parser to run K nearest neighbors search queries.
*/
@@ -389,7 +395,10 @@ public class DenseVectorField extends FloatPointField {
"Field Queries are not supported for Dense Vector fields. Please use the {!knn} query parser to run K nearest neighbors search queries.");
}
- /** Not Supported */
+ /**
+ * Unbounded ranges ({@code [* TO *]}) are treated as existence queries. Bounded range queries are
+ * not supported.
+ */
@Override
public Query getRangeQuery(
QParser parser,
@@ -398,6 +407,9 @@ public class DenseVectorField extends FloatPointField {
String part2,
boolean minInclusive,
boolean maxInclusive) {
+ if (part1 == null && part2 == null) {
+ return getExistenceQuery(parser, field);
+ }
throw new SolrException(
SolrException.ErrorCode.BAD_REQUEST,
"Range Queries are not supported for Dense Vector fields. Please use the {!knn} query parser to run K nearest neighbors search queries.");
diff --git a/solr/core/src/test/org/apache/solr/schema/DenseVectorFieldTest.java b/solr/core/src/test/org/apache/solr/schema/DenseVectorFieldTest.java
index c777ff43fc5..3ee93ae369e 100644
--- a/solr/core/src/test/org/apache/solr/schema/DenseVectorFieldTest.java
+++ b/solr/core/src/test/org/apache/solr/schema/DenseVectorFieldTest.java
@@ -529,18 +529,36 @@ public class DenseVectorFieldTest extends AbstractBadConfigTestBase {
}
}
- /** Not Supported */
@Test
- public void query_existenceSearch_shouldThrowException() throws Exception {
+ public void query_existenceSearch_shouldMatchDocumentsWithVector() throws Exception {
try {
initCore("solrconfig-basic.xml", "schema-densevector.xml");
- assertQEx(
- "Running Existence queries on a dense vector field should raise an Exception",
- "Range Queries are not supported for Dense Vector fields."
- + " Please use the {!knn} query parser to run K nearest neighbors search queries.",
- req("q", "vector:[* TO *]", "fl", "vector"),
- SolrException.ErrorCode.BAD_REQUEST);
+ SolrInputDocument doc1 = new SolrInputDocument();
+ doc1.addField("id", "1");
+ doc1.addField("vector", List.of(1.0f, 2.0f, 3.0f, 4.0f));
+ assertU(adoc(doc1));
+
+ SolrInputDocument doc2 = new SolrInputDocument();
+ doc2.addField("id", "2");
+ assertU(adoc(doc2));
+
+ assertU(commit());
+
+ assertJQ(
+ req("q", "vector:[* TO *]", "fl", "id", "sort", "id asc"),
+ "/response/numFound==1",
+ "/response/docs/[0]/id=='1'");
+
+ assertJQ(
+ req("q", "vector:*", "fl", "id", "sort", "id asc"),
+ "/response/numFound==1",
+ "/response/docs/[0]/id=='1'");
+
+ assertJQ(
+ req("q", "-vector:*", "fl", "id", "sort", "id asc"),
+ "/response/numFound==1",
+ "/response/docs/[0]/id=='2'");
} finally {
deleteCore();
}
diff --git a/solr/solr-ref-guide/modules/query-guide/pages/dense-vector-search.adoc b/solr/solr-ref-guide/modules/query-guide/pages/dense-vector-search.adoc
index 90eb122025f..8562228aec0 100644
--- a/solr/solr-ref-guide/modules/query-guide/pages/dense-vector-search.adoc
+++ b/solr/solr-ref-guide/modules/query-guide/pages/dense-vector-search.adoc
@@ -391,7 +391,9 @@ Apache Solr provides three query parsers that work with dense vector fields, tha
All parsers return scores for retrieved documents that are the approximate distance to the target vector (defined by the similarityFunction configured at indexing time) and both support "Pre-Filtering" the document graph to reduce the number of candidate vectors evaluated (without needing to compute their vector similarity distances).
-Common parameters for both query parsers are:
+While field queries and bounded range queries are not supported on `DenseVectorField`, existence queries such as `vector:*` (or the equivalent `vector:[* TO *]`) can be used to match documents that have a value for the vector field.
+
+Common parameters for all query parsers are:
`f`::
+
(solr) 06/06: Remove ASF License headers when possible. (#4609)
Posted by ds...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dsmiley pushed a commit to branch branch_9x
in repository https://gitbox.apache.org/repos/asf/solr.git
commit 21e6542287ef7699c9da9414d5baf67a61809ed0
Author: David Smiley <ds...@apache.org>
AuthorDate: Wed Jul 8 22:06:56 2026 -0400
Remove ASF License headers when possible. (#4609)
These files are not "released" (not in the source or any other release anymore).
(cherry picked from commit bfd7ff1af03490aa51ce8508a63931c8eff484da)
---
AGENTS.md | 16 ----------------
dev-docs/README.adoc | 16 ----------------
dev-docs/gradle-help/README.md | 17 -----------------
dev-docs/lucene-upgrade.md | 17 -----------------
dev-docs/pmc-chair.adoc | 16 ----------------
dev-docs/ref-guide/antora.adoc | 16 ----------------
dev-docs/ref-guide/asciidoc-syntax.adoc | 16 ----------------
dev-docs/working-between-major-versions.adoc | 16 ----------------
8 files changed, 130 deletions(-)
diff --git a/AGENTS.md b/AGENTS.md
index 4395a2a4ee5..46b8c69a2a4 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -1,19 +1,3 @@
-<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
- -->
# AGENTS.md for Apache Solr
While README.md and CONTRIBUTING.md are mainly written for humans, this file is a condensed knowledge base for LLM coding agents on the Solr codebase. See https://agents.md for more info and how to make various coding assistants consume this file. Also see `dev-docs/how-to-contribute.adoc` for some guidelines when using genAI to contribute to Solr.
diff --git a/dev-docs/README.adoc b/dev-docs/README.adoc
index bb0d17d75f9..173aeacbd4b 100644
--- a/dev-docs/README.adoc
+++ b/dev-docs/README.adoc
@@ -1,20 +1,4 @@
= Solr Developer Docs
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements. See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership. The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// with the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied. See the License for the
-// specific language governing permissions and limitations
-// under the License.
This directory includes information for Solr developers. There are some other
sources of information for learning about developing on Solr:
diff --git a/dev-docs/gradle-help/README.md b/dev-docs/gradle-help/README.md
index 58df221401d..d3a43d90fae 100644
--- a/dev-docs/gradle-help/README.md
+++ b/dev-docs/gradle-help/README.md
@@ -1,20 +1,3 @@
-<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
# Gradle Help Documentation
This directory contains text files that provide help documentation for various Gradle tasks and project workflows in Solr.
diff --git a/dev-docs/lucene-upgrade.md b/dev-docs/lucene-upgrade.md
index 21f6051f223..c3c080626f6 100644
--- a/dev-docs/lucene-upgrade.md
+++ b/dev-docs/lucene-upgrade.md
@@ -1,20 +1,3 @@
-<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
- -->
-
# Lucene upgrade steps
## Read
diff --git a/dev-docs/pmc-chair.adoc b/dev-docs/pmc-chair.adoc
index 8c5f3b1ad23..24cb1db8045 100644
--- a/dev-docs/pmc-chair.adoc
+++ b/dev-docs/pmc-chair.adoc
@@ -1,21 +1,5 @@
= Tips & Tricks for PMC Chair
:toc: left
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements. See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership. The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// with the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied. See the License for the
-// specific language governing permissions and limitations
-// under the License.
Congratulations on becoming the Chair of the Lucene PMC! Thank you for accepting the role.
diff --git a/dev-docs/ref-guide/antora.adoc b/dev-docs/ref-guide/antora.adoc
index 2f3306f35c7..b661f7a521b 100644
--- a/dev-docs/ref-guide/antora.adoc
+++ b/dev-docs/ref-guide/antora.adoc
@@ -1,21 +1,5 @@
= Working with HTML Templates
:toc:
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements. See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership. The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// with the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied. See the License for the
-// specific language governing permissions and limitations
-// under the License.
The Solr Ref Guide uses Antora to build the HTML version of the site.
diff --git a/dev-docs/ref-guide/asciidoc-syntax.adoc b/dev-docs/ref-guide/asciidoc-syntax.adoc
index 510bea5aec7..2b590e145ef 100644
--- a/dev-docs/ref-guide/asciidoc-syntax.adoc
+++ b/dev-docs/ref-guide/asciidoc-syntax.adoc
@@ -1,21 +1,5 @@
= AsciiDoc Syntax Cheatsheet
:toc:
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements. See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership. The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// with the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied. See the License for the
-// specific language governing permissions and limitations
-// under the License.
The definitive manual on AsciiDoc syntax is in the https://docs.asciidoctor.org/asciidoctor.js/latest[Asciidoctor.js documentation].
To help people get started, however, here is a simpler cheat sheet.
diff --git a/dev-docs/working-between-major-versions.adoc b/dev-docs/working-between-major-versions.adoc
index 4c41fb8073a..af61cda5e47 100644
--- a/dev-docs/working-between-major-versions.adoc
+++ b/dev-docs/working-between-major-versions.adoc
@@ -1,20 +1,4 @@
= Working between Multiple Major versions
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements. See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership. The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// with the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied. See the License for the
-// specific language governing permissions and limitations
-// under the License.
Working between multiple major versions of `solr` is often a necessary part of committing code, due to backports and testing.
For some versions, this is an even bigger issue because `8.x` and `9.x` use different build systems, ant and gradle, and different repository locations.
(solr) 03/06: Develocity: publishing opt-in locally
Posted by ds...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dsmiley pushed a commit to branch branch_9x
in repository https://gitbox.apache.org/repos/asf/solr.git
commit ed8fa346349c8f851cb321f79d7fbc606cff3e51
Author: David Smiley <ds...@apache.org>
AuthorDate: Fri Aug 7 00:54:33 2026 -0400
Develocity: publishing opt-in locally
(cherry picked from commit 1aa47efe9767026207e6c623b5d505572daef857)
---
gradle/develocity.gradle | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/gradle/develocity.gradle b/gradle/develocity.gradle
index 811b0ed7b77..5fe41f18935 100644
--- a/gradle/develocity.gradle
+++ b/gradle/develocity.gradle
@@ -27,7 +27,9 @@ develocity {
buildScan {
uploadInBackground = !isCIBuild
- publishing.onlyIf { it.authenticated }
+ // Publish on CI (assume key env var set) or locally when opted-in via -Pscan
+ def isScanOptIn = providers.gradleProperty('scan').present
+ publishing.onlyIf { (isCIBuild || isScanOptIn) && it.authenticated }
obfuscation {
ipAddresses { addresses -> addresses.collect { address -> "0.0.0.0"} }
}