You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@solr.apache.org by ds...@apache.org on 2024/02/04 00:26:21 UTC
(solr) 02/02: Optimize ZkStateReader.refreshCollectionList (#2217)
This is an automated email from the ASF dual-hosted git repository.
dsmiley pushed a commit to branch branch_9x
in repository https://gitbox.apache.org/repos/asf/solr.git
commit 563acd2cac85dbdb5ad58ea3ceb2630720dadf8d
Author: David Smiley <ds...@apache.org>
AuthorDate: Sat Feb 3 16:28:11 2024 -0500
Optimize ZkStateReader.refreshCollectionList (#2217)
Avoid O(N^2) for many collections.
The arg to retain(col) should ideally always be a Set.
---
solr/CHANGES.txt | 5 ++++-
.../src/java/org/apache/solr/common/cloud/ZkStateReader.java | 2 +-
2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/solr/CHANGES.txt b/solr/CHANGES.txt
index a34cfdbcff2..dc99b116d59 100644
--- a/solr/CHANGES.txt
+++ b/solr/CHANGES.txt
@@ -21,6 +21,9 @@ Optimizations
---------------------
* SOLR-17144: Close searcherExecutor thread per core after 1 minute (Pierre Salagnac, Christine Poerschke)
+* GITHUB#2217: Scale to 10K+ collections better in ZkStateReader.refreshCollectionsList (David Smiley)
+
+
Bug Fixes
---------------------
(No changes)
@@ -103,7 +106,7 @@ Improvements
Now available at `GET /api/node/commands/someRequestId` (Sanjay Dutt via Jason Gerlowski)
* SOLR-17068: bin/solr post CLI use of options is now aligned closely with bin/post CLI tool, and is consistently
- referenced throughout the Ref Guide and source code, and is used through out our tests. The bin/post tool
+ referenced throughout the Ref Guide and source code, and is used through out our tests. The bin/post tool
remains and has been tested to work. (Eric Pugh)
Optimizations
diff --git a/solr/solrj-zookeeper/src/java/org/apache/solr/common/cloud/ZkStateReader.java b/solr/solrj-zookeeper/src/java/org/apache/solr/common/cloud/ZkStateReader.java
index d3f27730139..ffb19c3a32f 100644
--- a/solr/solrj-zookeeper/src/java/org/apache/solr/common/cloud/ZkStateReader.java
+++ b/solr/solrj-zookeeper/src/java/org/apache/solr/common/cloud/ZkStateReader.java
@@ -676,7 +676,7 @@ public class ZkStateReader implements SolrCloseable {
// Don't mess with watchedCollections, they should self-manage.
// First, drop any children that disappeared.
- this.lazyCollectionStates.keySet().retainAll(children);
+ this.lazyCollectionStates.keySet().retainAll(new HashSet<>(children)); // Set avoids O(N^2)
for (String coll : children) {
// We will create an eager collection for any interesting collections, so don't add to lazy.
if (!collectionWatches.watchedCollections().contains(coll)) {