You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "dsmiley (via GitHub)" <gi...@apache.org> on 2026/06/09 13:39:01 UTC

[PR] Refactor CLI StreamTool SolrClientCache usage [solr]

dsmiley opened a new pull request, #4513:
URL: https://github.com/apache/solr/pull/4513

   * don't call SolrClientCache.setBasicAuthCredentials, to be deprecated soon   (primary motivation).  Instead set this on the client passed into SCC.
   * insist on a StreamContext, even if "remote mode".  Code was a bit of a mess, doing redundant things or not doing what it "should" do


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


Re: [PR] Refactor CLI StreamTool SolrClientCache usage [solr]

Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #4513:
URL: https://github.com/apache/solr/pull/4513#issuecomment-4742746935

   i have tried it...   so if thet ests all pass, I'll merge.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


Re: [PR] Refactor CLI StreamTool SolrClientCache usage [solr]

Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on code in PR #4513:
URL: https://github.com/apache/solr/pull/4513#discussion_r3391083692


##########
solr/core/src/java/org/apache/solr/cli/StreamTool.java:
##########
@@ -310,16 +312,8 @@ private PushBackStream doRemoteMode(CommandLine cli, String expr) throws Excepti
           "The stdin() expression is only usable with --worker local set up.");
     }
 
-    final SolrStream solrStream =
-        new SolrStream(solrUrl + "/solr/" + collection, params("qt", "/stream", "expr", expr));
-
-    String credentials = cli.getOptionValue(CommonCLIOptions.CREDENTIALS_OPTION);
-    if (credentials != null) {
-      String username = credentials.split(":")[0];
-      String password = credentials.split(":")[1];
-      solrStream.setCredentials(username, password);
-    }
-    return new PushBackStream(solrStream);
+    return new PushBackStream(
+        new SolrStream(solrUrl + "/solr/" + collection, params("qt", "/stream", "expr", expr)));

Review Comment:
   i feel like this should be `"/solr/" + collection + "/stream", params("expr",expr)`...   but that is a bigger change.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


Re: [PR] Refactor CLI StreamTool SolrClientCache usage [solr]

Posted by "dsmiley (via GitHub)" <gi...@apache.org>.
dsmiley commented on code in PR #4513:
URL: https://github.com/apache/solr/pull/4513#discussion_r3381155254


##########
solr/core/src/java/org/apache/solr/cli/CLIUtils.java:
##########
@@ -180,7 +180,7 @@ public static SolrClient getSolrClient(CommandLine cli) throws Exception {
    * is used, and warns those users. In the future we'll have urls ending with /api as well.
    *
    * @param solrUrl The user supplied url to Solr.
-   * @return the solrUrl in the format that Solr expects to see internally.
+   * @return a URL without any path, e.g. {@code http://localhost:8983}
    */
   public static String normalizeSolrUrl(String solrUrl) {

Review Comment:
   IMO these "normalize" methods are non-obvious since the URL normalization I'm used to seeing within Solr produce Solr URLs ending with "/solr".  The javadoc here refers to what I'm used to as "legacy".  Wow... am I a dinosaur or has v2 taken over?  SolrJ didn't get the memo, I can tell you that.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


Re: [PR] Refactor CLI StreamTool SolrClientCache usage [solr]

Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on code in PR #4513:
URL: https://github.com/apache/solr/pull/4513#discussion_r3381422752


##########
solr/core/src/java/org/apache/solr/cli/CLIUtils.java:
##########
@@ -180,7 +180,7 @@ public static SolrClient getSolrClient(CommandLine cli) throws Exception {
    * is used, and warns those users. In the future we'll have urls ending with /api as well.
    *
    * @param solrUrl The user supplied url to Solr.
-   * @return the solrUrl in the format that Solr expects to see internally.
+   * @return a URL without any path, e.g. {@code http://localhost:8983}
    */
   public static String normalizeSolrUrl(String solrUrl) {

Review Comment:
   I think a little bit dinosaur...   But not a lot dinosaur...   We've been trying to simpliy users lives so they don't have to think about /solr or /v2 or /api all the time, unless is critical.   So, we normalize to the root version, and then whatever tool can append what it needs...



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


Re: [PR] Refactor CLI StreamTool SolrClientCache usage [solr]

Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #4513:
URL: https://github.com/apache/solr/pull/4513#issuecomment-4698889984

   @dsmiley make sure after my chagnes we still are doing what you watned...  I had to move some things around to make it work.   Took opportunity to imrpvoe where validation of inputs takes place.  
   
   I marked it as no-changelog as this isnt' user facing.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


Re: [PR] Refactor CLI StreamTool SolrClientCache usage [solr]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #4513:
URL: https://github.com/apache/solr/pull/4513#discussion_r3436444894


##########
solr/core/src/java/org/apache/solr/cli/StreamTool.java:
##########
@@ -226,36 +240,58 @@ public void runImpl(CommandLine cli) throws Exception {
         }
       }
     } finally {
-      pushBackStream.close();
-      solrClientCache.close();
+      if (pushBackStream != null) {
+        pushBackStream.close();
+      }
+      streamContext.getSolrClientCache().close();
     }
 
     echoIfVerbose("StreamTool -- Done.");
   }
 
+  private StreamContext createStreamContext(CommandLine cli) throws Exception {
+    var jettyClientBuilder = new HttpJettySolrClient.Builder();
+    String credentials = cli.getOptionValue(CommonCLIOptions.CREDENTIALS_OPTION);
+    if (credentials != null) {
+      String[] userPass = credentials.split(":");
+      jettyClientBuilder.withBasicAuthCredentials(userPass[0], userPass[1]);
+    }

Review Comment:
   `--credentials` parsing via `split(":")` can throw (no colon) and also breaks passwords containing ':' (only the 2nd segment is used). Prefer the existing `withOptionalBasicAuthCredentials(credentials)` helper which validates formatting and preserves everything after the first ':'.



##########
solr/core/src/java/org/apache/solr/cli/StreamTool.java:
##########
@@ -166,14 +164,30 @@ public void runImpl(CommandLine cli) throws Exception {
       }
     }
 
-    PushBackStream pushBackStream;
-    if (execution.equalsIgnoreCase("local")) {
-      pushBackStream = doLocalMode(cli, expr);
-    } else {
-      pushBackStream = doRemoteMode(cli, expr);
+    // Validate inputs before opening any connection to Solr.
+    boolean local = execution.equalsIgnoreCase("local");
+    if (!local) {
+      if (!cli.hasOption(COLLECTION_OPTION)) {
+        throw new IllegalStateException(
+            "You must provide --name COLLECTION with --execution remote parameter.");
+      }
+      if (expr.toLowerCase(Locale.ROOT).contains("stdin(")) {
+        throw new IllegalStateException(
+            "The stdin() expression is only usable with --worker local set up.");
+      }

Review Comment:
   The error message mentions a non-existent `--worker local` option; this tool uses `--execution local`. This can confuse users when `stdin()` is used with remote execution.



##########
solr/core/src/java/org/apache/solr/cli/StreamTool.java:
##########
@@ -226,36 +240,58 @@ public void runImpl(CommandLine cli) throws Exception {
         }
       }
     } finally {
-      pushBackStream.close();
-      solrClientCache.close();
+      if (pushBackStream != null) {
+        pushBackStream.close();
+      }
+      streamContext.getSolrClientCache().close();
     }
 
     echoIfVerbose("StreamTool -- Done.");
   }
 
+  private StreamContext createStreamContext(CommandLine cli) throws Exception {
+    var jettyClientBuilder = new HttpJettySolrClient.Builder();
+    String credentials = cli.getOptionValue(CommonCLIOptions.CREDENTIALS_OPTION);
+    if (credentials != null) {
+      String[] userPass = credentials.split(":");
+      jettyClientBuilder.withBasicAuthCredentials(userPass[0], userPass[1]);
+    }
+    HttpJettySolrClient client = jettyClientBuilder.build();
+
+    // subclass so we can ensure our client is closed when the cache is closed
+    var solrClientCache =
+        new SolrClientCache(client) {
+          @Override
+          public synchronized void close() {
+            super.close();
+            client.close();
+          }

Review Comment:
   `SolrClientCache.close()` is designed to be non-throwing (it uses `IOUtils.closeQuietly`). Calling `client.close()` directly here can throw a `RuntimeException` and potentially mask an earlier failure from the stream execution. Use `IOUtils.closeQuietly(client)` in a `finally` to ensure cleanup never overrides the primary exception.



##########
solr/core/src/java/org/apache/solr/cli/StreamTool.java:
##########
@@ -226,36 +240,58 @@ public void runImpl(CommandLine cli) throws Exception {
         }
       }
     } finally {
-      pushBackStream.close();
-      solrClientCache.close();
+      if (pushBackStream != null) {
+        pushBackStream.close();
+      }
+      streamContext.getSolrClientCache().close();
     }
 
     echoIfVerbose("StreamTool -- Done.");
   }
 
+  private StreamContext createStreamContext(CommandLine cli) throws Exception {
+    var jettyClientBuilder = new HttpJettySolrClient.Builder();
+    String credentials = cli.getOptionValue(CommonCLIOptions.CREDENTIALS_OPTION);
+    if (credentials != null) {
+      String[] userPass = credentials.split(":");
+      jettyClientBuilder.withBasicAuthCredentials(userPass[0], userPass[1]);
+    }
+    HttpJettySolrClient client = jettyClientBuilder.build();
+
+    // subclass so we can ensure our client is closed when the cache is closed
+    var solrClientCache =
+        new SolrClientCache(client) {
+          @Override
+          public synchronized void close() {
+            super.close();
+            client.close();
+          }
+        };
+
+    var solrConnection = CLIUtils.getSolrConnection(cli);
+    echoIfVerbose("Connecting to Solr at " + solrConnection);
+
+    StreamContext streamContext = new StreamContext();
+    streamContext.setSolrClientCache(solrClientCache);
+
+    StreamFactory streamFactory = new DefaultStreamFactory();
+    streamFactory.withDefaultSolrConnection(solrConnection);
+    streamContext.setStreamFactory(streamFactory);
+    return streamContext;

Review Comment:
   If anything throws after constructing `client`/`solrClientCache` (e.g., `CLIUtils.getSolrConnection(cli)` can throw), the current method will leak the underlying Jetty client because the cache is only closed by the caller on the success path. Wrap the remainder of the method in a try/catch and close the cache on failure.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


Re: [PR] Refactor CLI StreamTool SolrClientCache usage [solr]

Posted by "dsmiley (via GitHub)" <gi...@apache.org>.
dsmiley commented on code in PR #4513:
URL: https://github.com/apache/solr/pull/4513#discussion_r3410023427


##########
solr/core/src/java/org/apache/solr/cli/StreamTool.java:
##########
@@ -257,7 +273,11 @@ public synchronized void close() {
 
     StreamContext streamContext = new StreamContext();
     streamContext.setSolrClientCache(solrClientCache);
-    streamContext.getStreamFactory().withDefaultSolrConnection(solrConnection);
+
+    StreamFactory streamFactory = new StreamFactory();
+    Lang.register(streamFactory);

Review Comment:
   Can use `DefaultStreamFactory` more simply, which calls Lang.register



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


Re: [PR] Refactor CLI StreamTool SolrClientCache usage [solr]

Posted by "dsmiley (via GitHub)" <gi...@apache.org>.
dsmiley commented on code in PR #4513:
URL: https://github.com/apache/solr/pull/4513#discussion_r3382037359


##########
solr/core/src/java/org/apache/solr/cli/CLIUtils.java:
##########
@@ -180,7 +180,7 @@ public static SolrClient getSolrClient(CommandLine cli) throws Exception {
    * is used, and warns those users. In the future we'll have urls ending with /api as well.
    *
    * @param solrUrl The user supplied url to Solr.
-   * @return the solrUrl in the format that Solr expects to see internally.
+   * @return a URL without any path, e.g. {@code http://localhost:8983}
    */
   public static String normalizeSolrUrl(String solrUrl) {

Review Comment:
   IMO URL utility methods should be centralized to URLUtil.  And naming reconsidered.  If this method produces URLs without any path, then this is an _opportunity_ to very simply state that in the method name rather than ambiguous "normalization", (it once wasn't ambiguous but in our transition period, is ambiguous).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


Re: [PR] Refactor CLI StreamTool SolrClientCache usage [solr]

Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on code in PR #4513:
URL: https://github.com/apache/solr/pull/4513#discussion_r3391009707


##########
solr/core/src/java/org/apache/solr/cli/StreamTool.java:
##########
@@ -227,35 +227,52 @@ public void runImpl(CommandLine cli) throws Exception {
       }
     } finally {
       pushBackStream.close();
-      solrClientCache.close();
+      streamContext.getSolrClientCache().close();
     }
 
     echoIfVerbose("StreamTool -- Done.");
   }
 
+  private StreamContext createStreamContext(CommandLine cli) throws Exception {
+    var jettyClientBuilder = new HttpJettySolrClient.Builder();
+    String credentials = cli.getOptionValue(CommonCLIOptions.CREDENTIALS_OPTION);
+    if (credentials != null) {
+      String[] userPass = credentials.split(":");
+      jettyClientBuilder.withBasicAuthCredentials(userPass[0], userPass[1]);
+    }
+    HttpJettySolrClient client = jettyClientBuilder.build();
+
+    // subclass so we can ensure our client is closed when the cache is closed

Review Comment:
   This is some magic!  If this isn't there, do we see errors?  



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


Re: [PR] Refactor CLI StreamTool SolrClientCache usage [solr]

Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on code in PR #4513:
URL: https://github.com/apache/solr/pull/4513#discussion_r3436397742


##########
solr/core/src/java/org/apache/solr/cli/StreamTool.java:
##########
@@ -257,7 +273,11 @@ public synchronized void close() {
 
     StreamContext streamContext = new StreamContext();
     streamContext.setSolrClientCache(solrClientCache);
-    streamContext.getStreamFactory().withDefaultSolrConnection(solrConnection);
+
+    StreamFactory streamFactory = new StreamFactory();
+    Lang.register(streamFactory);

Review Comment:
   Nice!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


Re: [PR] Refactor CLI StreamTool SolrClientCache usage [solr]

Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on code in PR #4513:
URL: https://github.com/apache/solr/pull/4513#discussion_r3448858457


##########
solr/core/src/java/org/apache/solr/cli/StreamTool.java:
##########
@@ -166,14 +164,30 @@ public void runImpl(CommandLine cli) throws Exception {
       }
     }
 
-    PushBackStream pushBackStream;
-    if (execution.equalsIgnoreCase("local")) {
-      pushBackStream = doLocalMode(cli, expr);
-    } else {
-      pushBackStream = doRemoteMode(cli, expr);
+    // Validate inputs before opening any connection to Solr.
+    boolean local = execution.equalsIgnoreCase("local");
+    if (!local) {
+      if (!cli.hasOption(COLLECTION_OPTION)) {
+        throw new IllegalStateException(
+            "You must provide --name COLLECTION with --execution remote parameter.");
+      }
+      if (expr.toLowerCase(Locale.ROOT).contains("stdin(")) {
+        throw new IllegalStateException(
+            "The stdin() expression is only usable with --worker local set up.");
+      }

Review Comment:
   nice



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


Re: [PR] Refactor CLI StreamTool SolrClientCache usage [solr]

Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #4513:
URL: https://github.com/apache/solr/pull/4513#issuecomment-4661305903

   Is #4510 potentially overlapping with this work?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


Re: [PR] Refactor CLI StreamTool SolrClientCache usage [solr]

Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh merged PR #4513:
URL: https://github.com/apache/solr/pull/4513


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org


Re: [PR] Refactor CLI StreamTool SolrClientCache usage [solr]

Posted by "dsmiley (via GitHub)" <gi...@apache.org>.
dsmiley commented on PR #4513:
URL: https://github.com/apache/solr/pull/4513#issuecomment-4660338426

   Disclaimer:  I haven't tested or even used it in a realistic way.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org