You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "Daniel Lowe (Jira)" <ji...@apache.org> on 2025/11/05 14:11:00 UTC

[jira] [Commented] (SOLR-17433) SolrStream by default creates a Http2SolrClient that can only stream for 60 seconds

    [ https://issues.apache.org/jira/browse/SOLR-17433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18035589#comment-18035589 ] 

Daniel Lowe commented on SOLR-17433:
------------------------------------

{quote}there are 2 timeouts *that can be set on the SolrClientCache* that affect the streaming request
...
You can set both of these limits *via system properties* if the defaults are not sufficient for your usecase:
{quote}
Setting it via system properties as opposed to on the specific SolrClientCache sounds more like a workaround rather than a solution given a 60 second timeout is reasonable for most other types of request

{quote}SOLR-17143: if this setting this as a system propery is not an option you could also pass a custom SolrClientCache to the StreamContext with a client that is already configured for a higher timeout. {quote}

While this does definitely offer a solution, as empirically using the HTTP1 implementation that had more appopriate timouts did avoid the issue, this still seems quite clunky to simply get what I think would be the expected out of the box behaviour i.e. that streams continue streaming until they exhaust their content.

> SolrStream by default creates a Http2SolrClient that can only stream for 60 seconds
> -----------------------------------------------------------------------------------
>
>                 Key: SOLR-17433
>                 URL: https://issues.apache.org/jira/browse/SOLR-17433
>             Project: Solr
>          Issue Type: Bug
>          Components: SolrJ, streaming expressions
>    Affects Versions: 9.4, 9.5, 9.6
>            Reporter: Daniel Lowe
>            Priority: Major
>
> This behaviour changed in SolrJ 9.4, in earlier versions a {{HttpSolrClient}} was directly instantiated {{{}in SolrStream's open() method, while from SolrJ 9.4 a client is instantiated via a SolrClientCache{}}}.
> I believe the expected behaviour is that the client that the SolrClientCache instantiates should have appropriate timeout settings for streaming usage. I think the 60 second timeout arises from {{{}SolrClientCache{}}}'s MIN_TIMEOUT of 60 seconds. I note that that this is inconsistent with HttpClientUtil.DEFAULT_SO_TIMEOUT (600 seconds) which is used by default if you manually instantiate a {{Http2SolrClient or HttpJdkSolrClient.}}
> Given what I think the typical use case is for SolrStream ideally I don't think there should even be a timeout assuming the stream is still returning content.
> I think you can restore the original behaviour by using:
>  
> {code:java}
> context.setSolrClientCache(new SolrClientCache(new HttpSolrClient.Builder(url).build().getHttpClient())); {code}
>  
>  
> Example code:
>  
> {code:java}
> ModifiableSolrParams params = new ModifiableSolrParams();
> params.set("expr", "search(<your_collection_name>,qt=\"/export\",q=\"" + someLongRunningQuery + "\", fl=\"id\", sort=\"id asc\")");
> params.set("qt", "/stream");
> try (TupleStream solrStream = new SolrStream(url, params)) {
>   StreamContext context = new StreamContext();
>   solrStream.setStreamContext(context);
>   solrStream.open();
>   Tuple tuple;
>   while (!(tuple = solrStream.read()).EOF) {
>   }
> }
> {code}
>  
>  
> {code:java}
> Caused by: java.util.concurrent.TimeoutException: Total timeout 60000 ms elapsed	at org.eclipse.jetty.client.HttpConnection$RequestTimeouts.onExpired(HttpConnection.java:342)	at org.eclipse.jetty.client.HttpConnection$RequestTimeouts.onExpired(HttpConnection.java:322)	at org.eclipse.jetty.io.CyclicTimeouts.onTimeoutExpired(CyclicTimeouts.java:110)	at org.eclipse.jetty.io.CyclicTimeouts$Timeouts.onTimeoutExpired(CyclicTimeouts.java:197)	at org.eclipse.jetty.io.CyclicTimeout$Wakeup.run(CyclicTimeout.java:294)	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)	at java.base/java.lang.Thread.run(Thread.java:829){code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org