[jira] [Created] (SOLR-18488) HttpJettySolrClient: selector work is scheduled on the shared client executor instead of a dedicated thread, adding latency under load

["Robert Lucarini (Jira)" <ji...@apache.org>]

Robert Lucarini created SOLR-18488:
--------------------------------------

             Summary: HttpJettySolrClient: selector work is scheduled on the shared client executor instead of a dedicated thread, adding latency under load
                 Key: SOLR-18488
                 URL: https://issues.apache.org/jira/browse/SOLR-18488
             Project: Solr
          Issue Type: Improvement
          Components: clients - java
    Affects Versions: 10.0
            Reporter: Robert Lucarini


HttpJettySolrClient's ManagedSelector does not run its NIO select loop on a dedicated thread. Instead, each selector's select() call is "produced" as a task via AdaptiveExecutionStrategy and dispatched onto the same general-purpose executor pool used for all other client work (connection setup, request writing, response parsing). Confirmed via live thread dumps under sustained concurrent load: with the default useHttp1_1(true) configuration (2 selectors), both selector threads consistently appear as h2sc-* executor-pool threads running ManagedSelector$SelectorProducer.produce() → AdaptiveExecutionStrategy.produceTask(), not as separately-named, always-running threads.

This differs from HttpJdkSolrClient's underlying jdk.internal.net.http.HttpClientImpl, which creates exactly one dedicated SelectorManager thread at construction time that runs continuously for the life of the client and never competes with request-processing work for a thread-pool slot.

Under high-concurrency load (60 VUs, small/frequent requests), this appears to add a measurable, consistent latency overhead compared to HttpJdkSolrClient — real A/B benchmark, same workload, same cluster, 5 reps: p99 latency increased 61-119% per rep (overall +68.8%, CI [35.6%, 99.2%]) when switching from HttpJdkSolrClient to HttpJettySolrClient, with no other configuration changes. This is plausibly explained by the extra "produce → wait for a free executor thread to pick up the select task" dispatch hop occurring on every I/O-readiness cycle, which the JDK client's dedicated-thread design avoids entirely.

There is currently no way to configure this: the ClientConnector/selector construction happens inside HttpJettySolrClient.createHttpClient(), which is private, and HttpJettySolrClient.Builder exposes no method to control selector count, execution strategy, or whether selection runs on a dedicated thread vs. the shared executor.

Ask: expose a Builder option to either (a) increase the selector count independently of useHttp1_1, or (b) run selection on a dedicated thread rather than the shared client executor — similar in spirit to the existing useHttp1_1/withMaxConnectionsPerHost builder options.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org