Response to the first HTTP request in a TCP session takes much longer with Solr 9.x compared to previous versions of Solr

["Litzba, Bernd" <Be...@centric.eu.INVALID>]

Hi all,

I have a question regarding Solr 9.x:

After we updated to Solr 9.x, we noticed that the first get/post request on a new TCP session takes much longer compared to older versions of Solr (we tested Solr 6.x and 8.x) - but only if this first request is sent via a proxy. Without a proxy Solr9.x is as fast as the previous versions.
The logging in Solr does not show anything suspicious. First, we assumed it has something to do with the embedded jetty version (got updated from jetty 9.x to 10.x in Solr 9.2) but Solr 9.0/9.1 also show this unexpected delay.

Example Solr 8.11.4: Response to the first Request on a new TCP session is very fast:
[cid:image001.png@01DB2BB4.DFA80600]

Example Solr 9.0: Response to the first Request on a new TCP session is quite slow (5 seconds):
[cid:image002.png@01DB2BB4.DFA80600]

Does anyone have an idea why there is this different behaviour and where to start searching?
Cheers



Classification: Restricted (V2)

This e-mail is personal. For our full disclaimer, please visit www.centric.eu/disclaimer.

Re: Response to the first HTTP request in a TCP session takes much longer with Solr 9.x compared to previous versions of Solr

[Corrado Fiore <cp...@gmail.com>]

Hi,

we observed something similar a couple of weeks ago.  When requests were 
being originated not from the same private subnet as Solr’s but from a 
Kubernetes pod, latency would spike to ~500 msec instead of the usual 
2-3 msec.  In our case the culprit turned out to be the Java security 
manager sandbox, which is enabled by default:

~~~
# Runs solr in java security manager sandbox. This can protect against 
some attacks.
# Runtime properties are passed to the security policy file 
(server/etc/security.policy)
# You can also tweak via standard JDK files such as ~/.java.policy, see 
https://s.apache.org/java8policy
# This is experimental! It may not work at all with Hadoop/HDFS 
features.
# SOLR_SECURITY_MANAGER_ENABLED=true
~~~

When we set it to false, the extra latency disappeared.

Hope this helps.

Kind regards,
Corrado

On 31 Oct 2024, at 16:49, Litzba, Bernd wrote:

> Hi all,
>
> I have a question regarding Solr 9.x:
>
> After we updated to Solr 9.x, we noticed that the first get/post 
> request on a new TCP session takes much longer compared to older 
> versions of Solr (we tested Solr 6.x and 8.x) - but only if this first 
> request is sent via a proxy. Without a proxy Solr9.x is as fast as the 
> previous versions.