You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Marcos Peña <m3...@qad.com.INVALID> on 2024/04/11 12:28:01 UTC
Tomcat 9.0.83 - SSL handshake stops working for Google API calls after a while
Hi,
I am looking for help with a strange issue we are experiencing when trying
to use Google APIs from a web application that is deployed on Tomcat 9.0.83.
After a few hours of the server being up and running, all calls to the
Google APIs fail because of SSL handshake errors. Attaching the SSL logs
for your reference.
I see some differences in the ClientHello message. When the handshake
fails, all TLSv1.3 ciphers are ignored, there is no "session id" and
TLSv1.2 is sent as the only supported version.
The Tomcat connector configuration is as follows:
<Connector port="8443"
protocol="com.precisionsoftware.tomcat.Http11Nio2Protocol" proxyPort="443"
SSLEnabled="true"
connectionTimeout="60000"
maxThreads="300"
minSpareThreads="50"
acceptCount="250"
maxKeepAliveRequests="1"
maxPostSize="-1"
relaxedQueryChars='[]|{}^\`"<>'
enableLookups="true"
disableUploadTimeout="true"
URIEncoding="UTF-8"
compression="force"
scheme="https"
secure="true"
clientAuth="false"
sslProtocol="TLS"
sslEnabledProtocols="TLSv1.2+TLSv1.3"
keyAlias="1"
keystoreFile="../wildcard_odqad.pfx"
keystorePass="thepassword"
ciphers="TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256,TLS_AES_128_CCM_SHA256,TLS_AES_128_CCM_8_SHA256"/>
I updated Tomcat to use the most recent native library - 2.0.7 - but that
did not help. Below an extract from the server log.
2024-04-11 02:12:47,507 INFO
[org.apache.catalina.core.AprLifecycleListener:134] (main) Loaded Apache
Tomcat Native library [2.0.7] using APR version [1.7.4].
2024-04-11 02:12:47,507 INFO
[org.apache.catalina.core.AprLifecycleListener:134] (main) APR
capabilities: IPv6 [true], sendfile [true], accept filters [false], random
[true], UDS [true].
2024-04-11 02:12:47,507 INFO
[org.apache.catalina.core.AprLifecycleListener:134] (main) APR/OpenSSL
configuration: useAprConnector [false], useOpenSSL [true]
2024-04-11 02:12:47,514 INFO
[org.apache.catalina.core.AprLifecycleListener:370] (main) OpenSSL
successfully initialized [OpenSSL 3.0.13 30 Jan 2024]
I am not very familiar with the SSL handshake process and do not really
understand what can make it stop working.
Thanks,
Marcos
Re: [OT] Tomcat 9.0.83 - SSL handshake stops working for Google API calls after a while
Posted by Chuck Caldarale <n8...@gmail.com>.
> On Apr 11, 2024, at 07:28, Marcos Peña <m3...@qad.com.INVALID> wrote:
>
> I am looking for help with a strange issue we are experiencing when trying to use Google APIs from a web application that is deployed on Tomcat 9.0.83.
As Chris noted, this has nothing to do with Tomcat. The stack trace shows that the failure is coming from your application code:
com.precisionsoftware.trax.service.translation.Transliterator
> After a few hours of the server being up and running, all calls to the Google APIs fail because of SSL handshake errors.
My quite limited experience with Google APIs (using rclone) shows that the client access token has to be renewed fairly frequently (rclone does this automatically). I don’t know if the token is used as part of the TLS handshake, nor if your code is refreshing the token when needed.
> The Tomcat connector configuration is as follows:
> <Connector port="8443" protocol="com.precisionsoftware.tomcat.Http11Nio2Protocol" proxyPort="443" SSLEnabled="true"
Unrelated to your problem, but that protocol class setting seems very odd; have you copied the Tomcat class into your own package?
- Chuck
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Tomcat 9.0.83 - SSL handshake stops working for Google API calls after a while
Posted by Christopher Schultz <ch...@christopherschultz.net>.
Marcos,
On 4/11/24 09:52, Marcos Peña wrote:
> Thanks for your replies.
>
> My bad assuming the connector configuration applied to all connections but
> it makes total sense that applies to incoming connections. That helps a lot.
>
> I have been trying to solve this problem for several days and I was a bit
> desperate. I could not find anything in the application code that would
> change it at runtime...up to a few minutes ago. I think there is a
> SSLSocketFactory that is changing the ciphers at runtime.
That's exactly the kind of thing I would expect to discover given your
description of the symptoms.
-chris
> On Thu, Apr 11, 2024 at 2:25 PM Christopher Schultz <
> chris@christopherschultz.net> wrote:
>
>> Marcos,
>>
>> Marking as "off topic" because this is not Tomcat-related. Please see
>> below...
>>
>> On 4/11/24 08:28, Marcos Peña wrote:
>>> Hi,
>>>
>>> I am looking for help with a strange issue we are experiencing when
>>> trying to use Google APIs from a web application that is deployed on
>>> Tomcat 9.0.83.
>>>
>>> After a few hours of the server being up and running, all calls to the
>>> Google APIs fail because of SSL handshake errors. Attaching the SSL logs
>>> for your reference.
>>>
>>> I see some differences in the ClientHello message. When the handshake
>>> fails, all TLSv1.3 ciphers are ignored, there is no "session id" and
>>> TLSv1.2 is sent as the only supported version.
>>>
>>> The Tomcat connector configuration is as follows:
>>> <Connector port="8443"
>>> protocol="com.precisionsoftware.tomcat.Http11Nio2Protocol"
>>> proxyPort="443" SSLEnabled="true"
>>> connectionTimeout="60000"
>>> maxThreads="300"
>>> minSpareThreads="50"
>>> acceptCount="250"
>>> maxKeepAliveRequests="1"
>>> maxPostSize="-1"
>>> relaxedQueryChars='[]|{}^\`"<>'
>>> enableLookups="true"
>>> disableUploadTimeout="true"
>>> URIEncoding="UTF-8"
>>> compression="force"
>>> scheme="https"
>>> secure="true"
>>> clientAuth="false"
>>> sslProtocol="TLS"
>>> sslEnabledProtocols="TLSv1.2+TLSv1.3"
>>> keyAlias="1"
>>> keystoreFile="../wildcard_odqad.pfx"
>>> keystorePass="thepassword"
>>>
>>>
>> ciphers="TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256,TLS_AES_128_CCM_SHA256,TLS_AES_128_CCM_8_SHA256"/>
>>
>> This configuration is only relevant for INCOMING connections. It has no
>> bearing on your outgoing HTTP / TLS connections.
>>
>>> I updated Tomcat to use the most recent native library - 2.0.7 - but
>>> that did not help. Below an extract from the server log.
>>
>> Your use (or not) of tcnative is only relevant for incoming connections.
>> Without significant work, your outgoing connections will not be using
>> tcnative for any TLS communications.
>>
>>> 2024-04-11 02:12:47,507 INFO
>>> [org.apache.catalina.core.AprLifecycleListener:134] (main) Loaded
>>> Apache Tomcat Native library [2.0.7] using APR version [1.7.4].
>>> 2024-04-11 02:12:47,507 INFO
>>> [org.apache.catalina.core.AprLifecycleListener:134] (main) APR
>>> capabilities: IPv6 [true], sendfile [true], accept filters [false],
>>> random [true], UDS [true].
>>> 2024-04-11 02:12:47,507 INFO
>>> [org.apache.catalina.core.AprLifecycleListener:134] (main) APR/OpenSSL
>>> configuration: useAprConnector [false], useOpenSSL [true]
>>> 2024-04-11 02:12:47,514 INFO
>>> [org.apache.catalina.core.AprLifecycleListener:370] (main) OpenSSL
>>> successfully initialized [OpenSSL 3.0.13 30 Jan 2024]
>>>
>>> I am not very familiar with the SSL handshake process and do not really
>>> understand what can make it stop working.
>>
>> My guess is that your /outbound/ HTTP connection manager is having its
>> configuration changed at runtime. You will have to look at how your
>> application establishes these outbound connections to determine why the
>> rules are changing.
>>
>> -chris
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: [OT] Tomcat 9.0.83 - SSL handshake stops working for Google API calls after a while
Posted by Marcos Peña <m3...@qad.com.INVALID>.
Thanks for your replies.
My bad assuming the connector configuration applied to all connections but
it makes total sense that applies to incoming connections. That helps a lot.
I have been trying to solve this problem for several days and I was a bit
desperate. I could not find anything in the application code that would
change it at runtime...up to a few minutes ago. I think there is a
SSLSocketFactory that is changing the ciphers at runtime.
Regards,
Marcos
On Thu, Apr 11, 2024 at 2:25 PM Christopher Schultz <
chris@christopherschultz.net> wrote:
> Marcos,
>
> Marking as "off topic" because this is not Tomcat-related. Please see
> below...
>
> On 4/11/24 08:28, Marcos Peña wrote:
> > Hi,
> >
> > I am looking for help with a strange issue we are experiencing when
> > trying to use Google APIs from a web application that is deployed on
> > Tomcat 9.0.83.
> >
> > After a few hours of the server being up and running, all calls to the
> > Google APIs fail because of SSL handshake errors. Attaching the SSL logs
> > for your reference.
> >
> > I see some differences in the ClientHello message. When the handshake
> > fails, all TLSv1.3 ciphers are ignored, there is no "session id" and
> > TLSv1.2 is sent as the only supported version.
> >
> > The Tomcat connector configuration is as follows:
> > <Connector port="8443"
> > protocol="com.precisionsoftware.tomcat.Http11Nio2Protocol"
> > proxyPort="443" SSLEnabled="true"
> > connectionTimeout="60000"
> > maxThreads="300"
> > minSpareThreads="50"
> > acceptCount="250"
> > maxKeepAliveRequests="1"
> > maxPostSize="-1"
> > relaxedQueryChars='[]|{}^\`"<>'
> > enableLookups="true"
> > disableUploadTimeout="true"
> > URIEncoding="UTF-8"
> > compression="force"
> > scheme="https"
> > secure="true"
> > clientAuth="false"
> > sslProtocol="TLS"
> > sslEnabledProtocols="TLSv1.2+TLSv1.3"
> > keyAlias="1"
> > keystoreFile="../wildcard_odqad.pfx"
> > keystorePass="thepassword"
> >
> >
> ciphers="TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256,TLS_AES_128_CCM_SHA256,TLS_AES_128_CCM_8_SHA256"/>
>
> This configuration is only relevant for INCOMING connections. It has no
> bearing on your outgoing HTTP / TLS connections.
>
> > I updated Tomcat to use the most recent native library - 2.0.7 - but
> > that did not help. Below an extract from the server log.
>
> Your use (or not) of tcnative is only relevant for incoming connections.
> Without significant work, your outgoing connections will not be using
> tcnative for any TLS communications.
>
> > 2024-04-11 02:12:47,507 INFO
> > [org.apache.catalina.core.AprLifecycleListener:134] (main) Loaded
> > Apache Tomcat Native library [2.0.7] using APR version [1.7.4].
> > 2024-04-11 02:12:47,507 INFO
> > [org.apache.catalina.core.AprLifecycleListener:134] (main) APR
> > capabilities: IPv6 [true], sendfile [true], accept filters [false],
> > random [true], UDS [true].
> > 2024-04-11 02:12:47,507 INFO
> > [org.apache.catalina.core.AprLifecycleListener:134] (main) APR/OpenSSL
> > configuration: useAprConnector [false], useOpenSSL [true]
> > 2024-04-11 02:12:47,514 INFO
> > [org.apache.catalina.core.AprLifecycleListener:370] (main) OpenSSL
> > successfully initialized [OpenSSL 3.0.13 30 Jan 2024]
> >
> > I am not very familiar with the SSL handshake process and do not really
> > understand what can make it stop working.
>
> My guess is that your /outbound/ HTTP connection manager is having its
> configuration changed at runtime. You will have to look at how your
> application establishes these outbound connections to determine why the
> rules are changing.
>
> -chris
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: [OT] Tomcat 9.0.83 - SSL handshake stops working for Google API calls after a while
Posted by Christopher Schultz <ch...@christopherschultz.net>.
Marcos,
Marking as "off topic" because this is not Tomcat-related. Please see
below...
On 4/11/24 08:28, Marcos Peña wrote:
> Hi,
>
> I am looking for help with a strange issue we are experiencing when
> trying to use Google APIs from a web application that is deployed on
> Tomcat 9.0.83.
>
> After a few hours of the server being up and running, all calls to the
> Google APIs fail because of SSL handshake errors. Attaching the SSL logs
> for your reference.
>
> I see some differences in the ClientHello message. When the handshake
> fails, all TLSv1.3 ciphers are ignored, there is no "session id" and
> TLSv1.2 is sent as the only supported version.
>
> The Tomcat connector configuration is as follows:
> <Connector port="8443"
> protocol="com.precisionsoftware.tomcat.Http11Nio2Protocol"
> proxyPort="443" SSLEnabled="true"
> connectionTimeout="60000"
> maxThreads="300"
> minSpareThreads="50"
> acceptCount="250"
> maxKeepAliveRequests="1"
> maxPostSize="-1"
> relaxedQueryChars='[]|{}^\`"<>'
> enableLookups="true"
> disableUploadTimeout="true"
> URIEncoding="UTF-8"
> compression="force"
> scheme="https"
> secure="true"
> clientAuth="false"
> sslProtocol="TLS"
> sslEnabledProtocols="TLSv1.2+TLSv1.3"
> keyAlias="1"
> keystoreFile="../wildcard_odqad.pfx"
> keystorePass="thepassword"
>
> ciphers="TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256,TLS_AES_128_CCM_SHA256,TLS_AES_128_CCM_8_SHA256"/>
This configuration is only relevant for INCOMING connections. It has no
bearing on your outgoing HTTP / TLS connections.
> I updated Tomcat to use the most recent native library - 2.0.7 - but
> that did not help. Below an extract from the server log.
Your use (or not) of tcnative is only relevant for incoming connections.
Without significant work, your outgoing connections will not be using
tcnative for any TLS communications.
> 2024-04-11 02:12:47,507 INFO
> [org.apache.catalina.core.AprLifecycleListener:134] (main) Loaded
> Apache Tomcat Native library [2.0.7] using APR version [1.7.4].
> 2024-04-11 02:12:47,507 INFO
> [org.apache.catalina.core.AprLifecycleListener:134] (main) APR
> capabilities: IPv6 [true], sendfile [true], accept filters [false],
> random [true], UDS [true].
> 2024-04-11 02:12:47,507 INFO
> [org.apache.catalina.core.AprLifecycleListener:134] (main) APR/OpenSSL
> configuration: useAprConnector [false], useOpenSSL [true]
> 2024-04-11 02:12:47,514 INFO
> [org.apache.catalina.core.AprLifecycleListener:370] (main) OpenSSL
> successfully initialized [OpenSSL 3.0.13 30 Jan 2024]
>
> I am not very familiar with the SSL handshake process and do not really
> understand what can make it stop working.
My guess is that your /outbound/ HTTP connection manager is having its
configuration changed at runtime. You will have to look at how your
application establishes these outbound connections to determine why the
rules are changing.
-chris
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 9.0.83 - SSL handshake stops working for Google API calls after a while
Posted by Simon Matter <si...@invoca.ch>.
Hi,
> Hi,
>
> I am looking for help with a strange issue we are experiencing when trying
> to use Google APIs from a web application that is deployed on Tomcat
> 9.0.83.
>
> After a few hours of the server being up and running, all calls to the
> Google APIs fail because of SSL handshake errors. Attaching the SSL logs
> for your reference.
Without knowing exactly how it would look like, are you 100% sure you're
not running out of entropy for some reason?
At least it doesn't hurt to have available entropy in monitoring some how.
Regards,
Simon
>
> I see some differences in the ClientHello message. When the handshake
> fails, all TLSv1.3 ciphers are ignored, there is no "session id" and
> TLSv1.2 is sent as the only supported version.
>
> The Tomcat connector configuration is as follows:
> <Connector port="8443"
> protocol="com.precisionsoftware.tomcat.Http11Nio2Protocol" proxyPort="443"
> SSLEnabled="true"
> connectionTimeout="60000"
> maxThreads="300"
> minSpareThreads="50"
> acceptCount="250"
> maxKeepAliveRequests="1"
> maxPostSize="-1"
> relaxedQueryChars='[]|{}^\`"<>'
> enableLookups="true"
> disableUploadTimeout="true"
> URIEncoding="UTF-8"
> compression="force"
> scheme="https"
> secure="true"
> clientAuth="false"
> sslProtocol="TLS"
> sslEnabledProtocols="TLSv1.2+TLSv1.3"
> keyAlias="1"
> keystoreFile="../wildcard_odqad.pfx"
> keystorePass="thepassword"
>
> ciphers="TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256,TLS_AES_128_CCM_SHA256,TLS_AES_128_CCM_8_SHA256"/>
>
> I updated Tomcat to use the most recent native library - 2.0.7 - but that
> did not help. Below an extract from the server log.
>
> 2024-04-11 02:12:47,507 INFO
> [org.apache.catalina.core.AprLifecycleListener:134] (main) Loaded Apache
> Tomcat Native library [2.0.7] using APR version [1.7.4].
> 2024-04-11 02:12:47,507 INFO
> [org.apache.catalina.core.AprLifecycleListener:134] (main) APR
> capabilities: IPv6 [true], sendfile [true], accept filters [false], random
> [true], UDS [true].
> 2024-04-11 02:12:47,507 INFO
> [org.apache.catalina.core.AprLifecycleListener:134] (main) APR/OpenSSL
> configuration: useAprConnector [false], useOpenSSL [true]
> 2024-04-11 02:12:47,514 INFO
> [org.apache.catalina.core.AprLifecycleListener:370] (main) OpenSSL
> successfully initialized [OpenSSL 3.0.13 30 Jan 2024]
>
> I am not very familiar with the SSL handshake process and do not really
> understand what can make it stop working.
>
> Thanks,
> Marcos
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org