You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by logo <lo...@kreuser.name> on 2019/12/14 17:29:35 UTC

JMX reloadSslHostConfigs fails with javax.management.RuntimeOperationsException

Hi all,

I have a problem with the reloading of my ssl configs after an update of certs from LetsEncrypt or my internal CA.

Tomcat is 8.5.50, JDK is 11.0.5+10-post-Debian-2

I have used basically Christopher Schultz’s tool from https://github.com/ChristopherSchultz/apache-tomcat-stuff/tree/master/bin.


▶ curl -u user:pass "https://host:8443/manager/jmxproxy?invoke=Catalina:type=ProtocolHandler,port=8443&op=reloadSslHostConfigs"
Error - javax.management.RuntimeOperationsException: Exception invoking method reloadSslHostConfigs
javax.management.RuntimeOperationsException: Exception invoking method reloadSslHostConfigs
	at org.apache.tomcat.util.modeler.BaseModelMBean.invoke(BaseModelMBean.java:295)
	at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:809)
	at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
	at org.apache.catalina.manager.JMXProxyServlet.invokeOperationInternal(JMXProxyServlet.java:273)
	at org.apache.catalina.manager.JMXProxyServlet.invokeOperation(JMXProxyServlet.java:207)
	at org.apache.catalina.manager.JMXProxyServlet.doGet(JMXProxyServlet.java:116)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:109)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:668)
	at org.apache.catalina.valves.RequestFilterValve.process(RequestFilterValve.java:348)
	at org.apache.catalina.valves.RemoteAddrValve.invoke(RemoteAddrValve.java:52)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:678)
	at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
	at org.apache.coyote.http2.StreamProcessor.service(StreamProcessor.java:324)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
	at org.apache.coyote.http2.StreamProcessor.process(StreamProcessor.java:69)
	at org.apache.coyote.http2.StreamRunnable.run(StreamRunnable.java:35)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.IllegalArgumentException: java.lang.IllegalArgumentException
	at org.apache.tomcat.util.net.AbstractEndpoint.addSslHostConfig(AbstractEndpoint.java:252)
	at org.apache.tomcat.util.net.AbstractEndpoint.reloadSslHostConfig(AbstractEndpoint.java:311)
	at org.apache.tomcat.util.net.AbstractEndpoint.reloadSslHostConfigs(AbstractEndpoint.java:320)
	at org.apache.coyote.http11.AbstractHttp11Protocol.reloadSslHostConfigs(AbstractHttp11Protocol.java:517)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.apache.tomcat.util.modeler.BaseModelMBean.invoke(BaseModelMBean.java:287)
	... 38 more
Caused by: java.lang.IllegalArgumentException
	at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:100)
	at org.apache.tomcat.util.net.AbstractEndpoint.addSslHostConfig(AbstractEndpoint.java:250)
	... 46 more
Caused by: java.io.IOException
	at org.apache.tomcat.util.net.SSLUtilBase.getKeyManagers(SSLUtilBase.java:301)
	at org.apache.tomcat.util.net.SSLUtilBase.createSSLContext(SSLUtilBase.java:246)
	at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:98)
	... 47 more


It’s working after a restart of tomcat, then it may be successful a couple of times, but then something is happening (Access of the website?) and boom the exception occurs.
I am able to force this error, if I call a imx query with curl first.

curl -u user:pass "https://host:8443/manager/jmxproxy?qry=Catalina:type=ProtocolHandler,port=8443“                           


I have tried to debug the problem and tracked it down to an empty KeyStore on the „defaultSSLHostConfigName" due to an empty certificateKeystore and with a default non-existing certificateKeystoreFile "/home/tomcat/.keystore“ and an null certificateFile . I see some of my configured attributes - but not the cert file. From there on I need help. I have no clue where to look next. Maybe Mark or Chris may lead me to the spot where all this is reset. Chris, is your script working on 8.5?

Code is: AbstractEndpoint.java:252

    public void reloadSslHostConfig(String hostName) {
        SSLHostConfig sslHostConfig = sslHostConfigs.get(hostName);
        if (sslHostConfig == null) { 
            throw new IllegalArgumentException(
                    sm.getString("endpoint.unknownSslHostName", hostName));
        }
        addSslHostConfig(sslHostConfig, true); // << sslHostConfig has no Cert anymore, yet it is not null
    }


And in SSLUtilBase.java:301

        KeyStore ks = certificate.getCertificateKeystore(); // << none of the possible locations are filled anymore
...
        if (ks == null) {
            if (certificate.getCertificateFile() == null) {
                throw new IOException(sm.getString("jsse.noCertFile"));
            }
        ...


The site is still accessible in the browser even though the JMX requests fail????

My SSL config is:

 <Connector port="8443"
           protocol="org.apache.coyote.http11.Http11Nio2Protocol"
           sslImplementationName="org.apache.tomcat.util.net.jsse.JSSEImplementation"
           allowTrace="false"
           maxThreads="150"
           SSLEnabled="true"
           compression="off"
           scheme="https"
           server="Apache Tomcat"
           secure="true"
           defaultSSLHostConfigName="host" >
     <UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" compression="on" />

    <SSLHostConfig
            hostName="host"
            honorCipherOrder="true"
            certificateVerification="none"
            certificateRevocationListFile="${catalina.base}/conf/ssl/ca-bundle-client.crl"
            truststoreFile="${catalina.base}/conf/ssl/cacerts.jks"
            truststorePassword="changeit"
            protocols="TLSv1.2+TLSv1.3"
            ciphers="HIGH:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:!DSS">
     <Certificate certificateKeystoreFile="${catalina.base}/conf/ssl/docker.p12"
                  certificateKeystorePassword="changeit"
                  certificateKeyAlias="docker"
                  type="RSA" />
    </SSLHostConfig>
…

I have a couple more of them (self signed CA and LetsEncrypt). But in the reloadSslHostConfigs they are correctly initialized.

Any idea? Maybe it’s just a misunderstanding? Or a bug?

Peter


RE: UPDATED: JMX reloadSslHostConfigs fails with javax.management.RuntimeOperationsException

Posted by jo...@wellsfargo.com.INVALID.
Thank You!


Dream * Excel * Explore * Inspire
Jon McAlexander
Asst Vice President

Middleware Product Engineering
Enterprise CIO | Platform Services | Middleware | Infrastructure Solutions

Upcoming PTO: 11/8, 11/11, 11/15, 11/22, 11/28, 11/29, 12/2, 12/6, 12/13, 12/20 – 12/31

8080 Cobblestone Rd | Urbandale, IA 50322
MAC: F4469-010
Tel 515-988-2508 | Cell 515-988-2508

jonmcalexander@wellsfargo.com


This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation.


-----Original Message-----
From: Mark Thomas <ma...@apache.org> 
Sent: Monday, December 16, 2019 10:41 AM
To: users@tomcat.apache.org
Subject: Re: UPDATED: JMX reloadSslHostConfigs fails with javax.management.RuntimeOperationsException

On 16/12/2019 16:24, jonmcalexander@wellsfargo.com.INVALID wrote:

<snip/>

> Hi Mark,
> Is there an ETA for 9.0.31 and 8.5.51 beings that .30 and .50 just dropped?

9.0.x and 8.5.x releases currently happen on a roughly monthly schedule.
On that basis, the next release round will be early January 2020.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: UPDATED: JMX reloadSslHostConfigs fails with javax.management.RuntimeOperationsException

Posted by Mark Thomas <ma...@apache.org>.
On 16/12/2019 16:24, jonmcalexander@wellsfargo.com.INVALID wrote:

<snip/>

> Hi Mark,
> Is there an ETA for 9.0.31 and 8.5.51 beings that .30 and .50 just dropped?

9.0.x and 8.5.x releases currently happen on a roughly monthly schedule.
On that basis, the next release round will be early January 2020.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


RE: UPDATED: JMX reloadSslHostConfigs fails with javax.management.RuntimeOperationsException

Posted by jo...@wellsfargo.com.INVALID.
-----Original Message-----
From: Mark Thomas <ma...@apache.org> 
Sent: Monday, December 16, 2019 9:05 AM
To: users@tomcat.apache.org
Subject: Re: UPDATED: JMX reloadSslHostConfigs fails with javax.management.RuntimeOperationsException

On 16/12/2019 12:55, Mark Thomas wrote:
> On 15/12/2019 09:33, logo wrote:
> 
> <snip/>
>>
>> Mark can you confirm that this is a bug?
> 
> Confirmed.
> 
> I'm looking at options to fix this now.

> Fixed in:
> - master for 9.0.31 onwards
> - 8.5.x for 8.5.51 onwards

> Thanks for reporting this and for the analysis.

> Mark

Hi Mark,
Is there an ETA for 9.0.31 and 8.5.51 beings that .30 and .50 just dropped?

Thanks,
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: UPDATED: JMX reloadSslHostConfigs fails with javax.management.RuntimeOperationsException

Posted by Mark Thomas <ma...@apache.org>.
On 16/12/2019 12:55, Mark Thomas wrote:
> On 15/12/2019 09:33, logo wrote:
> 
> <snip/>
>>
>> Mark can you confirm that this is a bug?
> 
> Confirmed.
> 
> I'm looking at options to fix this now.

Fixed in:
- master for 9.0.31 onwards
- 8.5.x for 8.5.51 onwards

Thanks for reporting this and for the analysis.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: UPDATED: JMX reloadSslHostConfigs fails with javax.management.RuntimeOperationsException

Posted by Mark Thomas <ma...@apache.org>.
On 15/12/2019 09:33, logo wrote:

<snip/>
> 
> Mark can you confirm that this is a bug?

Confirmed.

I'm looking at options to fix this now.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


UPDATED: JMX reloadSslHostConfigs fails with javax.management.RuntimeOperationsException

Posted by logo <lo...@kreuser.name>.
Hi all,

More analysis and now getting to a weird point:

Chain of events:
———————————

Start tomcat with SSL and JMX enabled…

Reload SSL config with JMX (this time for only one host)

> curl -u user:pass "https://docker2:8443/manager/jmxproxy?invoke=Catalina:type=ProtocolHandler,port=8443&op=reloadSslHostConfig&ps=docker2“

OK - Operation reloadSslHostConfig without return value

Now query the same certificate with JMX

> curl -u user:pass  'https://docker2:8443/manager/jmxproxy?qry=Catalina:type=SSLHostConfig,ThreadPool="https-jsse-nio2-8443",name="docker2"'

OK - Number of results: 1
…

I see that an *empty* Default certificate is added to SSLHostConfigs at SSLHostConfig.java:217

    private void registerDefaultCertificate() {
        if (defaultCertificate == null) {     // <<< ???
            defaultCertificate = new SSLHostConfigCertificate(
                    this, SSLHostConfigCertificate.Type.UNDEFINED);  // << MEEK
            certificates.add(defaultCertificate);
        }
    }

WAIT: it’s called from here:

    // ------------------------------- OpenSSL specific configuration properties

    // TODO: These certificate setters can be removed once it is no longer
    // necessary to support the old configuration attributes (Tomcat 10?).

    public String getCertificateChainFile() {
        registerDefaultCertificate();
        return defaultCertificate.getCertificateChainFile();
    }


And the callstack:

SSLHostConfig.registerDefaultCertificate() line: 217	
SSLHostConfig.getCertificateChainFile() line: 733	
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]	
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62	
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43	
Method.invoke(Object, Object...) line: 566	
BaseModelMBean.getAttribute(String) line: 176	
DefaultMBeanServerInterceptor.getAttribute(ObjectName, String) line: 641	
JmxMBeanServer.getAttribute(ObjectName, String) line: 678	


Now Reload SSL config with JMX

> curl -u user:pass "https://docker2:8443/manager/jmxproxy?invoke=Catalina:type=ProtocolHandler,port=8443&op=reloadSslHostConfig&ps=docker2“

javax.management.RuntimeOperationsException…


Apparently there are two certs in the config now for this host. The *empty* Default certificate is loaded and SSLUtilBase.java:301 fails as there is no certificate file in the Keystore.


Mark can you confirm that this is a bug?

Thanks and have a nice Sunday

Peter



> Am 14.12.2019 um 18:29 schrieb logo <lo...@kreuser.name>:
> 
> Hi all,
> 
> I have a problem with the reloading of my ssl configs after an update of certs from LetsEncrypt or my internal CA.
> 
> Tomcat is 8.5.50, JDK is 11.0.5+10-post-Debian-2
> 
> I have used basically Christopher Schultz’s tool from https://github.com/ChristopherSchultz/apache-tomcat-stuff/tree/master/bin <https://github.com/ChristopherSchultz/apache-tomcat-stuff/tree/master/bin>.
> 
> 
> ▶ curl -u user:pass "https://host:8443/manager/jmxproxy?invoke=Catalina:type=ProtocolHandler,port=8443&op=reloadSslHostConfigs <https://host:8443/manager/jmxproxy?invoke=Catalina:type=ProtocolHandler,port=8443&op=reloadSslHostConfigs>"
> Error - javax.management.RuntimeOperationsException: Exception invoking method reloadSslHostConfigs
> javax.management.RuntimeOperationsException: Exception invoking method reloadSslHostConfigs
> 	at org.apache.tomcat.util.modeler.BaseModelMBean.invoke(BaseModelMBean.java:295)
> 	at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:809)
> 	at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> 	at org.apache.catalina.manager.JMXProxyServlet.invokeOperationInternal(JMXProxyServlet.java:273)
> 	at org.apache.catalina.manager.JMXProxyServlet.invokeOperation(JMXProxyServlet.java:207)
> 	at org.apache.catalina.manager.JMXProxyServlet.doGet(JMXProxyServlet.java:116)
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
> 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
> 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> 	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> 	at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126)
> 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> 	at org.apache.catalina.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:109)
> 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> 	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
> 	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
> 	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:668)
> 	at org.apache.catalina.valves.RequestFilterValve.process(RequestFilterValve.java:348)
> 	at org.apache.catalina.valves.RemoteAddrValve.invoke(RemoteAddrValve.java:52)
> 	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
> 	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
> 	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
> 	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:678)
> 	at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747)
> 	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
> 	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
> 	at org.apache.coyote.http2.StreamProcessor.service(StreamProcessor.java:324)
> 	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
> 	at org.apache.coyote.http2.StreamProcessor.process(StreamProcessor.java:69)
> 	at org.apache.coyote.http2.StreamRunnable.run(StreamRunnable.java:35)
> 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> 	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> 	at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.lang.IllegalArgumentException: java.lang.IllegalArgumentException
> 	at org.apache.tomcat.util.net.AbstractEndpoint.addSslHostConfig(AbstractEndpoint.java:252)
> 	at org.apache.tomcat.util.net.AbstractEndpoint.reloadSslHostConfig(AbstractEndpoint.java:311)
> 	at org.apache.tomcat.util.net.AbstractEndpoint.reloadSslHostConfigs(AbstractEndpoint.java:320)
> 	at org.apache.coyote.http11.AbstractHttp11Protocol.reloadSslHostConfigs(AbstractHttp11Protocol.java:517)
> 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> 	at org.apache.tomcat.util.modeler.BaseModelMBean.invoke(BaseModelMBean.java:287)
> 	... 38 more
> Caused by: java.lang.IllegalArgumentException
> 	at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:100)
> 	at org.apache.tomcat.util.net.AbstractEndpoint.addSslHostConfig(AbstractEndpoint.java:250)
> 	... 46 more
> Caused by: java.io.IOException
> 	at org.apache.tomcat.util.net.SSLUtilBase.getKeyManagers(SSLUtilBase.java:301)
> 	at org.apache.tomcat.util.net.SSLUtilBase.createSSLContext(SSLUtilBase.java:246)
> 	at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:98)
> 	... 47 more
> 
> 
> It’s working after a restart of tomcat, then it may be successful a couple of times, but then something is happening (Access of the website?) and boom the exception occurs.
> I am able to force this error, if I call a imx query with curl first.
> 
> curl -u user:pass "https://host:8443/manager/jmxproxy?qry=Catalina:type=ProtocolHandler,port=8443 <https://host:8443/manager/jmxproxy?qry=Catalina:type=ProtocolHandler,port=8443>“                           
> 
> 
> I have tried to debug the problem and tracked it down to an empty KeyStore on the „defaultSSLHostConfigName" due to an empty certificateKeystore and with a default non-existing certificateKeystoreFile "/home/tomcat/.keystore“ and an null certificateFile . I see some of my configured attributes - but not the cert file. From there on I need help. I have no clue where to look next. Maybe Mark or Chris may lead me to the spot where all this is reset. Chris, is your script working on 8.5?
> 
> Code is: AbstractEndpoint.java:252
> 
>     public void reloadSslHostConfig(String hostName) {
>         SSLHostConfig sslHostConfig = sslHostConfigs.get(hostName);
>         if (sslHostConfig == null) { 
>             throw new IllegalArgumentException(
>                     sm.getString("endpoint.unknownSslHostName", hostName));
>         }
>         addSslHostConfig(sslHostConfig, true); // << sslHostConfig has no Cert anymore, yet it is not null
>     }
> 
> 
> And in SSLUtilBase.java:301
> 
>         KeyStore ks = certificate.getCertificateKeystore(); // << none of the possible locations are filled anymore
> ...
>         if (ks == null) {
>             if (certificate.getCertificateFile() == null) {
>                 throw new IOException(sm.getString("jsse.noCertFile"));
>             }
>         ...
> 
> 
> The site is still accessible in the browser even though the JMX requests fail????
> 
> My SSL config is:
> 
>  <Connector port="8443"
>            protocol="org.apache.coyote.http11.Http11Nio2Protocol"
>            sslImplementationName="org.apache.tomcat.util.net.jsse.JSSEImplementation"
>            allowTrace="false"
>            maxThreads="150"
>            SSLEnabled="true"
>            compression="off"
>            scheme="https"
>            server="Apache Tomcat"
>            secure="true"
>            defaultSSLHostConfigName="host" >
>      <UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" compression="on" />
> 
>     <SSLHostConfig
>             hostName="host"
>             honorCipherOrder="true"
>             certificateVerification="none"
>             certificateRevocationListFile="${catalina.base}/conf/ssl/ca-bundle-client.crl"
>             truststoreFile="${catalina.base}/conf/ssl/cacerts.jks"
>             truststorePassword="changeit"
>             protocols="TLSv1.2+TLSv1.3"
>             ciphers="HIGH:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:!DSS">
>      <Certificate certificateKeystoreFile="${catalina.base}/conf/ssl/docker.p12"
>                   certificateKeystorePassword="changeit"
>                   certificateKeyAlias="docker"
>                   type="RSA" />
>     </SSLHostConfig>
> …
> 
> I have a couple more of them (self signed CA and LetsEncrypt). But in the reloadSslHostConfigs they are correctly initialized.
> 
> Any idea? Maybe it’s just a misunderstanding? Or a bug?
> 
> Peter
>