You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by "James H. H. Lampert" <ja...@touchtonecorp.com> on 2020/08/05 23:46:11 UTC
Let's Encrypt cert worked fine in 8.5.57, but connector fails in
8.5.40
Ladies and Gentlemen:
I've now proceeded to the "real" server, with the Tomcat portion of the
procedure refined to give me plenty of "undo" capability. And it turns
out I need it.
It seems that with the unwanted update to 7.0.57 that happened on
launching the test spot instances, the Let's Encrypt certs worked just fine.
But applying the procedure to the *real* development instance (7.0.40)
blew up in my face, failing to open the connectors. Here is an excerpt
from catalina.out, showing the stacktraces.
> 05-Aug-2020 23:00:52.038 WARNING [main] org.apache.catalina.startup.SetAllPropertiesRule.begin [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'bufferSize' to '1024' did not find a matching property.
> 05-Aug-2020 23:00:52.085 WARNING [main] org.apache.catalina.startup.SetAllPropertiesRule.begin [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'bufferSize' to '1024' did not find a matching property.
> 05-Aug-2020 23:00:52.189 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version: Apache Tomcat/8.5.40
> 05-Aug-2020 23:00:52.189 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built: May 2 2019 18:02:51 UTC
> 05-Aug-2020 23:00:52.194 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server number: 8.5.40.0
> 05-Aug-2020 23:00:52.194 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name: Linux
> 05-Aug-2020 23:00:52.194 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version: 4.14.121-85.96.amzn1.x86_64
> 05-Aug-2020 23:00:52.194 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture: amd64
> 05-Aug-2020 23:00:52.195 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-0.43.amzn1.x86_64/jre
> 05-Aug-2020 23:00:52.195 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version: 1.8.0_201-b09
> 05-Aug-2020 23:00:52.195 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor: Oracle Corporation
> 05-Aug-2020 23:00:52.195 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE: /usr/share/tomcat8
> 05-Aug-2020 23:00:52.196 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME: /usr/share/tomcat8
> 05-Aug-2020 23:00:52.196 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/usr/share/tomcat8
> 05-Aug-2020 23:00:52.196 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/usr/share/tomcat8
> 05-Aug-2020 23:00:52.197 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.endorsed.dirs=
> 05-Aug-2020 23:00:52.197 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/var/cache/tomcat8/temp
> 05-Aug-2020 23:00:52.197 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/usr/share/tomcat8/conf/logging.properties
> 05-Aug-2020 23:00:52.197 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> 05-Aug-2020 23:00:52.198 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib]
> 05-Aug-2020 23:00:52.422 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["https-jsse-nio-8443"]
> 05-Aug-2020 23:00:52.848 SEVERE [main] org.apache.catalina.core.StandardService.initInternal Failed to initialize connector [Connector[HTTP/1.1-8443]]
> org.apache.catalina.LifecycleException: Failed to initialize component [Connector[HTTP/1.1-8443]]
> at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:112)
> at org.apache.catalina.core.StandardService.initInternal(StandardService.java:552)
> at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
> at org.apache.catalina.core.StandardServer.initInternal(StandardServer.java:875)
> at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
> at org.apache.catalina.startup.Catalina.load(Catalina.java:639)
> at org.apache.catalina.startup.Catalina.load(Catalina.java:662)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:309)
> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:492)
> Caused by: org.apache.catalina.LifecycleException: Protocol handler initialization failed
> at org.apache.catalina.connector.Connector.initInternal(Connector.java:995)
> at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
> ... 12 more
> Caused by: java.lang.IllegalArgumentException: Cannot store non-PrivateKeys
> at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:100)
> at org.apache.tomcat.util.net.AbstractJsseEndpoint.initialiseSsl(AbstractJsseEndpoint.java:72)
> at org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:244)
> at org.apache.tomcat.util.net.AbstractEndpoint.init(AbstractEndpoint.java:1105)
> at org.apache.tomcat.util.net.AbstractJsseEndpoint.init(AbstractJsseEndpoint.java:224)
> at org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:581)
> at org.apache.coyote.http11.AbstractHttp11Protocol.init(AbstractHttp11Protocol.java:68)
> at org.apache.catalina.connector.Connector.initInternal(Connector.java:993)
> ... 13 more
> Caused by: java.security.KeyStoreException: Cannot store non-PrivateKeys
> at sun.security.provider.JavaKeyStore.engineSetKeyEntry(JavaKeyStore.java:261)
> at sun.security.provider.JavaKeyStore$JKS.engineSetKeyEntry(JavaKeyStore.java:56)
> at sun.security.provider.KeyStoreDelegator.engineSetKeyEntry(KeyStoreDelegator.java:117)
> at sun.security.provider.JavaKeyStore$DualFormatJKS.engineSetKeyEntry(JavaKeyStore.java:70)
> at java.security.KeyStore.setKeyEntry(KeyStore.java:1140)
> at org.apache.tomcat.util.net.SSLUtilBase.getKeyManagers(SSLUtilBase.java:313)
> at org.apache.tomcat.util.net.SSLUtilBase.createSSLContext(SSLUtilBase.java:239)
> at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:98)
> ... 20 more
>
> 05-Aug-2020 23:00:52.857 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["https-jsse-nio-7443"]
> 05-Aug-2020 23:00:52.861 SEVERE [main] org.apache.catalina.core.StandardService.initInternal Failed to initialize connector [Connector[HTTP/1.1-7443]]
> org.apache.catalina.LifecycleException: Failed to initialize component [Connector[HTTP/1.1-7443]]
> at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:112)
> at org.apache.catalina.core.StandardService.initInternal(StandardService.java:552)
> at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
> at org.apache.catalina.core.StandardServer.initInternal(StandardServer.java:875)
> at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
> at org.apache.catalina.startup.Catalina.load(Catalina.java:639)
> at org.apache.catalina.startup.Catalina.load(Catalina.java:662)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:309)
> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:492)
> Caused by: org.apache.catalina.LifecycleException: Protocol handler initialization failed
> at org.apache.catalina.connector.Connector.initInternal(Connector.java:995)
> at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
> ... 12 more
> Caused by: java.lang.IllegalArgumentException: Cannot store non-PrivateKeys
> at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:100)
> at org.apache.tomcat.util.net.AbstractJsseEndpoint.initialiseSsl(AbstractJsseEndpoint.java:72)
> at org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:244)
> at org.apache.tomcat.util.net.AbstractEndpoint.init(AbstractEndpoint.java:1105)
> at org.apache.tomcat.util.net.AbstractJsseEndpoint.init(AbstractJsseEndpoint.java:224)
> at org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:581)
> at org.apache.coyote.http11.AbstractHttp11Protocol.init(AbstractHttp11Protocol.java:68)
> at org.apache.catalina.connector.Connector.initInternal(Connector.java:993)
> ... 13 more
> Caused by: java.security.KeyStoreException: Cannot store non-PrivateKeys
> at sun.security.provider.JavaKeyStore.engineSetKeyEntry(JavaKeyStore.java:261)
> at sun.security.provider.JavaKeyStore$JKS.engineSetKeyEntry(JavaKeyStore.java:56)
> at sun.security.provider.KeyStoreDelegator.engineSetKeyEntry(KeyStoreDelegator.java:117)
> at sun.security.provider.JavaKeyStore$DualFormatJKS.engineSetKeyEntry(JavaKeyStore.java:70)
> at java.security.KeyStore.setKeyEntry(KeyStore.java:1140)
> at org.apache.tomcat.util.net.SSLUtilBase.getKeyManagers(SSLUtilBase.java:313)
> at org.apache.tomcat.util.net.SSLUtilBase.createSSLContext(SSLUtilBase.java:239)
> at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:98)
> ... 20 more
>
I suspect that I will need to bring the Tomcat up to 8.5.57 here, too,
before it will work. Presumably, that means tripping the same process
that made the mess on the test spot instance.
It seems that when the unwanted update happened,
1. /etc/tomcat8/server.xml was left at least relatively undisturbed: our
connectors were undamaged.
2. /etc/tomcat8/tomcat-users.xml was also left undisturbed.
3. /var/lib/tomcat8/webapps/manager/WEB-INF/web.xml was also left at
least relatively undisturbed: because of the size of our WAR files, we
increase the max-file-size and max-request-size from 50MB to 500M, and
that was as we left it.
4. /var/lib/tomcat8/webapps/manager/META-INF/context.xml, however, was
reset to the "factory" state, with the RemoteAddrValve active.
5. The default ROOT context overlaid our ROOT context, leaving
/var/lib/tomcat8/webapps/ROOT filled with both our files and the default
ones. Our ROOT.war, on the other hand, was left intact, and if I stop
Tomcat, remove the ROOT context directory, and then restart Tomcat, it
does unpack our ROOT.war correctly.
6. The /var/lib/tomcat8/webapps/examples context directory, which we
always remove, was reinstalled.
Can anybody make sense of why some things changed, while others were
left alone?
--
JHHL
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Correction, Re: Let's Encrypt cert worked fine in 8.5.57, but
connector fails in 8.5.40
Posted by "James H. H. Lampert" <ja...@touchtonecorp.com>.
I wrote:
. . .
> It seems that with the unwanted update to 7.0.57 that happened on
> launching the test spot instances, the Let's Encrypt certs worked just
> fine.
>
> But applying the procedure to the *real* development instance (7.0.40)
> blew up in my face, failing to open the connectors. Here is an excerpt
> from catalina.out, showing the stacktraces.
. . .
Oops. I'm so used to installing and running 7.0.xx on AS/400s that I
mistyped the above. Of course, I meant 8.5.57 and 8.5.40, respectively,
as given in the subject line and the rest of my post.
--
JHHL
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Let's Encrypt cert worked fine in 8.5.57, but connector fails in
8.5.40
Posted by "James H. H. Lampert" <ja...@touchtonecorp.com>.
On 8/5/20 5:04 PM, calder wrote:
>> Caused by: java.security.KeyStoreException:
>
> Cannot store non-PrivateKeys
>>
> If you pasted the full stack trace, then here we have the last "caused by",
> showing one issue
>> at
>> sun.security.provider.JavaKeyStore.engineSetKeyEntry(JavaKeyStore.java:261)
>>> at
>> sun.security.provider.JavaKeyStore$JKS.engineSetKeyEntry(JavaKeyStore.java:56)
>>> at
>> sun.security.provider.KeyStoreDelegator.engineSetKeyEntry(KeyStoreDelegator.java:117)
>>> at
>> sun.security.provider.JavaKeyStore$DualFormatJKS.engineSetKeyEntry(JavaKeyStore.java:70)
>>> at java.security.KeyStore.setKeyEntry(KeyStore.java:1140)
>>> at org.apache.tomcat.util.net
>> .SSLUtilBase.getKeyManagers(SSLUtilBase.java:313)
>>> at org.apache.tomcat.util.net
>> .SSLUtilBase.createSSLContext(SSLUtilBase.java:239)
>>> at org.apache.tomcat.util.net
>> .AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:98)
>>> ... 20 more
That is inded the full stack trace, or at least as much of it as went
into catalina.out.
And all the differences in domain names and filenames are accounted for,
canceling each other out, leaving only the "unwanted update" as the
biggest single difference between the two instances. Looking at the
Manager, I see that in addition to Tomcat being updated from 8.5.40 to
8.5.57, the JVM was evidently updated from 1.8.0_201-b09 to 1.8.0_252-b09.
--
JHHL
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Let's Encrypt cert worked fine in 8.5.57, but connector fails in 8.5.40
Posted by calder <ca...@gmail.com>.
On Wed, Aug 5, 2020, 18:46 James H. H. Lampert <ja...@touchtonecorp.com>
wrote:
> Ladies and Gentlemen:
>
> I've now proceeded to the "real" server, with the Tomcat portion of the
> procedure refined to give me plenty of "undo" capability. And it turns
> out I need it.
>
> It seems that with the unwanted update to 7.0.57 that happened on
> launching the test spot instances, the Let's Encrypt certs worked just
> fine.
>
> But applying the procedure to the *real* development instance (7.0.40)
> blew up in my face, failing to open the connectors. Here is an excerpt
> from catalina.out, showing the stacktraces.
>
> > 05-Aug-2020 23:00:52.038 WARNING [main]
> org.apache.catalina.startup.SetAllPropertiesRule.begin
> [SetAllPropertiesRule]{Server/Service/Connector} Setting property
> 'bufferSize' to '1024' did not find a matching property.
> > 05-Aug-2020 23:00:52.085 WARNING [main]
> org.apache.catalina.startup.SetAllPropertiesRule.begin
> [SetAllPropertiesRule]{Server/Service/Connector} Setting property
> 'bufferSize' to '1024' did not find a matching property.
> > 05-Aug-2020 23:00:52.189 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log Server version:
> Apache Tomcat/8.5.40
> > 05-Aug-2020 23:00:52.189 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log Server built:
> May 2 2019 18:02:51 UTC
> > 05-Aug-2020 23:00:52.194 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log Server number:
> 8.5.40.0
> > 05-Aug-2020 23:00:52.194 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log OS Name:
> Linux
> > 05-Aug-2020 23:00:52.194 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log OS Version:
> 4.14.121-85.96.amzn1.x86_64
> > 05-Aug-2020 23:00:52.194 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log Architecture:
> amd64
> > 05-Aug-2020 23:00:52.195 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log Java Home:
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-0.43.amzn1.x86_64/jre
> > 05-Aug-2020 23:00:52.195 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log JVM Version:
> 1.8.0_201-b09
> > 05-Aug-2020 23:00:52.195 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor:
> Oracle Corporation
> > 05-Aug-2020 23:00:52.195 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE:
> /usr/share/tomcat8
> > 05-Aug-2020 23:00:52.196 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME:
> /usr/share/tomcat8
> > 05-Aug-2020 23:00:52.196 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log Command line
> argument: -Dcatalina.base=/usr/share/tomcat8
> > 05-Aug-2020 23:00:52.196 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log Command line
> argument: -Dcatalina.home=/usr/share/tomcat8
> > 05-Aug-2020 23:00:52.197 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log Command line
> argument: -Djava.endorsed.dirs=
> > 05-Aug-2020 23:00:52.197 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log Command line
> argument: -Djava.io.tmpdir=/var/cache/tomcat8/temp
> > 05-Aug-2020 23:00:52.197 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log Command line
> argument:
> -Djava.util.logging.config.file=/usr/share/tomcat8/conf/logging.properties
> > 05-Aug-2020 23:00:52.197 INFO [main]
> org.apache.catalina.startup.VersionLoggerListener.log Command line
> argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> > 05-Aug-2020 23:00:52.198 INFO [main]
> org.apache.catalina.core.AprLifecycleListener.lifecycleEvent The APR based
> Apache Tomcat Native library which allows optimal performance in production
> environments was not found on the java.library.path:
> [/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib]
> > 05-Aug-2020 23:00:52.422 INFO [main]
> org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler
> ["https-jsse-nio-8443"]
> > 05-Aug-2020 23:00:52.848 SEVERE [main]
> org.apache.catalina.core.StandardService.initInternal Failed to initialize
> connector [Connector[HTTP/1.1-8443]]
> > org.apache.catalina.LifecycleException: Failed to initialize component
> [Connector[HTTP/1.1-8443]]
> > at
> org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:112)
> > at
> org.apache.catalina.core.StandardService.initInternal(StandardService.java:552)
> > at
> org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
> > at
> org.apache.catalina.core.StandardServer.initInternal(StandardServer.java:875)
> > at
> org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
> > at org.apache.catalina.startup.Catalina.load(Catalina.java:639)
> > at org.apache.catalina.startup.Catalina.load(Catalina.java:662)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:309)
> > at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:492)
> > Caused by: org.apache.catalina.LifecycleException: Protocol handler
> initialization failed
> > at
> org.apache.catalina.connector.Connector.initInternal(Connector.java:995)
> > at
> org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
> > ... 12 more
> > Caused by: java.lang.IllegalArgumentException: Cannot store
> non-PrivateKeys
> > at org.apache.tomcat.util.net
> .AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:100)
> > at org.apache.tomcat.util.net
> .AbstractJsseEndpoint.initialiseSsl(AbstractJsseEndpoint.java:72)
> > at org.apache.tomcat.util.net
> .NioEndpoint.bind(NioEndpoint.java:244)
> > at org.apache.tomcat.util.net
> .AbstractEndpoint.init(AbstractEndpoint.java:1105)
> > at org.apache.tomcat.util.net
> .AbstractJsseEndpoint.init(AbstractJsseEndpoint.java:224)
> > at
> org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:581)
> > at
> org.apache.coyote.http11.AbstractHttp11Protocol.init(AbstractHttp11Protocol.java:68)
> > at
> org.apache.catalina.connector.Connector.initInternal(Connector.java:993)
> > ... 13 more
> > Caused by: java.security.KeyStoreException: Cannot store non-PrivateKeys
> > at
> sun.security.provider.JavaKeyStore.engineSetKeyEntry(JavaKeyStore.java:261)
> > at
> sun.security.provider.JavaKeyStore$JKS.engineSetKeyEntry(JavaKeyStore.java:56)
> > at
> sun.security.provider.KeyStoreDelegator.engineSetKeyEntry(KeyStoreDelegator.java:117)
> > at
> sun.security.provider.JavaKeyStore$DualFormatJKS.engineSetKeyEntry(JavaKeyStore.java:70)
> > at java.security.KeyStore.setKeyEntry(KeyStore.java:1140)
> > at org.apache.tomcat.util.net
> .SSLUtilBase.getKeyManagers(SSLUtilBase.java:313)
> > at org.apache.tomcat.util.net
> .SSLUtilBase.createSSLContext(SSLUtilBase.java:239)
> > at org.apache.tomcat.util.net
> .AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:98)
> > ... 20 more
> >
> > 05-Aug-2020 23:00:52.857 INFO [main]
> org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler
> ["https-jsse-nio-7443"]
> > 05-Aug-2020 23:00:52.861 SEVERE [main]
> org.apache.catalina.core.StandardService.initInternal Failed to initialize
> connector [Connector[HTTP/1.1-7443]]
> > org.apache.catalina.LifecycleException: Failed to initialize component
> [Connector[HTTP/1.1-7443]]
> > at
> org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:112)
> > at
> org.apache.catalina.core.StandardService.initInternal(StandardService.java:552)
> > at
> org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
> > at
> org.apache.catalina.core.StandardServer.initInternal(StandardServer.java:875)
> > at
> org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
> > at org.apache.catalina.startup.Catalina.load(Catalina.java:639)
> > at org.apache.catalina.startup.Catalina.load(Catalina.java:662)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:309)
> > at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:492)
> > Caused by: org.apache.catalina.LifecycleException: Protocol handler
> initialization failed
> > at
> org.apache.catalina.connector.Connector.initInternal(Connector.java:995)
> > at
> org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
> > ... 12 more
> > Caused by: java.lang.IllegalArgumentException: Cannot store
> non-PrivateKeys
> > at org.apache.tomcat.util.net
> .AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:100)
> > at org.apache.tomcat.util.net
> .AbstractJsseEndpoint.initialiseSsl(AbstractJsseEndpoint.java:72)
> > at org.apache.tomcat.util.net
> .NioEndpoint.bind(NioEndpoint.java:244)
> > at org.apache.tomcat.util.net
> .AbstractEndpoint.init(AbstractEndpoint.java:1105)
> > at org.apache.tomcat.util.net
> .AbstractJsseEndpoint.init(AbstractJsseEndpoint.java:224)
> > at
> org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:581)
> > at
> org.apache.coyote.http11.AbstractHttp11Protocol.init(AbstractHttp11Protocol.java:68)
> > at
> org.apache.catalina.connector.Connector.initInternal(Connector.java:993)
> > ... 13 more
>
> Caused by: java.security.KeyStoreException:
Cannot store non-PrivateKeys
>
If you pasted the full stack trace, then here we have the last "caused by",
showing one issue
> at
> sun.security.provider.JavaKeyStore.engineSetKeyEntry(JavaKeyStore.java:261)
> > at
> sun.security.provider.JavaKeyStore$JKS.engineSetKeyEntry(JavaKeyStore.java:56)
> > at
> sun.security.provider.KeyStoreDelegator.engineSetKeyEntry(KeyStoreDelegator.java:117)
> > at
> sun.security.provider.JavaKeyStore$DualFormatJKS.engineSetKeyEntry(JavaKeyStore.java:70)
> > at java.security.KeyStore.setKeyEntry(KeyStore.java:1140)
> > at org.apache.tomcat.util.net
> .SSLUtilBase.getKeyManagers(SSLUtilBase.java:313)
> > at org.apache.tomcat.util.net
> .SSLUtilBase.createSSLContext(SSLUtilBase.java:239)
> > at org.apache.tomcat.util.net
> .AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:98)
> > ... 20 more
> >
>
>
> I suspect that I will need to bring the Tomcat up to 8.5.57 here, too,
> before it will work. Presumably, that means tripping the same process
> that made the mess on the test spot instance.
>
> It seems that when the unwanted update happened,
>
> 1. /etc/tomcat8/server.xml was left at least relatively undisturbed: our
> connectors were undamaged.
>
> 2. /etc/tomcat8/tomcat-users.xml was also left undisturbed.
>
> 3. /var/lib/tomcat8/webapps/manager/WEB-INF/web.xml was also left at
> least relatively undisturbed: because of the size of our WAR files, we
> increase the max-file-size and max-request-size from 50MB to 500M, and
> that was as we left it.
>
> 4. /var/lib/tomcat8/webapps/manager/META-INF/context.xml, however, was
> reset to the "factory" state, with the RemoteAddrValve active.
>
> 5. The default ROOT context overlaid our ROOT context, leaving
> /var/lib/tomcat8/webapps/ROOT filled with both our files and the default
> ones. Our ROOT.war, on the other hand, was left intact, and if I stop
> Tomcat, remove the ROOT context directory, and then restart Tomcat, it
> does unpack our ROOT.war correctly.
>
> 6. The /var/lib/tomcat8/webapps/examples context directory, which we
> always remove, was reinstalled.
>
> Can anybody make sense of why some things changed, while others were
> left alone?
>
Final results of my Let's Encrypt project
Posted by "James H. H. Lampert" <ja...@touchtonecorp.com>.
Ladies and Gentlemen:
The server that had me tearing my hair out has now been entirely
switched over to Let's Encrypt, and it's working quite well, so far.
Thanks to everybody on this List, on the Orange County Linux User Group
List, on Server Fault, and on the Bitnami support board, who assisted.
In particular, thanks to Christopher Schultz. It is always good to be
able to stand upon the shoulders of a giant.
Some things I learned that may be of use to others:
1. If one is unable to get Certbot to work in a given situation, Lego
may be a viable alternative. It does, however, require a brief server
shutdown to run, as it does need to take over the ports while operating.
2. If one is having trouble getting Lego to work when you have ports
mapped (e.g., 8443 appearing as 443 from the outside via iptables),
adding "--http.port :80" and/or "--tls.port :8443" to the lego
invocation may help.
3. If one is having trouble getting Tomcat to use .crt and .key files,
it is not difficult to turn them into a PKCS12 keystore, which Tomcat
can then use. (Again, thanks, Mr. Schultz!)
--
James H. H. Lampert
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Let's Encrypt cert worked fine in 8.5.57, but connector fails in
8.5.40
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
James,
On 8/6/20 14:10, James H. H. Lampert wrote:
> On 8/6/20 9:37 AM, Christopher Schultz wrote:
>> $ openssl pkcs12 -export \ -inkey /etc/tomcat8/test.foo.net.key
>> \ -
>
> Dear Mr. Schultz:
>
> Is there supposed to be something after that last hyphen? When I
> type that command, I just get a terminal window full of helptext.
>
>
> And if I try the second command,
>> $ openssl pkcs12 -export \ -in /etc/tomcat8/test.foo.net.crt \
>> -inkey /etc/tomcat8/test.foo.net.key \ -certfile
>> /etc/tomcat8/test.foo.net.issuer.crt \ -out
>> /etc/tomcat8/test.foo.net.p12 \ -chain
>
> without the first, I get:
>> Error unable to get local issuer certificate getting chain.
The -chain argument doesn't take a filename or anything. It tells
openssl to chain the certs together to make sure everything is okay.
You may not need -chain but I seem to always do it.
My slides for Let's Encrypt + Tomcat say:
$ openssl pkcs12 -export -in [cert] -inkey [key] -certfile [chain]
- -out [p12file]
So maybe you don't need the -chain.
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl8sV+gACgkQHPApP6U8
pFgQmBAAvGp6ve2xFrdEUgyH6I2lqrULcct46j+KTrxRRFg9B9hEuAGh6+BclG+3
EF10QmpsOOWm1UChk9J06bCA0QQNxCpY5QrCebHEdGxn8SRct0UeLGFMsTHhli7R
ZrLkTrleOReMzwv++69S+kastreo/TpqziAtDrvMXXmcEo25YEV8ll7RkP+hhxVC
mTnmGjVtjRMOm9Ujj7LYqtRj/A2HLI3BJp5ZJyQAQm4rSZ+kHkVgExlG0LiZjNnI
ZPJGo394JzzrcjK7szbpcWzQlokHmW96a0vPt0k0DokWc2JM0jLS0aKHL2OTdqV9
gPCgeyVRDgVzdqGVjCpubL4H2kRilBc4vo2H2JtTU+bXJfTr59gfsFybGtRjv2US
KZJDeTjqf++LhsVEoRpKSGCKru7aDS89JZ01F9zGkOGk3DTHJjqNv+gz3p3ilmiS
/DF+EE1JnRiIJ7qE89QxBFKOKWz3TJOsDNYf39S931ZkDbi4wuYh15JZE9DSYOrB
5PoDZC8xWL+EBtLUmCTj3fpuwIvHSvo34TNvsgNcdEHtd4P3pSLhBYTeUsqU8dzh
j3GGn9jLI/zdlQH65bcjC7MY28ut0n2kqqnwu/myjzPlKt0B7KSDAVcoXqHRS5WV
0lLPiOCuwYtDLTf55/1/zDqLGEr8tcTb99VeXe+RvPQEKmRfrjA=
=1ZK9
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Let's Encrypt cert worked fine in 8.5.57, but connector fails in
8.5.40
Posted by "James H. H. Lampert" <ja...@touchtonecorp.com>.
On 8/6/20 9:37 AM, Christopher Schultz wrote:
> $ openssl pkcs12 -export \
> -inkey /etc/tomcat8/test.foo.net.key \
> -
Dear Mr. Schultz:
Is there supposed to be something after that last hyphen? When I type
that command, I just get a terminal window full of helptext.
And if I try the second command,
> $ openssl pkcs12 -export \
> -in /etc/tomcat8/test.foo.net.crt \
> -inkey /etc/tomcat8/test.foo.net.key \
> -certfile /etc/tomcat8/test.foo.net.issuer.crt \
> -out /etc/tomcat8/test.foo.net.p12 \
> -chain
without the first, I get:
> Error unable to get local issuer certificate getting chain.
--
JHHL
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Let's Encrypt cert worked fine in 8.5.57, but connector fails in
8.5.40
Posted by "James H. H. Lampert" <ja...@touchtonecorp.com>.
On 8/6/20 10:10 AM, Christopher Schultz wrote:
>>> $ openssl pkcs12 -export \ -in /etc/tomcat8/test.foo.net.crt \
>>> -inkey /etc/tomcat8/test.foo.net.key \ -certfile
>>> /etc/tomcat8/test.foo.net.issuer.crt \ -out
>>> /etc/tomcat8/test.foo.net.p12 \ -chain
>>>
>>> Then reconfigure your <Certificate> to use your keystore.
Dear Mr. Schultz (et al):
It was a bit of a challenge to find out how to use a PKCS12 keystore in
the Certificate clause, but not that difficult. And the "-chain" was not
necessary.
At any rate, congratulations, you have just cut my proverbial Gordian knot!
In my case, there's obviously no need for the
> curl https://localhost/manager/jmxproxy?invoke=Catalina%3Atype
> %3DProtocolHandler%2Cport%3D8443%2Caddress%3D
> %22127.0.0.1%22&op=reloadSslHostConfigs
in my renewal script, as given in your presentation, because it's
already necessary to shut down Tomcat for the renewal: the known-good
procedure for getting a Let's Encrypt on an Amazon Linux (not "2")
instance with a Bitnami Trac/SVN stack uses Lego, rather than Certbot,
and Lego needs to take over all the ports in order to do its magic
(probably why Lego is not as popular as Certbot).
And likewise, since I'm generating a PKCS12 keystore, rather than using
the certificate and key files directly, I was able to cut out making
local copies of those files, and just reference the ones that Lego put
in /opt/trac-1.2.3-11/letsencrypt/certificates/ directly.
--
James H. H. Lampert
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Let's Encrypt cert worked fine in 8.5.57, but connector fails in
8.5.40
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
James,
On 8/6/20 13:03, James H. H. Lampert wrote:
> On 8/6/20 9:37 AM, Christopher Schultz wrote: . . .
>> As a short-term workaround, you can load your stuff into a
>> keystore like this:
>>
>> $ openssl pkcs12 -export \ -inkey /etc/tomcat8/test.foo.net.key
>> \ -
>>
>> $ openssl pkcs12 -export \ -in /etc/tomcat8/test.foo.net.crt \
>> -inkey /etc/tomcat8/test.foo.net.key \ -certfile
>> /etc/tomcat8/test.foo.net.issuer.crt \ -out
>> /etc/tomcat8/test.foo.net.p12 \ -chain
>>
>> Then reconfigure your <Certificate> to use your keystore.
>
> That could even be a permanent workaround if it can be done
> non-interactively.
It /can/ be done non-interactively. If you see this presentation on
using Let's Encrypt with Tomcat, there are a few slides toward the end
about automation.
http://tomcat.apache.org/presentations.html#latest-lets-encrypt
> How, I wonder, is it that the PEM files work just fine after the
> "unwanted update" that bumped Tomcat up from 40 to 57, and pulled
> in a slightly newer Java 1.8?
Oh. I completely forgot to look at the version numbers. And now I know
what the problem is, and why my mock-up test I just wrote works fine.
See the changelog for 8.5.51:
http://tomcat.apache.org/tomcat-8.5-doc/changelog.html#Tomcat_8.5.51_(ma
rkt)
Specifically, this entry:
Add: Add support for RFC 5915 formatted, unencrypted EC key files when
using a JSSE based TLS connector. (markt)
When using an unencrypted EC key (as it seems you are using), Java
doesn't know how to decode that type of file directly as we can with
e.g. RSA non-encrypted keys. So we have to help it along by
directly-parsing the ASN.1 [1] data and re-writing it into a form Java
can understand.
So you must use Tomcat 8.5.51 or later to use PEM-encoded
non-encrypted Elliptic curve key files (got all that?).
- -chris
[1] There is a special place in hell reserved for everyone responsible
for ASN.1
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl8sOXoACgkQHPApP6U8
pFjxYxAAxCiE18mXicHQJUUz8PUbhYa/aQEwdaj+IIPUpAhHExcb31Msa9Nw+xiC
YTXHwb5L51Sr/FGKqDfSvl84DQYZipkudwo2uyhIDJR++6+qkbNvMiXlOw757UXi
zkeQv91Cc6Q/rpaxiuifw2kUSejzl/yVxgAWb6XXS1RwRUwq2UEjM+aK3HvCm1C8
6yMkxi5E7ISxiQ1cui1IMGnov4BqIp+tJT+z+PlmtbBJAGHwY/DeW7hU2Yl1rn+B
yI3ydhSxK7cRVzNbvRMrNXjLCF0Lo8S1b0K+lqF85DlO91koQQMMEBrt9X9Q8E5m
xdV161SGfSV5N27JNTdHyI/OjL9ALGXyfBJcENP6J5ZaZ2xouO9XLd+jENyj0zAz
8r5wczDe3uPNiPAIGkm8eAP5O8qz48RZ/N7TKM/kpAlknb4B6zSZN0meteqopWu4
rkCvb4Z2YOU4EiqdtbxyJuoMaFzhgHquDQBZS0aKA56pVjHNi+adML/X9mCsQ+7Z
khZpuSnOcq1pynqW8EuX4L6IZUtqX3K192kLOOvnNnj/Fys5MToj3259hXoz7LYn
2Pncj0E+Y7W3H54CviF0vj6nPmCFkOIaFfQ+/GdLexDrJ34LpNtkMyWAz3LPVYhh
8moCiMTDowX1UGPYmIJ+IlkdhsXBycmqToMBzgPnKlJKHUk5kuM=
=5DNS
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Let's Encrypt cert worked fine in 8.5.57, but connector fails in
8.5.40
Posted by "James H. H. Lampert" <ja...@touchtonecorp.com>.
On 8/6/20 9:37 AM, Christopher Schultz wrote:
. . .
> As a short-term workaround, you can load your stuff into a keystore
> like this:
>
> $ openssl pkcs12 -export \
> -inkey /etc/tomcat8/test.foo.net.key \
> -
>
> $ openssl pkcs12 -export \
> -in /etc/tomcat8/test.foo.net.crt \
> -inkey /etc/tomcat8/test.foo.net.key \
> -certfile /etc/tomcat8/test.foo.net.issuer.crt \
> -out /etc/tomcat8/test.foo.net.p12 \
> -chain
>
> Then reconfigure your <Certificate> to use your keystore.
Dear Mr. Schultz:
Thanks.
That could even be a permanent workaround if it can be done
non-interactively.
How, I wonder, is it that the PEM files work just fine after the
"unwanted update" that bumped Tomcat up from 40 to 57, and pulled in a
slightly newer Java 1.8?
--
JHHL
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Let's Encrypt cert worked fine in 8.5.57, but connector fails in
8.5.40
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
James,
On 8/5/20 19:46, James H. H. Lampert wrote:
> I've now proceeded to the "real" server, with the Tomcat portion of
> the procedure refined to give me plenty of "undo" capability. And
> it turns out I need it.
>
> It seems that with the unwanted update to 7.0.57 that happened on
> launching the test spot instances, the Let's Encrypt certs worked
> just fine.
>
> But applying the procedure to the *real* development instance
> (7.0.40) blew up in my face, failing to open the connectors. Here
> is an excerpt from catalina.out, showing the stacktraces.
>
>> 05-Aug-2020 23:00:52.189 INFO [main]
>> org.apache.catalina.startup.VersionLoggerListener.log Server
>> version: Apache Tomcat/8.5.40
>>
>> [snip]
>>
>> 05-Aug-2020 23:00:52.195 INFO [main]
>> org.apache.catalina.startup.VersionLoggerListener.log JVM
>> Version: 1.8.0_201-b09
>>
>> [snip]
>>
>> Caused by: java.security.KeyStoreException: Cannot store
>> non-PrivateKeys at
>> sun.security.provider.JavaKeyStore.engineSetKeyEntry(JavaKeyStore.jav
a:261)
>>
>>
>>
at
>> sun.security.provider.JavaKeyStore$JKS.engineSetKeyEntry(JavaKeyStore
.java:56)
>>
>>
>>
at
>> sun.security.provider.KeyStoreDelegator.engineSetKeyEntry(KeyStoreDel
egator.java:117)
>>
>>
>>
at
>> sun.security.provider.JavaKeyStore$DualFormatJKS.engineSetKeyEntry(Ja
vaKeyStore.java:70)
>>
>>
>>
at java.security.KeyStore.setKeyEntry(KeyStore.java:1140)
>> at
>> org.apache.tomcat.util.net.SSLUtilBase.getKeyManagers(SSLUtilBase.jav
a:313)
*sigh*
Okay,
>>
this is a confluence of:
1. You are using PEM files instead of a Java keystore
1a. Tomcat handles this by creating an in-memory KeyStore and loading
your key + certificate into it
2. Java 1.8 doesn't like non-private keys in KeyStores for some reason
2a. Java somehow thinks your key + cert aren't "private" :(
- From your other thread, I think you are doing this:
<Certificate certificateFile="/etc/tomcat8/test.foo.net.crt"
certificateKeyFile="/etc/tomcat8/test.foo.net.key"
certificateChainFile="/etc/tomcat8/test.foo.net.issuer.crt"
/>
I think the problem is that there is no "password". Here is the line
of code bombing:
// Switch to in-memory key store
ksUsed = KeyStore.getInstance("JKS");
ksUsed.load(null, null);
ksUsed.setKeyEntry(keyAlias,
privateKeyFile.getPrivateKey(), keyPass.toCharArray(),
chain.toArray(new Certificate[0]));
It's the call to setKeyEntry which fails. The key alias is "tomcat"
unless you have explicitly set the alias. The cert and key are
obvious, but the keypass is probably empty.. or something.
The default key password is whatever the keystore password is. The
default keystore password is "changeit". So I think this is why we
aren't getting an NPE when we call keyPass.toCharArray.
This works in other scenarios. Not sure what is the exact problem with
yours.
ks.load(null, null) looks suspicious. Maybe it's not okay to use a
non-initialized KeyStore object. But I think maybe it should be:
ks.load(null, keyPass.toCharArray());
I'll have too play with this a little locally to see what the problem is
.
As a short-term workaround, you can load your stuff into a keystore
like this:
$ openssl pkcs12 -export \
-inkey /etc/tomcat8/test.foo.net.key \
-
$ openssl pkcs12 -export \
-in /etc/tomcat8/test.foo.net.crt \
-inkey /etc/tomcat8/test.foo.net.key \
-certfile /etc/tomcat8/test.foo.net.issuer.crt \
-out /etc/tomcat8/test.foo.net.p12 \
-chain
Then reconfigure your <Certificate> to use your keystore.
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl8sMeQACgkQHPApP6U8
pFh4HA/+KZTBU0ghzVqLmalXNJN3P89/FrhJqJWOfQyn3noqUBRhFsyDcOK0z+uR
HxC85OClpZ1LaJwNE4LRArG26chWTGMQm4Z7u1UzhWtz0pIa7wijGj3fQx/EXmQW
ePdOlAcAJnFKUZJr5giDAT+Sl8OC76NbfaN/fz6gqESXxqdxRxHPTrGBVgHol7v1
p4fNiU0T+cw2wQNwq30tHT378wNsC2xozotw/vdr2EQbX7HK/S+tRFJziupUcHzt
cJAWymUiE6Vfw19zSRGF/Fp9s9o/fCaCJKSVl2CEMbR8MdytjmTaQspAK9CXXXpo
Ue8wDuDMNRB6afq3ftoNYJQwfNCvOADTw5L0Xwr5hb+r8xnRBZQQodtBNVznZ1Of
6lnkrqVAYpuUklDCbpWTB52LjE08IRhTaBCJPuueQL9Yxlt4nO2NndJHLTvANB0L
sqEbmRLsROD/eDCaSq7VZzWAnu17C1iO0i7ztsr3JUjregY9EoCs/YOxX71jicHF
10B4HMmqX18DuJtWTSiMQSvy3JqVcCPOIGBRIWxTKS93xGsr5MeAYnBxKrsOJuF1
L+uD56u2pZwkkT3HHiHfXB/db+1mE5GugkY3qBIrOMaTUtS+UPddxpOu59fgPhvC
e9SPhx9pEgUlfuFcFvbWwhv9K1mlAx8PcZMZrwGPEO1ibjfqSaU=
=Fprl
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org