You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by bu...@apache.org on 2019/06/13 15:18:52 UTC
[Bug 63500] New: Core dump using APR tomcat native with
certificateRevocationListFile
https://bz.apache.org/bugzilla/show_bug.cgi?id=63500
Bug ID: 63500
Summary: Core dump using APR tomcat native with
certificateRevocationListFile
Product: Tomcat 9
Version: 9.0.21
Hardware: PC
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: Connectors
Assignee: dev@tomcat.apache.org
Reporter: bruno.pmarinho@gmail.com
Target Milestone: -----
I'm using tomcat 9.0.21 and tomcat-native 1.2.21 on top of CentOS 7.6.1810.
Current issue is this one from catalina.out:
-----
13-Jun-2019 10:43:32.132 INFO [main]
org.apache.catalina.core.AprLifecycleListener.lifecycleEvent Loaded APR based
Apache Tomcat Native library [1.2.21] using APR version [1.4.8].
13-Jun-2019 10:43:32.135 INFO [main]
org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR capabilities:
IPv6 [true], sendfile [true], accept filters [false], random [true].
13-Jun-2019 10:43:32.135 INFO [main]
org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR/OpenSSL
configuration: useAprConnector [false], useOpenSSL [true]
13-Jun-2019 10:43:32.141 INFO [main]
org.apache.catalina.core.AprLifecycleListener.initializeSSL OpenSSL
successfully initialized [OpenSSL 1.0.2k-fips 26 Jan 2017]
13-Jun-2019 10:43:32.518 INFO [main] org.apache.coyote.AbstractProtocol.init
Initializing ProtocolHandler ["http-apr-2007"]
13-Jun-2019 10:43:32.545 INFO [main] org.apache.coyote.AbstractProtocol.init
Initializing ProtocolHandler ["https-openssl-apr-2008"]
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f989117c860, pid=23986, tid=0x00007f9910185700
#
# JRE version: OpenJDK Runtime Environment (8.0_191-b12) (build 1.8.0_191-b12)
# Java VM: OpenJDK 64-Bit Server VM (25.191-b12 mixed mode linux-amd64
compressed oops)
# Problematic frame:
# C [libcrypto.so.10+0x168860] X509_STORE_set_flags+0x0
#
# Failed to write core dump. Core dumps have been disabled. To enable core
dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /opt/temp/java_error23986.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#
-----
The crash log shows
-----
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j
org.apache.tomcat.jni.SSLContext.setCARevocation(JLjava/lang/String;Ljava/lang/String;)Z+0
j
org.apache.tomcat.util.net.openssl.OpenSSLContext.addCertificate(Lorg/apache/tomcat/util/net/SSLHostConfigCertificate;)V+77
j
org.apache.tomcat.util.net.openssl.OpenSSLContext.init([Ljavax/net/ssl/KeyManager;[Ljavax/net/ssl/TrustManager;Ljava/security/SecureRandom;)V+191
j
org.apache.tomcat.util.net.SSLUtilBase.createSSLContext(Ljava/util/List;)Lorg/apache/tomcat/util/net/SSLContext;+16
j
org.apache.tomcat.util.net.AprEndpoint.createSSLContext(Lorg/apache/tomcat/util/net/SSLHostConfig;)V+81
-----
Which is the code
https://github.com/apache/tomcat-native/blob/7a20a500587dc92944dee05cee5b388ae8443842/native/src/sslcontext.c#L640
Connector configuration:
-----
<Listener className="org.apache.catalina.core.AprLifecycleListener"
SSLEngine="on" SSLRandomSeed="builtin" />
<Connector
allowTrace="false"
URIEncoding="UTF-8"
acceptorThreadCount="2"
acceptCount="400"
connectionTimeout="20000"
maxKeepAliveRequests="100"
keepAliveTimeout="5000"
maxThreads="400"
minSpareThreads="25"
tcpNoDelay="true"
processorCache="750"
protocol="org.apache.coyote.http11.Http11AprProtocol"
port="2008"
scheme="https"
secure="true"
SSLEnabled="true">
<SSLHostConfig
certificateVerification="optional"
caCertificateFile="/certs/server/x509_trusted.crt"
certificateRevocationListFile="/certs/server/x509_trusted.crl"
protocols="all -SSLv3 -TLSv1 -TLSv1.1 -TLSv1.3"
ciphers="ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256"
honorCipherOrder="true"
disableCompression="true">
<Certificate
certificateFile="/certs/server/server.crt"
certificateChainFile="/certs/server/server_ca_chain.crt"
certificateKeyFile="/certs/server/server.key"
certificateKeyPassword="abc" />
</SSLHostConfig>
</Connector>
-----
When I remove the line setting the revogation list,
"certificateRevocationListFile=\"/certs/server/x509_trusted.crl\"", it just
works. Same PEM file works in old tomcat-7.0.69 using also APR and tomcat
native.
Tried versions back until 9.0.17 with same issue. With 9.0.16 and 9.014 I have
different issue.
On tomcat 9.0.14 same setup worked but on shutdown I had a core dump. With
tomcat 9.0.16 same setup worked but on shutdown I had an even weird core dump.
Starting tomcat 9.0.17 it stopped starting. I noticed that tomcat native
changed from 1.2.19 to 1.2.21 when that happen.
Tested also building a tomcat-native with latest openssl and latest apr as
static linked in with same issue. So I assume it's not some weird library issue
but something else.
Happy to provide more data or even do other tests.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 63500] Core dump using APR tomcat native with
certificateRevocationListFile
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63500
--- Comment #11 from Bruno <br...@gmail.com> ---
Using the fix with tomcat 9.0.22 and tomcat-native 1.2.23 and the server
doesn't core dump anymore when I add CRL to SSLHostConfig using configuration
certificateRevocationListFile in format
-----BEGIN X509 CRL-----
But have a new issue, I'm unable to use valid X509 client certs. I get the
message from curl
curl: (35) error:1401E418:SSL routines:CONNECT_CR_FINISHED:tlsv1 alert unknown
ca
If I remove the CRL config client certs work as expected.
Is there a way to understand what is the issue? Was unable to find a way to
enable SSL logging being OpenSSL.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 63500] Core dump using APR tomcat native with
certificateRevocationListFile
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63500
--- Comment #3 from Bruno <br...@gmail.com> ---
Created attachment 36624
--> https://bz.apache.org/bugzilla/attachment.cgi?id=36624&action=edit
Native stack trace from core dump of same process
gdb java core
where
bt full
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 63500] Core dump using APR tomcat native with
certificateRevocationListFile
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63500
--- Comment #9 from Bruno <br...@gmail.com> ---
Thank you for the investigation and fix.
What is the release cycle for tomcat native and tomcat 9?
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 63500] Core dump using APR tomcat native with
certificateRevocationListFile
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63500
--- Comment #6 from Mark Thomas <ma...@apache.org> ---
The stack trace suggests the fix for bug 62094 was incorrect.
Looking at the trace, the proposed patch and the commit, I think the patch was
not applied correctly.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 63500] Core dump using APR tomcat native with
certificateRevocationListFile
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63500
Mark Thomas <ma...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |FIXED
--- Comment #8 from Mark Thomas <ma...@apache.org> ---
I've dug into this some more. I see why the applied patch for bug 62094 differs
from the proposed patch but the change made was incomplete. That was the cause
of the crash observed here.
Further testing found crashes on invalid CRL files and on shutdown. These have
also been fixed.
I have tested this with a CRL and both a revoked and non-revoked cert and
everything appears to as expected.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 63500] Core dump using APR tomcat native with
certificateRevocationListFile
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63500
Mark Thomas <ma...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Component|Connectors |Library
Version|9.0.21 |1.2.21
Product|Tomcat 9 |Tomcat Native
Target Milestone|----- |---
--- Comment #7 from Mark Thomas <ma...@apache.org> ---
Correcting product
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 63500] Core dump using APR tomcat native with
certificateRevocationListFile
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63500
--- Comment #2 from Bruno <br...@gmail.com> ---
Created attachment 36623
--> https://bz.apache.org/bugzilla/attachment.cgi?id=36623&action=edit
Error report generated by the JVM
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 63500] Core dump using APR tomcat native with
certificateRevocationListFile
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63500
--- Comment #10 from Mark Thomas <ma...@apache.org> ---
Current plans are:
- 9.0.x early July
- Native 1.2.22 before then so it can be packaged in the next 9.0.x release
You can follow progress on the dev@ list.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 63500] Core dump using APR tomcat native with
certificateRevocationListFile
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63500
--- Comment #5 from Bruno <br...@gmail.com> ---
Created attachment 36625
--> https://bz.apache.org/bugzilla/attachment.cgi?id=36625&action=edit
Native stack trace from core dump of same process with debuginfo
Installed the debug info and retrieved new native trace.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 63500] Core dump using APR tomcat native with
certificateRevocationListFile
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63500
--- Comment #1 from Christopher Schultz <ch...@christopherschultz.net> ---
Looks like this fails on startup. Can you enable core dumps, re-try, and get a
native stack trace of the error?
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 63500] Core dump using APR tomcat native with
certificateRevocationListFile
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=63500
--- Comment #4 from Bruno <br...@gmail.com> ---
It does fail right at the beginning.
Attached the traces.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org