You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by "Hrivnak, Dan" <Da...@gallup.com> on 2016/01/21 20:57:54 UTC
TLSv1.2 handshake failure on outgoing connections
Environments:
* Mac OS X 10.10.5; Tomcat 7.0.67, 8.0.30; Java 1.8.0_60
* RHEL 6 (Kernel 2.6.32); Tomcat 7.0.67; Java 1.8.0_60
Problem:
Making an outgoing HTTPS connection from Axis2 client code living inside the war, I get a failure during the TLSv1.2 handshake saying “Could not generate DH keypair”. Unlike most examples I found online, there was no additional information about the key size. The same client code when run from a unit test using plain Java works just fine. Below are snippets of one difference I noticed with the Server key in the logs:
Running from within Tomcat:
*** ECDH ServerKeyExchange
Signature Algorithm SHA1withRSA
Server key: Sun EC public key, 256 bits
public x coord: 112918107330736490567973848952126837545983212398065462286267971433368342872647
public y coord: 30155777565237297899065179509488316850099974838272315813007505317208002177712
parameters: secp256r1 [NIST P-256, X9.62 prime256v1] (1.2.840.10045.3.1.7)
http-bio-8080-exec-6, handling exception: java.lang.RuntimeException: Could not generate DH keypair
%% Invalidated: [Session-4, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384]
http-bio-8080-exec-6, SEND TLSv1.2 ALERT: fatal, description = internal_error
Running from plain Java (within IntelliJ as a JUnit test in case that matters):
*** ECDH ServerKeyExchange
Signature Algorithm SHA1withRSA
Server key: EC Public Key
X: 726ad077a87d97604c4507989bb1d6c4715ee23399e42543e19dc39048abe3cb
Y: 904cde963f872bd32691e86565e6f0ab09ebf833ee93edd0200a9d81299410e2
*** ServerHelloDone
*** ECDHClientKeyExchange
ECDH Public value: { 4, 19, 187, 197, 193, 165, 157, 121, 79, 161, 160, 25, 239, 100, 105, 199, 101, 160, 54, 96, 128, 159, 61, 83, 144, 237, 233, 235, 118, 100, 47, 50, 85, 98, 192, 79, 174, 211, 10, 218, 35, 207, 203, 3, 88, 41, 100, 126, 223, 10, 139, 18, 101, 59, 243, 152, 125, 4, 241, 201, 153, 232, 172, 74, 0 }
main, WRITE: TLSv1.2 Handshake, length = 70
Note the difference in the "Server key". Is Tomcat somehow intercepting the outgoing connection and handling it itself? If so, where would I configure the security settings for that type of connection? Everything I've been able to find relates to configuring Tomcat as the server not as the client for SSL/TLS-related things. Please let me know if there is more information that would help!
Thank you,
Dan Hrivnak
All information in this message is confidential and may be legally privileged. Only intended recipients are authorized to use it.
Re: TLSv1.2 handshake failure on outgoing connections
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Dan,
On 1/29/16 3:55 PM, Hrivnak, Dan wrote:
> In case anyone was following this or seeing similar issues, I was
> able to track it down. When debugging into the Axis library code
> itself I was able to see one more frame of the stack trace
> (java.security.InvalidAlgorithmParameterException: parameter
> object not a ECParameterSpec) before it got swallowed up, which led
> me to this article:
> http://iwang.github.io/support/2014/03/14/cxf-cause-https-error.html
>
>
>
> Basically, my classpath had a version of the bouncycastle library
> meant for JDK 1.4, causing the issue with the ECDH key exchange
> during the TLS handshake. Removing it solved the problem!
Oh, man. That'll definitely do it.
> Now, in case you are still reading, I can explain why the problem
> only appeared in the context of running inside Tomcat. Looking at
> the maven dependency tree to see where bouncycastle for JDK 1.4
> came from, I learned that jfreechart listed it as a dependency.
Nice! I'm always careful to make sure that all my charts are
encrypted, so it's great that jFreeChart pulls BC as a dependency. (WTF?
)
> Axis, on the other hand, listed bouncycastle for JDK 1.5+ in its
> transitive dependencies. Since my unit test was inside a module
> that wasn’t concerned with jfreechart, its classpath only had the
> newer bouncycastle library. But Tomcat, since its classpath
> contained everything my entire application cared about, had both
> versions of bouncycastle. And apparently the classloader decided to
> pick up the old one at runtime.
Heisenbugs all the way down.
> The moral of the story is that Byteman will be a great tool to
> keep handy for times when a stack trace is dropped on the floor by
> code you don’t control.
Looks like a handy tool, especially being able to predictably stall
programs for multi-threaded testing. Force the JVM to behave like
those theoretical "thread A does this then thread B does this"
interleave diagrams that show the potential for deadlock/whatever? Sweet
.
I'm glad you got to the bottom of this.
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlar6PcACgkQ9CaO5/Lv0PBd9wCgrN/jOVqqiTazXN5YSKvUn2hh
l1sAnjcD9/pwcZyAcz2yb6Bw7wSJGHG+
=cNHp
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: TLSv1.2 handshake failure on outgoing connections
Posted by "Hrivnak, Dan" <Da...@gallup.com>.
In case anyone was following this or seeing similar issues, I was able to track it down. When debugging into the Axis library code itself I was able to see one more frame of the stack trace (java.security.InvalidAlgorithmParameterException: parameter object not a ECParameterSpec) before it got swallowed up, which led me to this article: http://iwang.github.io/support/2014/03/14/cxf-cause-https-error.html
Basically, my classpath had a version of the bouncycastle library meant for JDK 1.4, causing the issue with the ECDH key exchange during the TLS handshake. Removing it solved the problem!
Now, in case you are still reading, I can explain why the problem only appeared in the context of running inside Tomcat. Looking at the maven dependency tree to see where bouncycastle for JDK 1.4 came from, I learned that jfreechart listed it as a dependency. Axis, on the other hand, listed bouncycastle for JDK 1.5+ in its transitive dependencies. Since my unit test was inside a module that wasn’t concerned with jfreechart, its classpath only had the newer bouncycastle library. But Tomcat, since its classpath contained everything my entire application cared about, had both versions of bouncycastle. And apparently the classloader decided to pick up the old one at runtime.
The moral of the story is that Byteman will be a great tool to keep handy for times when a stack trace is dropped on the floor by code you don’t control.
Thanks to anyone who took the time to help me out!
Dan Hrivnak
On 1/25/16, 5:19 PM, "Christopher Schultz" <ch...@christopherschultz.net> wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>Dan,
>
>On 1/25/16 9:28 AM, Hrivnak, Dan wrote:
>> Thank you Chris! I’m glad to hear that Tomcat should have nothing
>> to do with this as that helps narrow down what I should look at.
>> The unit test (really an integration test) spins up an app server
>> using Guice and makes a call to the same remote service (verified
>> same URL, certificate chains, etc). The only difference I can find
>> is that one is running within Tomcat and one isn’t. The actual
>> client code is using Axis 2 to call a SOAP service, so the raw HTTP
>> connection code is inside the Axis library unfortunately.
>
>You might want to check the effective system properties to see if
>there is a JVM-wide trust store in use, or any other TLS-related
>properties. For instance, it's possible to enable/disable protocols
>and such using system properties, and it's also possible to configure
>everything explicitly using code.
>
>Tomcat does the latter for incoming connections, but, as I mentioned,
>doesn't have any hooks into outgoing connections. You should be
>dealing with JSSE, though through Axis in your case.
>
>Good luck,
>- -chris
>
>-----BEGIN PGP SIGNATURE-----
>Comment: GPGTools - http://gpgtools.org
>Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
>iEYEARECAAYFAlamrZ4ACgkQ9CaO5/Lv0PDUYwCgtYzYzC/ZhuqxUX3/8PkF5rEF
>ETEAn0a92HLNgFmIK0z/4zWsv4qP2YAz
>=svtS
>-----END PGP SIGNATURE-----
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: users-help@tomcat.apache.org
>
All information in this message is confidential and may be legally privileged. Only intended recipients are authorized to use it.
Re: TLSv1.2 handshake failure on outgoing connections
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Dan,
On 1/25/16 9:28 AM, Hrivnak, Dan wrote:
> Thank you Chris! I’m glad to hear that Tomcat should have nothing
> to do with this as that helps narrow down what I should look at.
> The unit test (really an integration test) spins up an app server
> using Guice and makes a call to the same remote service (verified
> same URL, certificate chains, etc). The only difference I can find
> is that one is running within Tomcat and one isn’t. The actual
> client code is using Axis 2 to call a SOAP service, so the raw HTTP
> connection code is inside the Axis library unfortunately.
You might want to check the effective system properties to see if
there is a JVM-wide trust store in use, or any other TLS-related
properties. For instance, it's possible to enable/disable protocols
and such using system properties, and it's also possible to configure
everything explicitly using code.
Tomcat does the latter for incoming connections, but, as I mentioned,
doesn't have any hooks into outgoing connections. You should be
dealing with JSSE, though through Axis in your case.
Good luck,
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlamrZ4ACgkQ9CaO5/Lv0PDUYwCgtYzYzC/ZhuqxUX3/8PkF5rEF
ETEAn0a92HLNgFmIK0z/4zWsv4qP2YAz
=svtS
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: TLSv1.2 handshake failure on outgoing connections
Posted by "Hrivnak, Dan" <Da...@gallup.com>.
Thank you Chris! I’m glad to hear that Tomcat should have nothing to do with this as that helps narrow down what I should look at. The unit test (really an integration test) spins up an app server using Guice and makes a call to the same remote service (verified same URL, certificate chains, etc). The only difference I can find is that one is running within Tomcat and one isn’t. The actual client code is using Axis 2 to call a SOAP service, so the raw HTTP connection code is inside the Axis library unfortunately.
Thanks again for your help!
Dan Hrivnak
On 1/22/16, 2:36 PM, "Christopher Schultz" <ch...@christopherschultz.net> wrote:
>Dan,
>
>On 1/21/16 2:57 PM, Hrivnak, Dan wrote:
>> Environments:
>> * Mac OS X 10.10.5; Tomcat 7.0.67, 8.0.30; Java 1.8.0_60
>> * RHEL 6 (Kernel 2.6.32); Tomcat 7.0.67; Java 1.8.0_60
>>
>> Problem:
>> Making an outgoing HTTPS connection from Axis2 client code living inside the war, I get a failure during the TLSv1.2 handshake saying “Could not generate DH keypair”. Unlike most examples I found online, there was no additional information about the key size. The same client code when run from a unit test using plain Java works just fine. Below are snippets of one difference I noticed with the Server key in the logs:
>>
>>
>>
>> Running from within Tomcat:
>> *** ECDH ServerKeyExchange
>> Signature Algorithm SHA1withRSA
>> Server key: Sun EC public key, 256 bits
>> public x coord: 112918107330736490567973848952126837545983212398065462286267971433368342872647
>> public y coord: 30155777565237297899065179509488316850099974838272315813007505317208002177712
>> parameters: secp256r1 [NIST P-256, X9.62 prime256v1] (1.2.840.10045.3.1.7)
>> http-bio-8080-exec-6, handling exception: java.lang.RuntimeException: Could not generate DH keypair
>> %% Invalidated: [Session-4, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384]
>> http-bio-8080-exec-6, SEND TLSv1.2 ALERT: fatal, description = internal_error
>>
>>
>>
>> Running from plain Java (within IntelliJ as a JUnit test in case that matters):
>> *** ECDH ServerKeyExchange
>> Signature Algorithm SHA1withRSA
>> Server key: EC Public Key
>> X: 726ad077a87d97604c4507989bb1d6c4715ee23399e42543e19dc39048abe3cb
>> Y: 904cde963f872bd32691e86565e6f0ab09ebf833ee93edd0200a9d81299410e2
>>
>> *** ServerHelloDone
>> *** ECDHClientKeyExchange
>> ECDH Public value: { 4, 19, 187, 197, 193, 165, 157, 121, 79, 161, 160, 25, 239, 100, 105, 199, 101, 160, 54, 96, 128, 159, 61, 83, 144, 237, 233, 235, 118, 100, 47, 50, 85, 98, 192, 79, 174, 211, 10, 218, 35, 207, 203, 3, 88, 41, 100, 126, 223, 10, 139, 18, 101, 59, 243, 152, 125, 4, 241, 201, 153, 232, 172, 74, 0 }
>> main, WRITE: TLSv1.2 Handshake, length = 70
>>
>>
>> Note the difference in the "Server key". Is Tomcat somehow intercepting the outgoing connection and handling it itself? If so, where would I configure the security settings for that type of connection? Everything I've been able to find relates to configuring Tomcat as the server not as the client for SSL/TLS-related things. Please let me know if there is more information that would help!
>
>Tomcat has no part in this conversation.
>
>Something definitely looks fishy, here. What does your unit test code
>look like? What about the code that runs from within the webapp?
>
>Are you sure you are contacting the same URL in both cases?
>
>-chris
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: users-help@tomcat.apache.org
>
All information in this message is confidential and may be legally privileged. Only intended recipients are authorized to use it.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: TLSv1.2 handshake failure on outgoing connections
Posted by Christopher Schultz <ch...@christopherschultz.net>.
Dan,
On 1/21/16 2:57 PM, Hrivnak, Dan wrote:
> Environments:
> * Mac OS X 10.10.5; Tomcat 7.0.67, 8.0.30; Java 1.8.0_60
> * RHEL 6 (Kernel 2.6.32); Tomcat 7.0.67; Java 1.8.0_60
>
> Problem:
> Making an outgoing HTTPS connection from Axis2 client code living inside the war, I get a failure during the TLSv1.2 handshake saying “Could not generate DH keypair”. Unlike most examples I found online, there was no additional information about the key size. The same client code when run from a unit test using plain Java works just fine. Below are snippets of one difference I noticed with the Server key in the logs:
>
>
>
> Running from within Tomcat:
> *** ECDH ServerKeyExchange
> Signature Algorithm SHA1withRSA
> Server key: Sun EC public key, 256 bits
> public x coord: 112918107330736490567973848952126837545983212398065462286267971433368342872647
> public y coord: 30155777565237297899065179509488316850099974838272315813007505317208002177712
> parameters: secp256r1 [NIST P-256, X9.62 prime256v1] (1.2.840.10045.3.1.7)
> http-bio-8080-exec-6, handling exception: java.lang.RuntimeException: Could not generate DH keypair
> %% Invalidated: [Session-4, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384]
> http-bio-8080-exec-6, SEND TLSv1.2 ALERT: fatal, description = internal_error
>
>
>
> Running from plain Java (within IntelliJ as a JUnit test in case that matters):
> *** ECDH ServerKeyExchange
> Signature Algorithm SHA1withRSA
> Server key: EC Public Key
> X: 726ad077a87d97604c4507989bb1d6c4715ee23399e42543e19dc39048abe3cb
> Y: 904cde963f872bd32691e86565e6f0ab09ebf833ee93edd0200a9d81299410e2
>
> *** ServerHelloDone
> *** ECDHClientKeyExchange
> ECDH Public value: { 4, 19, 187, 197, 193, 165, 157, 121, 79, 161, 160, 25, 239, 100, 105, 199, 101, 160, 54, 96, 128, 159, 61, 83, 144, 237, 233, 235, 118, 100, 47, 50, 85, 98, 192, 79, 174, 211, 10, 218, 35, 207, 203, 3, 88, 41, 100, 126, 223, 10, 139, 18, 101, 59, 243, 152, 125, 4, 241, 201, 153, 232, 172, 74, 0 }
> main, WRITE: TLSv1.2 Handshake, length = 70
>
>
> Note the difference in the "Server key". Is Tomcat somehow intercepting the outgoing connection and handling it itself? If so, where would I configure the security settings for that type of connection? Everything I've been able to find relates to configuring Tomcat as the server not as the client for SSL/TLS-related things. Please let me know if there is more information that would help!
Tomcat has no part in this conversation.
Something definitely looks fishy, here. What does your unit test code
look like? What about the code that runs from within the webapp?
Are you sure you are contacting the same URL in both cases?
-chris
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org