You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Mark Thomas <ma...@apache.org> on 2016/02/14 22:45:25 UTC
NIO + JSSE + NIO + OpenSSL
All,
In preparation for the connector selection webinar next week, I just did
a quick test of NIO + JSSE and NIO + OpenSSL.
I was working with 9.0.x trunk including my JASPIC patch
NIO + JSSE ~8200 requests/second
Add the native lib to $CATALINA_BASE/bin and restart.
No other changes at all.
NIO + OpenSSL ~12300 requests/second
Simply dropping in the native library improves TLS performance by
roughly 50%.
Kudos to remm and jfclere.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: NIO + JSSE + NIO + OpenSSL
Posted by Rémy Maucherat <re...@apache.org>.
2016-02-15 9:11 GMT+01:00 Mark Thomas <ma...@apache.org>:
> > Direct buffers help
> > OpenSSL a lot for example (socket.directBuffer and socket.directSslBuffer
> > to true). Also one important item is to make sure the tests all use the
> > same cipher, especially with ab (JSSE might not use the same cipher as
> > OpenSSL), something like: ab -k -Z "AES128-GCM-SHA256" forces testing of
> > this common AES-GCM cipher. Newer and more secure ciphers are often way
> > slower, no surprise there.
>
> Good point. I'll double check that. What I was really after was a some
> numbers to back up a (probably over simplified) "drop in the native
> library and turo-charge your TLS performance" claim.
>
Ok ! I also started with "good enough" benchmarks initially which indicated
OpenSSL was worthwhile, then realized the results could be more accurate.
>
> > Last, APR is still significantly faster for me, which is rather normal.
> > It's not that critical at this performance level, probably, but it's here
> > to stay.
>
> An in depth comparison between the three options would be useful at some
> point.
>
> Have you done much performance tuning of NIO + OpenSSL?
>
The code looks ok to me, as long as direct buffers are used. It needs to
copy data to the OpenSSL BIO and back, that's the main cost besides
encryption and decryption, and I don't think this can be optimized. My
thinking is APR is faster since it skips these two copy operations.
However, the SSL behaviors simply destroys fancier IO APIs (the async
scatter/gather I wanted to introduce):
- JSSE only wants big buffers ...
- OpenSSL prefers direct buffers
So much flexibility ! :(
Rémy
Re: NIO + JSSE + NIO + OpenSSL
Posted by Rémy Maucherat <re...@apache.org>.
2016-02-15 19:47 GMT+01:00 Mark Thomas <ma...@apache.org>:
> I tested ECDHE-RSA-AES128-GCM-SHA256 which is Mozilla's recommended
> cipher for 'modern' httpd configurations.
>
> I saw a 20% improvement with NIO+OpenSSL compared to NIO+JSSE on windows.
>
> The figures are going to vary with OS, test client, target resource etc.
>
Yes, that's most likely the case, it also probably depends on the CPU. My
own table was made with: ab -k -Z "XXX" -c 100 -n 1000000
https://127.0.0.1:XXXX/tomcat.gif
For the fastest ciphers APR was ahead but not too much compared with
NIO+OpenSSL. So there shouldn't be any huge optimization issue.
> but it does seem reasonable to say that - in the general case -
> NIO+OpenSSL is a better choice than NIO+JSSE.
>
I guess if you want to try HTTP/2, it probably helps :)
>
> Thanks all. I think I have enough data to fill in the gaps in my
> presentation tomorrow.
>
> Rémy
Re: NIO + JSSE + NIO + OpenSSL
Posted by Mark Thomas <ma...@apache.org>.
On 15/02/2016 16:47, Rémy Maucherat wrote:
> 2016-02-15 14:57 GMT+01:00 jean-frederic clere <jf...@gmail.com>:
>
>> Using a cipher that allow HTTP/2 to work with the standard browsers
>> (like firefox and chrome) make sense otherwise we would be benching an
>> old "unsafe" cipher.
>>
>> I can't rerun my apache con tests right now but that time
>> AES128-GCM-SHA256 was the cipher I used.
>>
>> I had done some extensive (?) benchmarking 6 months ago (more or less),
> and things are quite different now, cool :)
>
> Looking at the cipher list from my OpenSSL (Fedora 23 OpenSSL), there are
> only 8 ciphers left for the cipher suite that Tomcat uses [and TLS 1.2 and
> a RSA certificate]. Half with DHE, half with ECDHE. ab refuses to connect
> to JSSE with ECDHE and AES 256. With AES 128, a recent JDK 8 worked, but
> not OpenJDK 8 from Fedora [which is unusable at the moment since browsers
> refuse to connect as well].
>
> So here's the result array (in k reqs/s):
> ___________________________________________ OpenSSL
> JSSE APR
> ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH 63 NA
> 67
> ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH____ 37 NA
> 37
> DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH____ 22 30
> 22
> DHE-RSA-AES256-SHA256 TLSv1.2 Kx=DH______ 20 28
> 20
> ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH 65 30
> 70
> ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH____ 45 29
> 45
> DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH____ 22 29
> 23
> DHE-RSA-AES128-SHA256 TLSv1.2 Kx=DH______ 20 28
> 20
>
> So OpenSSL is much faster for me for ECDHE, but not with DHE. Browsers use
> ECDHE.
I tested ECDHE-RSA-AES128-GCM-SHA256 which is Mozilla's recommended
cipher for 'modern' httpd configurations.
I saw a 20% improvement with NIO+OpenSSL compared to NIO+JSSE on windows.
The figures are going to vary with OS, test client, target resource etc.
but it does seem reasonable to say that - in the general case -
NIO+OpenSSL is a better choice than NIO+JSSE.
Thanks all. I think I have enough data to fill in the gaps in my
presentation tomorrow.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: NIO + JSSE + NIO + OpenSSL
Posted by Rémy Maucherat <re...@apache.org>.
2016-02-15 14:57 GMT+01:00 jean-frederic clere <jf...@gmail.com>:
> Using a cipher that allow HTTP/2 to work with the standard browsers
> (like firefox and chrome) make sense otherwise we would be benching an
> old "unsafe" cipher.
>
> I can't rerun my apache con tests right now but that time
> AES128-GCM-SHA256 was the cipher I used.
>
> I had done some extensive (?) benchmarking 6 months ago (more or less),
and things are quite different now, cool :)
Looking at the cipher list from my OpenSSL (Fedora 23 OpenSSL), there are
only 8 ciphers left for the cipher suite that Tomcat uses [and TLS 1.2 and
a RSA certificate]. Half with DHE, half with ECDHE. ab refuses to connect
to JSSE with ECDHE and AES 256. With AES 128, a recent JDK 8 worked, but
not OpenJDK 8 from Fedora [which is unusable at the moment since browsers
refuse to connect as well].
So here's the result array (in k reqs/s):
___________________________________________ OpenSSL
JSSE APR
ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH 63 NA
67
ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH____ 37 NA
37
DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH____ 22 30
22
DHE-RSA-AES256-SHA256 TLSv1.2 Kx=DH______ 20 28
20
ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH 65 30
70
ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH____ 45 29
45
DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH____ 22 29
23
DHE-RSA-AES128-SHA256 TLSv1.2 Kx=DH______ 20 28
20
So OpenSSL is much faster for me for ECDHE, but not with DHE. Browsers use
ECDHE.
Rémy
Re: NIO + JSSE + NIO + OpenSSL
Posted by jean-frederic clere <jf...@gmail.com>.
On 02/15/2016 11:30 AM, Rémy Maucherat wrote:
> 2016-02-15 10:45 GMT+01:00 Mark Thomas <ma...@apache.org>:
>
>> Looks like such a claim is indeed over simplified.
>>
>> Having tweaking the test so the same cipher is used, NIO+JSSE is about
>> 10% faster than NIO+OpenSSL :(
>>
>> Enabling direct buffers didn't seem to help.
>>
>
> Well, you're probably using an unoptimized cipher then. We'd need to
> determine the cipher(s) which should be tested.
>
> For example, with OpenSSL, ECDHE-RSA-AES256-GCM-SHA384 is fast (that's what
> ab connects with), while DHE-RSA-AES128-SHA256 is slow (ab connects to JSSE
> with that, so I forced it with -Z to compare, and OpenSSL is slightly
> slower than JSSE with it). With my OpenSSL, ECDHE-RSA-AES256-GCM-SHA384 is
> an order of magnitude faster than DHE-RSA-AES128-SHA256 (and it does sound
> more secure as well). JSSE doesn't have ECDHE-RSA-AES256-GCM-SHA384
> however. Also my Firefox 44 refuses to handshake with JSSE with its default
> configuration (there: ssl_error_no_cypher_overlap) :(
>
> So as I was saying, this SSL testing is hard ;)
Using a cipher that allow HTTP/2 to work with the standard browsers
(like firefox and chrome) make sense otherwise we would be benching an
old "unsafe" cipher.
I can't rerun my apache con tests right now but that time
AES128-GCM-SHA256 was the cipher I used.
Cheers
Jean-Frederic
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: NIO + JSSE + NIO + OpenSSL
Posted by Rémy Maucherat <re...@apache.org>.
2016-02-15 10:45 GMT+01:00 Mark Thomas <ma...@apache.org>:
> Looks like such a claim is indeed over simplified.
>
> Having tweaking the test so the same cipher is used, NIO+JSSE is about
> 10% faster than NIO+OpenSSL :(
>
> Enabling direct buffers didn't seem to help.
>
Well, you're probably using an unoptimized cipher then. We'd need to
determine the cipher(s) which should be tested.
For example, with OpenSSL, ECDHE-RSA-AES256-GCM-SHA384 is fast (that's what
ab connects with), while DHE-RSA-AES128-SHA256 is slow (ab connects to JSSE
with that, so I forced it with -Z to compare, and OpenSSL is slightly
slower than JSSE with it). With my OpenSSL, ECDHE-RSA-AES256-GCM-SHA384 is
an order of magnitude faster than DHE-RSA-AES128-SHA256 (and it does sound
more secure as well). JSSE doesn't have ECDHE-RSA-AES256-GCM-SHA384
however. Also my Firefox 44 refuses to handshake with JSSE with its default
configuration (there: ssl_error_no_cypher_overlap) :(
So as I was saying, this SSL testing is hard ;)
I believe OpenSSL is much faster in the real world though (and it
implements the latest security/features/etc).
>
> >> Last, APR is still significantly faster for me, which is rather normal.
> >> It's not that critical at this performance level, probably, but it's
> here
> >> to stay.
> >
> > An in depth comparison between the three options would be useful at some
> > point.
>
> This is definitely looking to be the case. I do wonder what role the
> size of the response plays. I was using very small response bodies in my
> test. If OpenSSL is faster at the encryption but comes with an overhead
> then at some point, as response size increases, it should become the
> better option.
>
I'm not convinced, I'm testing tomcat.gif so that should be small as well.
>
> > Have you done much performance tuning of NIO + OpenSSL?
>
> I'll probably take a quick look at this to see of there is anything
> obvious that can be done to improve things.
>
> No problem.
Rémy
Re: NIO + JSSE + NIO + OpenSSL
Posted by Mark Thomas <ma...@apache.org>.
On 15/02/2016 08:11, Mark Thomas wrote:
> On 14/02/2016 22:36, Rémy Maucherat wrote:
>> SSL tests are difficult however, what do you use ?
>
> The same test as I was using for JASPIC.
>
>> Direct buffers help
>> OpenSSL a lot for example (socket.directBuffer and socket.directSslBuffer
>> to true). Also one important item is to make sure the tests all use the
>> same cipher, especially with ab (JSSE might not use the same cipher as
>> OpenSSL), something like: ab -k -Z "AES128-GCM-SHA256" forces testing of
>> this common AES-GCM cipher. Newer and more secure ciphers are often way
>> slower, no surprise there.
>
> Good point. I'll double check that. What I was really after was a some
> numbers to back up a (probably over simplified) "drop in the native
> library and turo-charge your TLS performance" claim.
Looks like such a claim is indeed over simplified.
Having tweaking the test so the same cipher is used, NIO+JSSE is about
10% faster than NIO+OpenSSL :(
Enabling direct buffers didn't seem to help.
>> Last, APR is still significantly faster for me, which is rather normal.
>> It's not that critical at this performance level, probably, but it's here
>> to stay.
>
> An in depth comparison between the three options would be useful at some
> point.
This is definitely looking to be the case. I do wonder what role the
size of the response plays. I was using very small response bodies in my
test. If OpenSSL is faster at the encryption but comes with an overhead
then at some point, as response size increases, it should become the
better option.
> Have you done much performance tuning of NIO + OpenSSL?
I'll probably take a quick look at this to see of there is anything
obvious that can be done to improve things.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: NIO + JSSE + NIO + OpenSSL
Posted by Mark Thomas <ma...@apache.org>.
On 14/02/2016 22:36, Rémy Maucherat wrote:
> 2016-02-14 22:45 GMT+01:00 Mark Thomas <ma...@apache.org>:
>
>> All,
>>
>> In preparation for the connector selection webinar next week, I just did
>> a quick test of NIO + JSSE and NIO + OpenSSL.
>>
>> I was working with 9.0.x trunk including my JASPIC patch
>>
>> NIO + JSSE ~8200 requests/second
>>
>> Add the native lib to $CATALINA_BASE/bin and restart.
>> No other changes at all.
>>
>> NIO + OpenSSL ~12300 requests/second
>>
>>
>> Simply dropping in the native library improves TLS performance by
>> roughly 50%.
>>
>> Kudos to remm and jfclere.
>>
>
> Thanks !
>
> SSL tests are difficult however, what do you use ?
The same test as I was using for JASPIC.
> Direct buffers help
> OpenSSL a lot for example (socket.directBuffer and socket.directSslBuffer
> to true). Also one important item is to make sure the tests all use the
> same cipher, especially with ab (JSSE might not use the same cipher as
> OpenSSL), something like: ab -k -Z "AES128-GCM-SHA256" forces testing of
> this common AES-GCM cipher. Newer and more secure ciphers are often way
> slower, no surprise there.
Good point. I'll double check that. What I was really after was a some
numbers to back up a (probably over simplified) "drop in the native
library and turo-charge your TLS performance" claim.
> Last, APR is still significantly faster for me, which is rather normal.
> It's not that critical at this performance level, probably, but it's here
> to stay.
An in depth comparison between the three options would be useful at some
point.
Have you done much performance tuning of NIO + OpenSSL?
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: NIO + JSSE + NIO + OpenSSL
Posted by Christopher Schultz <ch...@christopherschultz.net>.
Rémy,
On 2/14/16 5:36 PM, Rémy Maucherat wrote:
> 2016-02-14 22:45 GMT+01:00 Mark Thomas <ma...@apache.org>:
>
>> All,
>>
>> In preparation for the connector selection webinar next week, I just did
>> a quick test of NIO + JSSE and NIO + OpenSSL.
>>
>> I was working with 9.0.x trunk including my JASPIC patch
>>
>> NIO + JSSE ~8200 requests/second
>>
>> Add the native lib to $CATALINA_BASE/bin and restart.
>> No other changes at all.
>>
>> NIO + OpenSSL ~12300 requests/second
>>
>>
>> Simply dropping in the native library improves TLS performance by
>> roughly 50%.
>>
>> Kudos to remm and jfclere.
>>
>
> Thanks !
>
> SSL tests are difficult however, what do you use ? Direct buffers help
> OpenSSL a lot for example (socket.directBuffer and socket.directSslBuffer
> to true). Also one important item is to make sure the tests all use the
> same cipher, especially with ab (JSSE might not use the same cipher as
> OpenSSL), something like: ab -k -Z "AES128-GCM-SHA256" forces testing of
> this common AES-GCM cipher. Newer and more secure ciphers are often way
> slower, no surprise there.
+1
Make sure that the same cipher suite(s) are in use for these tests.
> Last, APR is still significantly faster for me, which is rather normal.
Agreed. JSSE adds-in a bunch of API layers whereas the APR connector is
almost straight-to-OpenSSL. I think it will always have a performance
advantage. (Especially because OpenSSL itself is finally slimming-down
after decades of cruft.)
> It's not that critical at this performance level, probably, but it's here
> to stay.
It just needs a good bit of real-world testing to make sure we have all
the corner-cases handled.
-chris
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: NIO + JSSE + NIO + OpenSSL
Posted by Rémy Maucherat <re...@apache.org>.
2016-02-14 22:45 GMT+01:00 Mark Thomas <ma...@apache.org>:
> All,
>
> In preparation for the connector selection webinar next week, I just did
> a quick test of NIO + JSSE and NIO + OpenSSL.
>
> I was working with 9.0.x trunk including my JASPIC patch
>
> NIO + JSSE ~8200 requests/second
>
> Add the native lib to $CATALINA_BASE/bin and restart.
> No other changes at all.
>
> NIO + OpenSSL ~12300 requests/second
>
>
> Simply dropping in the native library improves TLS performance by
> roughly 50%.
>
> Kudos to remm and jfclere.
>
Thanks !
SSL tests are difficult however, what do you use ? Direct buffers help
OpenSSL a lot for example (socket.directBuffer and socket.directSslBuffer
to true). Also one important item is to make sure the tests all use the
same cipher, especially with ab (JSSE might not use the same cipher as
OpenSSL), something like: ab -k -Z "AES128-GCM-SHA256" forces testing of
this common AES-GCM cipher. Newer and more secure ciphers are often way
slower, no surprise there.
Last, APR is still significantly faster for me, which is rather normal.
It's not that critical at this performance level, probably, but it's here
to stay.
Rémy