You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Jeff Trawick <tr...@gmail.com> on 2014/02/21 14:02:31 UTC

Fwd: [users@httpd] Performance drop in 2.4.7 versus 2.4.6

Including dev@httpd.apache.org...

---------- Forwarded message ----------
From: Cedric Roijakkers <Ce...@qnh.nl>
Date: Fri, Feb 21, 2014 at 7:44 AM
Subject: [users@httpd] Performance drop in 2.4.7 versus 2.4.6
To: "users@httpd.apache.org" <us...@httpd.apache.org>


 Hi All,



As most of you, we're running Apache in a production set-up for a rather
large website. Happily, for a long time might I even add.



Since business focus these days is on having a low as possible page load
time, I was shocked when we upgraded to version 2.4.7, and our monitoring
system started indicating that the time needed to serve pages has easily
quadrupled... As soon as we noticed this, we rolled back, and tried to figure
out what was going on. Searching the internet and this list did not result
in any hits, so I might as well post my question here.



Here is the situation:



We host a website, with a lot of classic html files, server-side includes
and a backend which is running on a Tomcat server which is proxied via
mod_proxy_http via Apache. The site is full HTTPS, and anybody coming in
via HTTP will be redirected to HTTPS. The server OS is Red Hat Enterprise
Linux 5.9, OpenSSL is 1.0.1f. Both versions of Apache (2.4.6 and 2.4.7) are
compiled with the same parameters, on the same machine.



For the timings, I'm using check_http from Nagios, which prints the time to
load the html page in milliseconds.



Here are the results when running on 2.4.6:



HTTP OK: Status line output matched "200" - 19091 bytes in 0.023 second
response time |time=0.022968s;2.000000;3.000000;0.000000 size=19091B;;;0

HTTP OK: Status line output matched "200" - 19091 bytes in 0.022 second
response time |time=0.022035s;2.000000;3.000000;0.000000 size=19091B;;;0

HTTP OK: Status line output matched "200" - 19091 bytes in 0.022 second
response time |time=0.021548s;2.000000;3.000000;0.000000 size=19091B;;;0

HTTP OK: Status line output matched "200" - 19091 bytes in 0.021 second
response time |time=0.021477s;2.000000;3.000000;0.000000 size=19091B;;;0

HTTP OK: Status line output matched "200" - 19091 bytes in 0.021 second
response time |time=0.021362s;2.000000;3.000000;0.000000 size=19091B;;;0

HTTP OK: Status line output matched "200" - 19091 bytes in 0.021 second
response time |time=0.021429s;2.000000;3.000000;0.000000 size=19091B;;;0

HTTP OK: Status line output matched "200" - 19091 bytes in 0.022 second
response time |time=0.022411s;2.000000;3.000000;0.000000 size=19091B;;;0

HTTP OK: Status line output matched "200" - 19091 bytes in 0.021 second
response time |time=0.021276s;2.000000;3.000000;0.000000 size=19091B;;;0

HTTP OK: Status line output matched "200" - 19091 bytes in 0.022 second
response time |time=0.021756s;2.000000;3.000000;0.000000 size=19091B;;;0



And here the results with 2.4.7:



HTTP OK: Status line output matched "200" - 19091 bytes in 0.094 second
response time |time=0.094451s;2.000000;3.000000;0.000000 size=19091B;;;0

HTTP OK: Status line output matched "200" - 19091 bytes in 0.085 second
response time |time=0.084865s;2.000000;3.000000;0.000000 size=19091B;;;0

HTTP OK: Status line output matched "200" - 19091 bytes in 0.085 second
response time |time=0.085055s;2.000000;3.000000;0.000000 size=19091B;;;0

HTTP OK: Status line output matched "200" - 19091 bytes in 0.086 second
response time |time=0.086268s;2.000000;3.000000;0.000000 size=19091B;;;0

HTTP OK: Status line output matched "200" - 19091 bytes in 0.086 second
response time |time=0.085889s;2.000000;3.000000;0.000000 size=19091B;;;0

HTTP OK: Status line output matched "200" - 19091 bytes in 0.096 second
response time |time=0.095538s;2.000000;3.000000;0.000000 size=19091B;;;0

HTTP OK: Status line output matched "200" - 19091 bytes in 0.086 second
response time |time=0.086159s;2.000000;3.000000;0.000000 size=19091B;;;0

HTTP OK: Status line output matched "200" - 19091 bytes in 0.086 second
response time |time=0.085750s;2.000000;3.000000;0.000000 size=19091B;;;0

HTTP OK: Status line output matched "200" - 19091 bytes in 0.086 second
response time |time=0.085835s;2.000000;3.000000;0.000000 size=19091B;;;0



Page load went from 0.02 seconds to 0.09. DocumentRoot, configuration, used
OpenSSL library are 100% identical in both cases. All I changed is the link
to the installation of Apache.



Is anybody else seeing the same behavior? Looking at the documentation,
2.4.7 has gained some performance improvements, but I'm seeing something
different on my end.



Greetings

Cedric







-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [users@httpd] Performance drop in 2.4.7 versus 2.4.6

Posted by Jim Jagielski <ji...@jaguNET.com>.
It would be interesting to see if removing SSL entirely
made any difference to the delta between 2.4.6 and 2.4.7...

On Feb 21, 2014, at 8:02 AM, Jeff Trawick <tr...@gmail.com> wrote:

> Including dev@httpd.apache.org...
> 
> ---------- Forwarded message ----------
> From: Cedric Roijakkers <Ce...@qnh.nl>
> Date: Fri, Feb 21, 2014 at 7:44 AM
> Subject: [users@httpd] Performance drop in 2.4.7 versus 2.4.6
> To: "users@httpd.apache.org" <us...@httpd.apache.org>
> 
> 
> Hi All,
> 
>  
> 
> As most of you, we’re running Apache in a production set-up for a rather large website. Happily, for a long time might I even add.
> 
>  
> 
> Since business focus these days is on having a low as possible page load time, I was shocked when we upgraded to version 2.4.7, and our monitoring system started indicating that the time needed to serve pages has easily quadrupled… As soon as we noticed this, we rolled back, and tried to figure out what was going on. Searching the internet and this list did not result in any hits, so I might as well post my question here.
> 
>  
> 
> Here is the situation:
> 
>  
> 
> We host a website, with a lot of classic html files, server-side includes and a backend which is running on a Tomcat server which is proxied via mod_proxy_http via Apache. The site is full HTTPS, and anybody coming in via HTTP will be redirected to HTTPS. The server OS is Red Hat Enterprise Linux 5.9, OpenSSL is 1.0.1f. Both versions of Apache (2.4.6 and 2.4.7) are compiled with the same parameters, on the same machine.
> 
>  
> 
> For the timings, I’m using check_http from Nagios, which prints the time to load the html page in milliseconds.
> 
>  
> 
> Here are the results when running on 2.4.6:
> 
>  
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.023 second response time |time=0.022968s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.022 second response time |time=0.022035s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.022 second response time |time=0.021548s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.021 second response time |time=0.021477s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.021 second response time |time=0.021362s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.021 second response time |time=0.021429s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.022 second response time |time=0.022411s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.021 second response time |time=0.021276s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.022 second response time |time=0.021756s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
>  
> 
> And here the results with 2.4.7:
> 
>  
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.094 second response time |time=0.094451s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.085 second response time |time=0.084865s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.085 second response time |time=0.085055s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.086 second response time |time=0.086268s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.086 second response time |time=0.085889s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.096 second response time |time=0.095538s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.086 second response time |time=0.086159s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.086 second response time |time=0.085750s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
> HTTP OK: Status line output matched "200" - 19091 bytes in 0.086 second response time |time=0.085835s;2.000000;3.000000;0.000000 size=19091B;;;0
> 
>  
> 
> Page load went from 0.02 seconds to 0.09. DocumentRoot, configuration, used OpenSSL library are 100% identical in both cases. All I changed is the link to the installation of Apache.
> 
>  
> 
> Is anybody else seeing the same behavior? Looking at the documentation, 2.4.7 has gained some performance improvements, but I’m seeing something different on my end.
> 
>  
> 
> Greetings
> 
> Cedric
> 
>  
> 
>  
> 
> 
> 
> 
> -- 
> Born in Roswell... married an alien...
> http://emptyhammock.com/


Re: Fwd: [users@httpd] Performance drop in 2.4.7 versus 2.4.6

Posted by Dr Stephen Henson <sh...@opensslfoundation.com>.
On 21/02/2014 13:13, Dr Stephen Henson wrote:
> On 21/02/2014 13:02, Jeff Trawick wrote:
>> Including dev@httpd.apache.org...
>>
>> Is anybody else seeing the same behavior? Looking at the documentation, 2.4.7
>> has gained some performance improvements, but I’m seeing something different on
>> my end.____
>>
> 
> Perhaps it's the increased DH parameter size? If it has increased from 1024 bits
> to 2048 that would have a significant effect.
> 
> OpenSSL 1.0.2 s_client can help check this, if you do:
> 
> openssl s_client -connect www.host.com:443
> 

Correction, since the site is using OpenSSL 1.0.1f that command by itself may
not be sufficient as ECDH is most likely to be used by default so it would need:

openssl s_client -connect www.host.com:443 -cipher DH

Steve.
-- 
Dr Stephen Henson. OpenSSL Software Foundation, Inc.
1829 Mount Ephraim Road
Adamstown, MD 21710
+1 877-673-6775
shenson@opensslfoundation.com

Re: [users@httpd] Performance drop in 2.4.7 versus 2.4.6

Posted by Reindl Harald <h....@thelounge.net>.
Am 21.02.2014 14:13, schrieb Dr Stephen Henson:
> On 21/02/2014 13:02, Jeff Trawick wrote:
>> Including dev@httpd.apache.org...
>>
>> Is anybody else seeing the same behavior? Looking at the documentation, 2.4.7
>> has gained some performance improvements, but I’m seeing something different on
>> my end
> 
> Perhaps it's the increased DH parameter size? If it has increased from 1024 bits
> to 2048 that would have a significant effect.
> 
> OpenSSL 1.0.2 s_client can help check this, if you do:
> 
> openssl s_client -connect www.host.com:443
> 
> it says (among lots of other stuff):
> Server Temp Key: DH, xxxx bits

most likely, on the other hand clients using DHE instead ECDHE
are rare these days, the others are using no PFS at all

HTTP OK: Status line output matched "200" - 19091 bytes in 0.023 second response time
|time=0.022968s;2.000000;3.000000;0.000000 size=19091B;;;0

is in any case not that fast since i see response times around 0.003 to 0.010 seconds
on CMS page with db-backends, below a https-ab-benchmark on a statical image with
a 4096 Bit RSA which is most likely larger than 99% out there

Server Hostname:        secure.thelounge.net
Server Port:            443
SSL/TLS Protocol:       TLSv1.2,ECDHE-RSA-AES128-GCM-SHA256,4096,128
Document Path:          **************************************
Document Length:        11019 bytes
Concurrency Level:      100
Time taken for tests:   39.741 seconds
Complete requests:      10000
Failed requests:        0
Total transferred:      113840000 bytes
HTML transferred:       110190000 bytes
Requests per second:    251.63 [#/sec] (mean)
Time per request:       397.413 [ms] (mean)
Time per request:       3.974 [ms] (mean, across all concurrent requests)
Transfer rate:          2797.39 [Kbytes/sec] received






Re: Fwd: [users@httpd] Performance drop in 2.4.7 versus 2.4.6

Posted by Dr Stephen Henson <sh...@opensslfoundation.com>.
On 21/02/2014 13:02, Jeff Trawick wrote:
> Including dev@httpd.apache.org...
> 
> Is anybody else seeing the same behavior? Looking at the documentation, 2.4.7
> has gained some performance improvements, but I’m seeing something different on
> my end.____
> 

Perhaps it's the increased DH parameter size? If it has increased from 1024 bits
to 2048 that would have a significant effect.

OpenSSL 1.0.2 s_client can help check this, if you do:

openssl s_client -connect www.host.com:443

it says (among lots of other stuff):

Server Temp Key: DH, xxxx bits

Steve.
-- 
Dr Stephen Henson. OpenSSL Software Foundation, Inc.
1829 Mount Ephraim Road
Adamstown, MD 21710
+1 877-673-6775
shenson@opensslfoundation.com