You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Ian Holsman <ia...@apache.org> on 2001/12/21 23:15:37 UTC
Benchmark: 29 vs 30
I finally got our load-generating machine to work properly.
Here are some benchmarks comparing v29 with v30 of apache
http://webperf.org/a2/caw/29/caw.html
http://webperf.org/a2/caw/30/caw.html
The test consists of the test grabbing a random (but identical) outer
page which include the same subcomponents, the number of requests slowly
over time to ~1,000 open sessions.
I think this shows that the new pools (really the only major change
between 29 & 30 ) has a significant impact on stability under high load
Ian
Re: Benchmark: 29 vs 30
Posted by Rodent of Unusual Size <Ke...@Golux.Com>.
Aaron Bannert wrote:
>
> Having some problems with the images, I think the spaces are freaking
> out my Linux/Netscape browser.
Indeed. Embedded spaces are illegal; the page that links to
the images needs to use encoding like 'Transport%20Error%20Analysis.gif'.
Curiously enough, '%20' encoding of spaces works, but '+'
encoding doesn't. I thought that historically '+' was a
shorthand for '%20', though 2396 doesn't acknowledge this;
1630 does, but only applies it to the query-string
component of the URI.
Maybe Roy can clear up my fuzzy memory on this..?
Any road, those embedded spaces in the links need to be
encoded.
--
#ken P-)}
Ken Coar, Sanagendamgagwedweinini http://Golux.Com/coar/
Author, developer, opinionist http://Apache-Server.Com/
"All right everyone! Step away from the glowing hamburger!"
Re: Benchmark: 29 vs 30
Posted by Aaron Bannert <aa...@clove.org>.
On Fri, Dec 21, 2001 at 02:15:37PM -0800, Ian Holsman wrote:
> I finally got our load-generating machine to work properly.
>
> Here are some benchmarks comparing v29 with v30 of apache
>
> http://webperf.org/a2/caw/29/caw.html
> http://webperf.org/a2/caw/30/caw.html
Having some problems with the images, I think the spaces are freaking
out my Linux/Netscape browser.
-aaron
Re: Benchmark: 29 vs 30
Posted by Ian Holsman <ia...@apache.org>.
Aaron Bannert wrote:
> On Sat, Dec 22, 2001 at 02:02:53PM +0100, Sander Striker wrote:
>
>>>From: David Reid [mailto:dreid@jetnet.co.uk]
>>>Sent: 22 December 2001 13:57
>>>
>>>The transaction stats were what jumped out at me - 7% increase in failed
>>>connections doesn't sound good to me :( But, then maybe I'm reading that
>>>wrong?
>>>
>>Which is what I saw first too. But when I talked to Ian over irc <snippet>:
>>
>>[00:22] <IanHolsman> hey.. did you see the benchmark..
>>[00:22] <IanHolsman> could you parse it
>>[00:22] <sander> Yes, saw it.
>>[00:22] <sander> No, could not parse.
>>[00:22] <IanHolsman> aah
>>[00:23] <IanHolsman> http://webperf.org/a2/caw/29/Current Total HTTP and TCP Errors vs Load 21-Dec-2001 1215.gif is probably the
>>best image
>>[00:24] <sander> Under a higher load we get more errors with v30?
>>[00:24] <IanHolsman> if you look on v29 errors start happening around 500 users sessions. with v30 they happen at 700
>>[00:24] <IanHolsman> no... under a higher load you get less.
>>[00:24] <sander> Ah
>>
>
> How are "Unsuccessful" transactions defined? Failed connect()ions, incomplete
> HTTP requests? Invalid data returned?
>
> -aaron
>
>
I think I forgot to mention that there was a core dump in .30.
as far as what a incomplete transaction is, I don't know. (I'll dig up
the books)
the test was a page fetch of a random page which had 7-10 includes. The
# of sessions was increased every minute so that by the end of the test
we had 1,000 open connections. we were getting ~100 connection/second
for a lot of the time, but the response time & failure rate was getting
worse as the number of sessions increased. This is probably better seen
if I expand the CPU load RRD graph so it is wider and only for the first
15 minutes.
both v30 & v29 couldn't handle the load at the end of the test, the
major difference IMHO was v29 bailed out much sooner than v30
v30 also had a core dump in a weird location, which points to a memory
corruption
http://webperf.org/a2/caw/cpu_comp.html
shows a comparision of server load, and transactions per second.
the chart is a bit funky underneath, but what it shows is that v30
experienced a much lower error rate and higher TPS than v29 did.
The next set of tests I do I will vary the Tranasactions per second.
as I find that easier to understand and explain to people.
Re: Benchmark: 29 vs 30
Posted by Aaron Bannert <aa...@clove.org>.
On Sat, Dec 22, 2001 at 02:02:53PM +0100, Sander Striker wrote:
> > From: David Reid [mailto:dreid@jetnet.co.uk]
> > Sent: 22 December 2001 13:57
>
> > The transaction stats were what jumped out at me - 7% increase in failed
> > connections doesn't sound good to me :( But, then maybe I'm reading that
> > wrong?
>
> Which is what I saw first too. But when I talked to Ian over irc <snippet>:
>
> [00:22] <IanHolsman> hey.. did you see the benchmark..
> [00:22] <IanHolsman> could you parse it
> [00:22] <sander> Yes, saw it.
> [00:22] <sander> No, could not parse.
> [00:22] <IanHolsman> aah
> [00:23] <IanHolsman> http://webperf.org/a2/caw/29/Current Total HTTP and TCP Errors vs Load 21-Dec-2001 1215.gif is probably the
> best image
> [00:24] <sander> Under a higher load we get more errors with v30?
> [00:24] <IanHolsman> if you look on v29 errors start happening around 500 users sessions. with v30 they happen at 700
> [00:24] <IanHolsman> no... under a higher load you get less.
> [00:24] <sander> Ah
How are "Unsuccessful" transactions defined? Failed connect()ions, incomplete
HTTP requests? Invalid data returned?
-aaron
Re: Benchmark: 29 vs 30
Posted by David Reid <dr...@jetnet.co.uk>.
Hmm, OK but the headline figures really do look negative. The jump for page
transaction unsucceful from 20.51% to 34.7%, even if the errors start later,
doesn't really bode well does it?
I assume the test was the same number of requests and so on, so I'd consider
a reduction in the overall succesful number of completions to be a bad
thing, not a good thing??
david
----- Original Message -----
From: "Sander Striker" <st...@apache.org>
To: <de...@httpd.apache.org>
Sent: Saturday, December 22, 2001 1:02 PM
Subject: RE: Benchmark: 29 vs 30
> > From: David Reid [mailto:dreid@jetnet.co.uk]
> > Sent: 22 December 2001 13:57
>
> > The transaction stats were what jumped out at me - 7% increase in failed
> > connections doesn't sound good to me :( But, then maybe I'm reading
that
> > wrong?
>
> Which is what I saw first too. But when I talked to Ian over irc
<snippet>:
>
> [00:22] <IanHolsman> hey.. did you see the benchmark..
> [00:22] <IanHolsman> could you parse it
> [00:22] <sander> Yes, saw it.
> [00:22] <sander> No, could not parse.
> [00:22] <IanHolsman> aah
> [00:23] <IanHolsman> http://webperf.org/a2/caw/29/Current Total HTTP and
TCP Errors vs Load 21-Dec-2001 1215.gif is probably the
> best image
> [00:24] <sander> Under a higher load we get more errors with v30?
> [00:24] <IanHolsman> if you look on v29 errors start happening around 500
users sessions. with v30 they happen at 700
> [00:24] <IanHolsman> no... under a higher load you get less.
> [00:24] <sander> Ah
>
> And this is why I thought this morning: "There must be more people
> having trouble parsing these results...".
>
> > david
>
> Sander
>
>
RE: Benchmark: 29 vs 30
Posted by Sander Striker <st...@apache.org>.
> From: David Reid [mailto:dreid@jetnet.co.uk]
> Sent: 22 December 2001 13:57
> The transaction stats were what jumped out at me - 7% increase in failed
> connections doesn't sound good to me :( But, then maybe I'm reading that
> wrong?
Which is what I saw first too. But when I talked to Ian over irc <snippet>:
[00:22] <IanHolsman> hey.. did you see the benchmark..
[00:22] <IanHolsman> could you parse it
[00:22] <sander> Yes, saw it.
[00:22] <sander> No, could not parse.
[00:22] <IanHolsman> aah
[00:23] <IanHolsman> http://webperf.org/a2/caw/29/Current Total HTTP and TCP Errors vs Load 21-Dec-2001 1215.gif is probably the
best image
[00:24] <sander> Under a higher load we get more errors with v30?
[00:24] <IanHolsman> if you look on v29 errors start happening around 500 users sessions. with v30 they happen at 700
[00:24] <IanHolsman> no... under a higher load you get less.
[00:24] <sander> Ah
And this is why I thought this morning: "There must be more people
having trouble parsing these results...".
> david
Sander
Re: Benchmark: 29 vs 30
Posted by David Reid <dr...@jetnet.co.uk>.
The transaction stats were what jumped out at me - 7% increase in failed
connections doesn't sound good to me :( But, then maybe I'm reading that
wrong?
david
> "Sander Striker" <st...@apache.org> writes:
>
> > Ian, could you please explain in words what these results mean?
>
> okay, maybe I won't feel quite so stupid now :)
>
> --
> Jeff Trawick | trawick@attglobal.net | PGP public key at web site:
> http://www.geocities.com/SiliconValley/Park/9289/
> Born in Roswell... married an alien...
>
Re: Benchmark: 29 vs 30
Posted by Jeff Trawick <tr...@attglobal.net>.
"Sander Striker" <st...@apache.org> writes:
> Ian, could you please explain in words what these results mean?
okay, maybe I won't feel quite so stupid now :)
--
Jeff Trawick | trawick@attglobal.net | PGP public key at web site:
http://www.geocities.com/SiliconValley/Park/9289/
Born in Roswell... married an alien...
RE: Benchmark: 29 vs 30
Posted by Sander Striker <st...@apache.org>.
> From: Ian Holsman [mailto:ianh@apache.org]
> Sent: 21 December 2001 23:16
Hi,
> I finally got our load-generating machine to work properly.
>
> Here are some benchmarks comparing v29 with v30 of apache
>
> http://webperf.org/a2/caw/29/caw.html
> http://webperf.org/a2/caw/30/caw.html
Thanks!
> The test consists of the test grabbing a random (but identical) outer
> page which include the same subcomponents, the number of requests slowly
> over time to ~1,000 open sessions.
>
> I think this shows that the new pools (really the only major change
> between 29 & 30 ) has a significant impact on stability under high load
Ian, could you please explain in words what these results mean?
I am sure there are more people like me who just misinterpret the
numbers, which is not what we are looking for ;)
> Ian
Sander