You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Joshua Chamas <jo...@chamas.com> on 2000/12/11 11:13:35 UTC
Linux Hello World Benchmarks...
Hey,
I have automated a portable Hello World test suite, but its not
CPAN ready, so if any would like to contribute, run, and comment
on the sources, give me a holler & I'll send them to you. What
it does is fire up a lean apache on a high port with only the
config necessary to run the benchmark.
RESULTS:
[hello]# ./bench.pl -time=60
...
Test Name Test File Hits/sec Total Hits Total Time Total Bytes
------------ ------------ ------------ ------------ ------------ ------------
Apache::ASP hello.asp 416.9 25016 hits 60.00 sec 4478401 byte
Apache::Registry CGI Raw hello_raw.re 736.9 44214 hits 60.00 sec 2299284 byte
Apache::Registry CGI.pm hello.reg 501.3 30080 hits 60.00 sec 4632628 byte
HTML Static hello.html 1211.6 50000 hits 41.27 sec 12450249 byt
HTML::Embperl hello.epl 510.7 30647 hits 60.01 sec 4843016 byte
HTML::Mason hello.mas 382.2 22934 hits 60.00 sec 3086640 byte
ModPerl Handler hello.bench 888.9 50000 hits 56.25 sec 6700268 byte
Template Toolkit hello.tt 181.6 10895 hits 60.01 sec 1481992 byte
**AxKit --> need someone to run benchmark suite with this working
Apache::ASP v2.07
HTML::Embperl 1.3.0
HTML::Mason 0.89
Template Toolkit 2.0
mod_perl 1.24
apache 1.3.14
linux 2.2.14
PIII 450x2
localhost interface
Note, this is the first benchmark that I've run of Apache::ASP on
Linux, which is nice to see because Linux is one of the faster OS's,
and it now looks bit more of a player, compared to what's listed at
http://www.chamas.com/bench/ when I benched it on Solaris x86.
Also, Mason seems to have improved significantly since I last
benched it. Good job Jonathan.
Lastly, I was unable to get AxKit to run without segfaulting ...
I had wanted to get at least a XPathScript benchmark, and maybe
Sablotron driven XSLT. The simple XSLT hello world is at
http://www.chamas.com/bench/#xml. The XPathScript HelloWorld
that I came up with was: <% print "Hello "; %>World ... but
maybe it should have been more like the XSLT hello world?
-- Josh
_________________________________________________________________
Joshua Chamas Chamas Enterprises Inc.
NodeWorks >> free web link monitoring Huntington Beach, CA USA
http://www.nodeworks.com 1-714-625-4051
Re: Linux Hello World Benchmarks...
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Joshua Chamas <jo...@chamas.com> writes:
> Joe Schaefer wrote:
> >
> > IME, simple mod_perl handlers typically run around 50% as fast as
> > HTML static pages. Your hello world benchmark seems to be slightly
> > misleading in this respect, since the content-length is small
> > relative to the header size.
> >
>
> I'll send you my benchmark suite separately so you can
> submit your results for http://www.chamas.com/bench. I have
> never seen modperl handler faster than static HTML.
Me neither - what I said was that modperl handlers are about twice
as slow as static pages. That's what I meant, but I guess didn't
make it clear.
Best.
--
Joe Schaefer
Re: Linux Hello World Benchmarks...
Posted by Joshua Chamas <jo...@chamas.com>.
Joe Schaefer wrote:
>
> IME, simple mod_perl handlers typically run around 50% as fast as
> HTML static pages. Your hello world benchmark seems to be slightly
> misleading in this respect, since the content-length is small
> relative to the header size.
>
I'll send you my benchmark suite separately so you can
submit your results for http://www.chamas.com/bench. I have
never seen modperl handler faster than static HTML.
> For your HTML Static, I would guess that the headers delivered are
> significantly larger than the ones returned by your modperl handler.
> Hence for 50000 hits, there is a significant discrepancy in the
> total bytes delivered. This skews the hits/sec numbers in the
> favor of content handlers that deliver shorter headers.
>
Yes, each environment delivers different headers, but that's
just what they do. The Hello World test is to show the
overhead of running in a particular environment. Generally
people only add code from there, and not optimize HelloWorld
further.
-- Josh
_________________________________________________________________
Joshua Chamas Chamas Enterprises Inc.
NodeWorks >> free web link monitoring Huntington Beach, CA USA
http://www.nodeworks.com 1-714-625-4051
Re: Linux Hello World Benchmarks...
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Joshua Chamas <jo...@chamas.com> writes:
> RESULTS:
>
> [hello]# ./bench.pl -time=60
> ...
> Test Name Test File Hits/sec Total Hits Total Time Total Bytes
> ------------ ------------ ------------ ------------ ------------ ------------
> HTML Static hello.html 1211.6 50000 hits 41.27 sec 12450249 byt
> ModPerl Handler hello.bench 888.9 50000 hits 56.25 sec 6700268 byte
[...]
IME, simple mod_perl handlers typically run around 50% as fast as
HTML static pages. Your hello world benchmark seems to be slightly
misleading in this respect, since the content-length is small
relative to the header size.
For your HTML Static, I would guess that the headers delivered are
significantly larger than the ones returned by your modperl handler.
Hence for 50000 hits, there is a significant discrepancy in the
total bytes delivered. This skews the hits/sec numbers in the
favor of content handlers that deliver shorter headers.
To use HTML Static as a baseline comparison for dynamic content
handlers, I think you should ensure that the headers delivered
are comparable to those that are dynamically generated.
Best.
--
Joe Schaefer
Re: Linux Hello World Benchmarks...
Posted by Joshua Chamas <jo...@chamas.com>.
"Alexander Farber (EED)" wrote:
>
> Hi Joshua,
>
> you sort the table at http://www.chamas.com/bench/ by Hits/s,
> but the ModPerl Handler was tested on PIII-500 x 2 and the Java
> thingies below - only PII-266.
>
> Is it an intended joke or do I misunderstand something?
>
The first page is marked "for fun" because there are no relevant
sortings that you can do between systems & benchmarkers, as there
are too many variables. Its sorted by speed to encourage people
to submit new results with their shiny new GHz CPUs.
You can try to derive some real meaning from tests done on
the same system by the same person using the same methodology ...
for these go to
http://www.chamas.com/bench/hello_bysystem.html
--Josh
_________________________________________________________________
Joshua Chamas Chamas Enterprises Inc.
NodeWorks >> free web link monitoring Huntington Beach, CA USA
http://www.nodeworks.com 1-714-625-4051
Re: Linux Hello World Benchmarks...
Posted by "Alexander Farber (EED)" <ee...@eed.ericsson.se>.
Hi Joshua,
Joshua Chamas wrote:
> Note, this is the first benchmark that I've run of Apache::ASP on
> Linux, which is nice to see because Linux is one of the faster OS's,
> and it now looks bit more of a player, compared to what's listed at
> http://www.chamas.com/bench/ when I benched it on Solaris x86.
you sort the table at http://www.chamas.com/bench/ by Hits/s,
but the ModPerl Handler was tested on PIII-500 x 2 and the Java
thingies below - only PII-266.
Is it an intended joke or do I misunderstand something?
Regards
Alex
Re: Linux Hello World Benchmarks...
Posted by Joshua Chamas <jo...@chamas.com>.
Gunther Birznieks wrote:
>
> Then it seems odd that there is such a huge discrepency between CGI.pm and
> no CGI.pm. If you preload CGI.pm in startup.pl does the difference go away?
>
I did preload CGI.pm. I'll send you the hello world
suite separately since you seem curious. Note that at 500
requests / second, the difference of 50 requests / sec is
really 1/500 - 1/550 = .00018 CPU seconds. The point is
that as these systems get more efficient, minor differences
in performance seem to make more of a difference.
The same .0018 CPU sec at 100 hits/sec would only make 102
hits / sec.
--Josh
Re: Linux Hello World Benchmarks...
Posted by Gunther Birznieks <gu...@extropia.com>.
Then it seems odd that there is such a huge discrepency between CGI.pm and
no CGI.pm. If you preload CGI.pm in startup.pl does the difference go away?
At 02:56 AM 12/11/2000 -0800, Joshua Chamas wrote:
>Gunther Birznieks wrote:
> >
> > Is CGI Raw decoding the get/post yourself? Or using the Apache::args,
> > Apache::Request::param mechanism?
> >
>
>In the hello world scripts, there is no get/post processing as
>part of the benchmark. Here's the code that's run:
>
> http://www.chamas.com/bench/#perlrawcgi
>
>The general language scripts are at http://www.chamas.com/bench/
>and are also in the hello.tar.gz bundle that I could send.
>The bundle also has the app configs in bench.pl. Its a whole 8K
>gzipped though, and I'm afraid Ask would hurt me if I posted it ;)
>
>-- Josh
>
>_________________________________________________________________
>Joshua Chamas Chamas Enterprises Inc.
>NodeWorks >> free web link monitoring Huntington Beach, CA USA
>http://www.nodeworks.com 1-714-625-4051
__________________________________________________
Gunther Birznieks (gunther.birznieks@extropia.com)
eXtropia - The Web Technology Company
http://www.extropia.com/
Re: Linux Hello World Benchmarks...
Posted by Joshua Chamas <jo...@chamas.com>.
Gunther Birznieks wrote:
>
> Is CGI Raw decoding the get/post yourself? Or using the Apache::args,
> Apache::Request::param mechanism?
>
In the hello world scripts, there is no get/post processing as
part of the benchmark. Here's the code that's run:
http://www.chamas.com/bench/#perlrawcgi
The general language scripts are at http://www.chamas.com/bench/
and are also in the hello.tar.gz bundle that I could send.
The bundle also has the app configs in bench.pl. Its a whole 8K
gzipped though, and I'm afraid Ask would hurt me if I posted it ;)
-- Josh
_________________________________________________________________
Joshua Chamas Chamas Enterprises Inc.
NodeWorks >> free web link monitoring Huntington Beach, CA USA
http://www.nodeworks.com 1-714-625-4051
Re: Linux Hello World Benchmarks...
Posted by Gunther Birznieks <gu...@extropia.com>.
Is CGI Raw decoding the get/post yourself? Or using the Apache::args,
Apache::Request::param mechanism?
At 02:13 AM 12/11/2000 -0800, Joshua Chamas wrote:
>Hey,
>
>I have automated a portable Hello World test suite, but its not
>CPAN ready, so if any would like to contribute, run, and comment
>on the sources, give me a holler & I'll send them to you. What
>it does is fire up a lean apache on a high port with only the
>config necessary to run the benchmark.
Re: Linux Hello World Benchmarks...
Posted by Matt Sergeant <ma...@sergeant.org>.
On Mon, 11 Dec 2000, Joshua Chamas wrote:
> Lastly, I was unable to get AxKit to run without segfaulting ...
http://axkit.org/faq.xml
Either you're running PHP on that server, or you have an Apache with expat
included. Do "nm /path/to/apache/bin/httpd | grep -i XML" to find out if
the latter is true.
--
<Matt/>
/|| ** Director and CTO **
//|| ** AxKit.com Ltd ** ** XML Application Serving **
// || ** http://axkit.org ** ** XSLT, XPathScript, XSP **
// \\| // ** Personal Web Site: http://sergeant.org/ **
\\//
//\\
// \\