You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Joshua Chamas <jo...@chamas.com> on 2000/12/16 23:15:40 UTC
Linux Hello World Benchmarks: +PHP,JSP,ePerl
Hey,
Still very rough, the hello world benchmark suite is available
for download at: http://www.chamas.com/bench/hello.tar.gz
You may run it like:
# to get started, see what tests will run, note you
# may need some CPAN modules installed to get this far
perl ./bench.pl -test
# to run tests for 1 minute ... shut down your programs
# and walk away for best results.
perl ./bench.pl -time=60
Here are my latest results, having added Resin/caucho/JSP
with a J2RE 1.3.0 IBM java engine, which other benchmarks
say is the fastest java on linux overall, & from previous
testing resin seems the fastest JSP.
I changed the SSI tests to look more like the others, which
also sped them considerably. Finally, I added tests for PHP,
mine is 4.0.3, & ePerl.
Test Name Test File Hits/sec Total Hits Total Time sec/Hits Bytes/Hit
------------ ---------- ---------- ---------- ---------- ---------- ----------
Apache::ASP hello.asp 414.3 24857 hits 60.00 sec 0.002414 179 bytes
Apache::Dispatch handler hello/worl 689.5 41375 hits 60.01 sec 0.001450 134 bytes
Apache::Registry CGI Raw hello_raw. 725.2 43514 hits 60.00 sec 0.001379 52 bytes
Apache::Registry CGI.pm hello.reg 491.5 29492 hits 60.00 sec 0.002035 154 bytes
Apache::SSI hello.shtm 584.6 35080 hits 60.01 sec 0.001711 137 bytes
Apache::ePerl hello.eper 359.8 21588 hits 60.00 sec 0.002780 155 bytes
HTML static hello.html 1195.2 50000 hits 41.83 sec 0.000837 249 bytes
HTML::Embperl hello.epl 510.8 30647 hits 60.00 sec 0.001958 158 bytes
HTML::Mason hello.mas 383.8 23030 hits 60.00 sec 0.002605 134 bytes
Template Toolkit hello.tt 553.6 33221 hits 60.01 sec 0.001806 136 bytes
mod_caucho JSP hello.jsp 859.9 50000 hits 58.15 sec 0.001163 156 bytes
mod_include SSI hello.shtm 1008.0 50000 hits 49.60 sec 0.000992 136 bytes
mod_perl handler hello.benc 886.3 50000 hits 56.42 sec 0.001128 134 bytes
mod_php PHP hello.php 750.8 45050 hits 60.00 sec 0.001332 163 bytes
As has been noted, my static html is probably slower than yours
relatively. I have a dual CPU system & have most apache modules
enabled by default, thus creating huge headers for static html.
I think the dual CPU nature of my system means my system will
spend more time waiting on SMP & network locking as the request
rate gets faster, but I don't know much about these things, so if
there is something to be gained here, please feel free to clarify
how this might impact the results.
--Josh
_________________________________________________________________
Joshua Chamas Chamas Enterprises Inc.
NodeWorks >> free web link monitoring Huntington Beach, CA USA
http://www.nodeworks.com 1-714-625-4051
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
Posted by Joshua Chamas <jo...@chamas.com>.
Gunther Birznieks wrote:
>
> For the raw benchmarks...
>
> OK, I finally got a little time to download and read some the hello.tar.gz.
>
> It's good to see TT is fairly fast. But it's a shame that the only way to
> get faster than PHP is to write a raw Mod_perl handler according to the
> benchmarks. All the other mod_perl tools seem slower.
>
I'm preparing a Hello World 2000 test which will test the runtime
properties of an environment. It will be relevant to the dynamic
template environments like ASP, Mason, Embperl, TT, JSP, PHP...
early results show the oft stated perl is faster than the rest
to be true. More later for RFC.
> JSP seems to also blow away mod_perl and PHP (except being almost
> equivalent to mod_perl handler speed). I assume Resin is precompiling JSP
> to Java classes and that maybe the JRE you are using does some very good
> hotspot on-the-fly machine-code compiling type technology?
>
The IBM JDK I believe does hotspot... blow away? The diff I see
between mod_perl & mod_caucho JSP is .000035 sec/hit of my system
time. Little differences seem to make a bigger impact when
things are running this fast. Note, resin does a lot of
work to compile a JSP, really poor compile time, but you
won't see that in this test.
> How does this benchmark stuff compare to the tests run at
> http://www.chamas.com/bench/
Almost identical, except the SSI test is changed to be more
fair/similar to the rest. Note that I didn't run all those
benchmarks which is why the numbers are much different. To
see the old results grouped by the system/who did them,
check out:
http://www.chamas.com/bench/hello_bysystem.html
> I notice that JSPs take quite a beating there but are running on a lower
> end machine on that set of tests. I presume the below tests are intended to
> replace the tests run on these various disparate machines.
>
I don't know what to do with those old benchmarks. I might replace
them with results that come in from people running the bench.pl
program. But it takes a lot of time to set things up, so I don't
know if I'll just lose all of it. Its a lot of data to display
in a relevant way.
> You also seem to have taken out tests? So you are no longer testing
> servlets only? It would be interesting to see if Servlet -> JSP dispatching
> (with is the recommended model of coding Java Servlets/JSPs these days)
> results in any slow down.
>
I never tested servlets before, that was someone else. I could
set up a servlet test too, but I'm a java newbie, but I do think
the resin mod_caucho stuff does it too, so it should be shortly
forthcoming.
-- Josh
_________________________________________________________________
Joshua Chamas Chamas Enterprises Inc.
NodeWorks >> free web link monitoring Huntington Beach, CA USA
http://www.nodeworks.com 1-714-625-4051
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
Posted by Joshua Chamas <jo...@chamas.com>.
newsreader@mediaone.net wrote:
>
> On Mon, Dec 18, 2000 at 10:37:16AM -0800, Joshua Chamas wrote:
> >
> > Please feel free to run the tests yourself, and if you give
> > me the results, I'll be sure to post them at a later date
> > at http://www.chamas.com/bench/ . You can grab the benchmarks
> > from http://www.chamas.com/bench/hello.tar.gz
> >
>
> I downloaded the tar ball just to fool around on my system. It will
> be nice to have a README file with a short instruction set.
>
> Do you compile apache for each test or do you just use
> one size fits all apache with everything compiled in?
One size fits all is the idea, but only the relevant parts
of that benchmark will be loaded into a custom httpd.conf
startup file for apache. The benchmarks get run at the
same time on the same apache, so any overhead due to modules
registering themselves into apache for handler stages
will be passed to all the tests.
Yes, I'll provide a README, and --help/-h help command
line docs in the next release.
-- Josh
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
Posted by ne...@mediaone.net.
On Mon, Dec 18, 2000 at 10:37:16AM -0800, Joshua Chamas wrote:
>
> Please feel free to run the tests yourself, and if you give
> me the results, I'll be sure to post them at a later date
> at http://www.chamas.com/bench/ . You can grab the benchmarks
> from http://www.chamas.com/bench/hello.tar.gz
>
I downloaded the tar ball just to fool around on my system. It will
be nice to have a README file with a short instruction set.
Do you compile apache for each test or do you just use
one size fits all apache with everything compiled in?
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
Posted by Joshua Chamas <jo...@chamas.com>.
JR Mayberry wrote:
>
> I strongly dislike the benchmarks on the below URL, its very
> misleading..
>
> The Modperl handler benchmark, which was done on a dual P3 500mhz on
> Linux does serious injustice to mod_perl. Anyone who uses Linux knows
> how horrible it is on SMP, I think some tests showed it uses as litle as
> 25% of the second processor..
>
Please feel free to run the tests yourself, and if you give
me the results, I'll be sure to post them at a later date
at http://www.chamas.com/bench/ . You can grab the benchmarks
from http://www.chamas.com/bench/hello.tar.gz
Its only with data from multiple platforms that people will
get a comfortable look at the environments cross platform
exactly as you say.
> I don't think the benchmarks can be taken for anything, you have
> benchmarks running on different Architectures other then PC based... 300
> mhz on Sun is totally different then 300 mhz on PC based products....
> and the real number that drives this benchmark is HITS/S/MHZ..
>
Yes, I've done these 300Mhz tests before on Solaris & WinNT
and the results are at http://www.chamas.com/bench/ ... got
to http://www.chamas.com/bench/hello_bysystem.html for anything
useful.
> I think for any real test that means anything, there needs to be some
> controlled factors, ie: the hardware and operating system..
>
The new thing this round of testing is a portable test suite
that anyone can run and control their factors as they please.
--Josh
_________________________________________________________________
Joshua Chamas Chamas Enterprises Inc.
NodeWorks >> free web link monitoring Huntington Beach, CA USA
http://www.nodeworks.com 1-714-625-4051
Re: [bordering on OT] Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
Posted by Roger Espel Llima <es...@iagora.net>.
On Thu, Jan 04, 2001 at 09:55:39AM -0500, Blue Lang wrote:
> Eh, ab isn't really made as anything other than the most coarsely-grained
> of benchmarks. Concurrency testing is useless because it will measure the
> ratio of requests/second/processor, not the scalability of requests from
> single to multiple processors.
Yeah, I agree 'ab' is a pretty coarse benchmark. However, it does
in a way measure how much the various processors are helping,
because running ab with -c 1 should pretty much ensure that apache
only uses one processor at a time (except for a slight overlap while
one process does the logging and another could be reading the next
request from another processor), and similarily -c 2 should let
apache use 2 processors at one given time. All approximately, of
course.
Anyway, on that 4way server it works that way; the requests per
second increase quickly with the concurrency up to 4, but don't
increase anymore after that. That is serving relatively slow
dynamic pages; with static content I'd expect more rapidly
diminishing returns.
--
Roger Espel Llima, espel@iagora.net
http://www.iagora.com/~espel/index.html
Re: Single proc ->multi proc
Posted by Joshua Chamas <jo...@chamas.com>.
modperl@mail.thefriend.com wrote:
>
> I've got 4 new machines coming in around the 22nd. I'll have physical
> access to them for two weeks before we colo them. Probably the easiest
> way to determine mod_perl's scalability by going to multiproc on linux
> would be for me to test them. They are dual proc machines, but I can pull
> a proc out of them for testing pretty easily durring the two weeks that we
> have them here.
> Thoughts?
> recomended scripts?
> methods?
You should check out the bench.pl I just posted that runs
a series of benchmarks for you. Its at:
http://www.chamas.com/bench/hello.tar.gz
Its takes the drudgery out of benchmarking. Run ./bench.pl -h
for online help.
--Josh
_________________________________________________________________
Joshua Chamas Chamas Enterprises Inc.
NodeWorks >> free web link monitoring Huntington Beach, CA USA
http://www.nodeworks.com 1-714-625-4051
Single proc ->multi proc
Posted by mo...@mail.thefriend.com.
I've got 4 new machines coming in around the 22nd. I'll have physical
access to them for two weeks before we colo them. Probably the easiest
way to determine mod_perl's scalability by going to multiproc on linux
would be for me to test them. They are dual proc machines, but I can pull
a proc out of them for testing pretty easily durring the two weeks that we
have them here.
Thoughts?
recomended scripts?
methods?
As for ab being coarse, I'm not sure I'd agree totaly with that. It
allows a fairly good amount of control. There is also once called
sslclient that is actually worse except it does ssl.
Scott
On Thu, 4 Jan 2001, Blue Lang wrote:
> On Thu, 4 Jan 2001, Roger Espel Llima wrote:
> > JR Mayberry <jr...@e-vend.net> wrote:
> > > Linux does serious injustice to mod_perl. Anyone who uses Linux knows
> > > how horrible it is on SMP, I think some tests showed it uses as litle as
> > > 25% of the second processor..
> > A simple benchmark with 'ab' shows the number of requests per second
> > almost double when the concurrency is increased from 1 to 2. With a
> > concurrency of 4, the number of requests per second increases to
> > about 3.2 times the original, which is not bad at all considering
> > that these are dynamic requests with DB queries.
> Eh, ab isn't really made as anything other than the most coarsely-grained
> of benchmarks. Concurrency testing is useless because it will measure the
> ratio of requests/second/processor, not the scalability of requests from
> single to multiple processors.
[bordering on OT] Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
Posted by Blue Lang <bl...@gator.net>.
On Thu, 4 Jan 2001, Roger Espel Llima wrote:
> JR Mayberry <jr...@e-vend.net> wrote:
> > Linux does serious injustice to mod_perl. Anyone who uses Linux knows
> > how horrible it is on SMP, I think some tests showed it uses as litle as
> > 25% of the second processor..
>
> A simple benchmark with 'ab' shows the number of requests per second
> almost double when the concurrency is increased from 1 to 2. With a
> concurrency of 4, the number of requests per second increases to
> about 3.2 times the original, which is not bad at all considering
> that these are dynamic requests with DB queries.
Eh, ab isn't really made as anything other than the most coarsely-grained
of benchmarks. Concurrency testing is useless because it will measure the
ratio of requests/second/processor, not the scalability of requests from
single to multiple processors.
IOW, you would see almost exactly that same increase in req/second on a
single processor, most likely, unless you have a really slow machine.
You'd have to tune your load to give you one req/second/processor and then
go from there for it to mean anything.
Of course the original poster's statement on linux using only 25% of a
second CPU is a fuddy and false generalization, but that's a different
story. :P
--
Blue Lang, Unix Voodoo Priest
202 Ashe Ave, Apt 3, Raleigh, NC. 919 835 1540
"I was born in a city of sharks and sailors!" - June of 44
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
Posted by Roger Espel Llima <es...@iagora.net>.
JR Mayberry <jr...@e-vend.net> wrote:
> The Modperl handler benchmark, which was done on a dual P3 500mhz on
> Linux does serious injustice to mod_perl. Anyone who uses Linux knows
> how horrible it is on SMP, I think some tests showed it uses as litle as
> 25% of the second processor..
It's an old post, but I simply cannot let this one pass uncommented.
I run a busy Apache/mod_perl server on a 4-way SMP Linux box (kernel
2.2.13 from VA Linux), and it sure seems to be using all CPUs quite
effectively.
A simple benchmark with 'ab' shows the number of requests per second
almost double when the concurrency is increased from 1 to 2. With a
concurrency of 4, the number of requests per second increases to
about 3.2 times the original, which is not bad at all considering
that these are dynamic requests with DB queries.
Anyway, I wouldn't expect the OS's SMP to be the limiting factor on
Apache's dynamic page performance. Apache uses multiple processes,
and dynamic page generation is generally CPU bound, not I/O bound.
--
Roger Espel Llima, espel@iagora.net
http://www.iagora.com/~espel/index.html
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
Posted by JR Mayberry <jr...@e-vend.net>.
I strongly dislike the benchmarks on the below URL, its very
misleading..
The Modperl handler benchmark, which was done on a dual P3 500mhz on
Linux does serious injustice to mod_perl. Anyone who uses Linux knows
how horrible it is on SMP, I think some tests showed it uses as litle as
25% of the second processor..
I don't think the benchmarks can be taken for anything, you have
benchmarks running on different Architectures other then PC based... 300
mhz on Sun is totally different then 300 mhz on PC based products....
and the real number that drives this benchmark is HITS/S/MHZ..
I think for any real test that means anything, there needs to be some
controlled factors, ie: the hardware and operating system..
Gunther Birznieks wrote:
>
> For the raw benchmarks...
>
> OK, I finally got a little time to download and read some the hello.tar.gz.
>
> It's good to see TT is fairly fast. But it's a shame that the only way to
> get faster than PHP is to write a raw Mod_perl handler according to the
> benchmarks. All the other mod_perl tools seem slower.
>
> JSP seems to also blow away mod_perl and PHP (except being almost
> equivalent to mod_perl handler speed). I assume Resin is precompiling JSP
> to Java classes and that maybe the JRE you are using does some very good
> hotspot on-the-fly machine-code compiling type technology?
>
> How does this benchmark stuff compare to the tests run at
>
> http://www.chamas.com/bench/
>
> I notice that JSPs take quite a beating there but are running on a lower
> end machine on that set of tests. I presume the below tests are intended to
> replace the tests run on these various disparate machines.
>
> You also seem to have taken out tests? So you are no longer testing
> servlets only? It would be interesting to see if Servlet -> JSP dispatching
> (with is the recommended model of coding Java Servlets/JSPs these days)
> results in any slow down.
>
> At 02:15 PM 12/16/2000 -0800, Joshua Chamas wrote:
> >Hey,
> >
> >Still very rough, the hello world benchmark suite is available
> >for download at: http://www.chamas.com/bench/hello.tar.gz
> >You may run it like:
> >
> > # to get started, see what tests will run, note you
> > # may need some CPAN modules installed to get this far
> > perl ./bench.pl -test
> >
> > # to run tests for 1 minute ... shut down your programs
> > # and walk away for best results.
> > perl ./bench.pl -time=60
> >
> >Here are my latest results, having added Resin/caucho/JSP
> >with a J2RE 1.3.0 IBM java engine, which other benchmarks
> >say is the fastest java on linux overall, & from previous
> >testing resin seems the fastest JSP.
> >
> >I changed the SSI tests to look more like the others, which
> >also sped them considerably. Finally, I added tests for PHP,
> >mine is 4.0.3, & ePerl.
> >
> >Test Name Test File Hits/sec Total Hits Total Time
> >sec/Hits Bytes/Hit
> >------------ ---------- ---------- ---------- ----------
> >---------- ----------
> >Apache::ASP hello.asp 414.3 24857 hits 60.00
> >sec 0.002414 179 bytes
> >Apache::Dispatch handler hello/worl 689.5 41375 hits 60.01
> >sec 0.001450 134 bytes
> >Apache::Registry CGI Raw hello_raw. 725.2 43514 hits 60.00
> >sec 0.001379 52 bytes
> >Apache::Registry CGI.pm hello.reg 491.5 29492 hits 60.00
> >sec 0.002035 154 bytes
> >Apache::SSI hello.shtm 584.6 35080 hits 60.01
> >sec 0.001711 137 bytes
> >Apache::ePerl hello.eper 359.8 21588 hits 60.00
> >sec 0.002780 155 bytes
> >HTML static hello.html 1195.2 50000 hits 41.83
> >sec 0.000837 249 bytes
> >HTML::Embperl hello.epl 510.8 30647 hits 60.00
> >sec 0.001958 158 bytes
> >HTML::Mason hello.mas 383.8 23030 hits 60.00
> >sec 0.002605 134 bytes
> >Template Toolkit hello.tt 553.6 33221 hits 60.01
> >sec 0.001806 136 bytes
> >mod_caucho JSP hello.jsp 859.9 50000 hits 58.15
> >sec 0.001163 156 bytes
> >mod_include SSI hello.shtm 1008.0 50000 hits 49.60
> >sec 0.000992 136 bytes
> >mod_perl handler hello.benc 886.3 50000 hits 56.42
> >sec 0.001128 134 bytes
> >mod_php PHP hello.php 750.8 45050 hits 60.00
> >sec 0.001332 163 bytes
> >
> >As has been noted, my static html is probably slower than yours
> >relatively. I have a dual CPU system & have most apache modules
> >enabled by default, thus creating huge headers for static html.
> >
> >I think the dual CPU nature of my system means my system will
> >spend more time waiting on SMP & network locking as the request
> >rate gets faster, but I don't know much about these things, so if
> >there is something to be gained here, please feel free to clarify
> >how this might impact the results.
> >
> >--Josh
> >
> >_________________________________________________________________
> >Joshua Chamas Chamas Enterprises Inc.
> >NodeWorks >> free web link monitoring Huntington Beach, CA USA
> >http://www.nodeworks.com 1-714-625-4051
>
> __________________________________________________
> Gunther Birznieks (gunther.birznieks@extropia.com)
> eXtropia - The Web Technology Company
> http://www.extropia.com/
--
______________________________________________________________________
JR Mayberry e-Vend.net Corporation
Programmer and Systems Administrator (888) 427-8743 x226 tel
jr.mayberry@e-vend.net http://www.e-vend.net
************************************************************************
The information in this message (including attachments) is confidential.
If the reader of this message is not the intended recipient or an agent
responsible for delivering it to the intended recipient, you are hereby
notified that you have received this document in error and that any
review, dissemination, distribution, or copying of this message is
strictly prohibited. If you have received this communication in error,
please notify us immediately by e-mail, and delete the original message.
***********************************************************************
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
Posted by "G.W. Haywood" <ge...@www.jubileegroup.co.uk>.
Hi all,
On Sun, 17 Dec 2000, Gerald Richter wrote:
> there are so many factors, so they are very difficult to compare.
True. But nevertheless I think it's a very useful bit of work because
the thing that stands out is that all (server) dynamic content comes at
a high cost in processor cycles. This won't come as a surprise to most
of you. Be aware too that even the fastest of these benchmarks is not
an outstandingly fast setup.
The lesson is obvious: if you're looking for high throughput, think
long and hard about your system architecture, in the widest possible
sense, and keep server-generated dynamic stuff to a minimum.
Having said that, most of us won't be pushing the limits most of the
time (and if we are we're probably secretly quite pleased about it:).
73,
Ged.
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
Posted by Gerald Richter <ri...@ecos.de>.
> For the raw benchmarks...
>
> OK, I finally got a little time to download and read some the
hello.tar.gz.
>
> It's good to see TT is fairly fast. But it's a shame that the only way to
> get faster than PHP is to write a raw Mod_perl handler according to the
> benchmarks. All the other mod_perl tools seem slower.
>
Everybody looking at this tests should be aware that it only test the
startup/setup overhead for a "null" request. The Hello World actualy does
nothing, so it highly depends on what features are initialized at the start
of the request, it even makes a difference if different systems sends a
different number of response http headers. For this reason this benchmarks
are _not_ a comparision of the speed of the different systems which would
manifest in real life. To get some usefull numbers there must be tests for
very different situations, at least with some dynamic content in it (Joshua
is working on that), another one with database access and for example some
session management to have persistent data. There are more factors, for
example the possibility of some systems to cache the generated output, the
possibility to build your page out of many components...
But even if somebody will generate such a huge benchmark suite there are so
many factors, so they are very difficult to compare.
Gerald
-------------------------------------------------------------
Gerald Richter ecos electronic communication services gmbh
Internetconnect * Webserver/-design/-datenbanken * Consulting
Post: Tulpenstrasse 5 D-55276 Dienheim b. Mainz
E-Mail: richter@ecos.de Voice: +49 6133 925151
WWW: http://www.ecos.de Fax: +49 6133 925152
-------------------------------------------------------------
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
Posted by Perrin Harkins <pe...@primenet.com>.
Gunther Birznieks wrote:
> But it's a shame that the only way to
> get faster than PHP is to write a raw Mod_perl handler according to the
> benchmarks. All the other mod_perl tools seem slower.
It makes sense though. All the other tools do more setup work on each
request: parsing input, manipulating it into objects, etc.
> JSP seems to also blow away mod_perl and PHP (except being almost
> equivalent to mod_perl handler speed).
Hmm... To me that means it's equivalent, not blowing away mod_perl.
> I assume Resin is precompiling JSP
> to Java classes and that maybe the JRE you are using does some very good
> hotspot on-the-fly machine-code compiling type technology?
The IBM JDK has a JIT, but no Hotspot equivalent. They have the fastest
JVM on x86 though. Check out the scores here:
http://www.volano.com/report.html
One thing I discovered when testing Caucho's JSP last year is that they
do a really good job of generating efficient servlets from JSP. I was
not able to write a faster "hello world" servlet than the one they
auto-generated from a JSP page.
By the way, as a cautionary tale on "hello world" tests:
These tests appear to show mod_include handling SSI pages twice as fast
as Apache::SSI. However, in tests we've done with real production pages
(30-60K, half dozen or so includes per page) Apache::SSI was actually a
little bit faster. Of course this was about a year ago so the tables
may have turned since then.
- Perrin
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
Posted by Gunther Birznieks <gu...@extropia.com>.
For the raw benchmarks...
OK, I finally got a little time to download and read some the hello.tar.gz.
It's good to see TT is fairly fast. But it's a shame that the only way to
get faster than PHP is to write a raw Mod_perl handler according to the
benchmarks. All the other mod_perl tools seem slower.
JSP seems to also blow away mod_perl and PHP (except being almost
equivalent to mod_perl handler speed). I assume Resin is precompiling JSP
to Java classes and that maybe the JRE you are using does some very good
hotspot on-the-fly machine-code compiling type technology?
How does this benchmark stuff compare to the tests run at
http://www.chamas.com/bench/
I notice that JSPs take quite a beating there but are running on a lower
end machine on that set of tests. I presume the below tests are intended to
replace the tests run on these various disparate machines.
You also seem to have taken out tests? So you are no longer testing
servlets only? It would be interesting to see if Servlet -> JSP dispatching
(with is the recommended model of coding Java Servlets/JSPs these days)
results in any slow down.
At 02:15 PM 12/16/2000 -0800, Joshua Chamas wrote:
>Hey,
>
>Still very rough, the hello world benchmark suite is available
>for download at: http://www.chamas.com/bench/hello.tar.gz
>You may run it like:
>
> # to get started, see what tests will run, note you
> # may need some CPAN modules installed to get this far
> perl ./bench.pl -test
>
> # to run tests for 1 minute ... shut down your programs
> # and walk away for best results.
> perl ./bench.pl -time=60
>
>Here are my latest results, having added Resin/caucho/JSP
>with a J2RE 1.3.0 IBM java engine, which other benchmarks
>say is the fastest java on linux overall, & from previous
>testing resin seems the fastest JSP.
>
>I changed the SSI tests to look more like the others, which
>also sped them considerably. Finally, I added tests for PHP,
>mine is 4.0.3, & ePerl.
>
>Test Name Test File Hits/sec Total Hits Total Time
>sec/Hits Bytes/Hit
>------------ ---------- ---------- ---------- ----------
>---------- ----------
>Apache::ASP hello.asp 414.3 24857 hits 60.00
>sec 0.002414 179 bytes
>Apache::Dispatch handler hello/worl 689.5 41375 hits 60.01
>sec 0.001450 134 bytes
>Apache::Registry CGI Raw hello_raw. 725.2 43514 hits 60.00
>sec 0.001379 52 bytes
>Apache::Registry CGI.pm hello.reg 491.5 29492 hits 60.00
>sec 0.002035 154 bytes
>Apache::SSI hello.shtm 584.6 35080 hits 60.01
>sec 0.001711 137 bytes
>Apache::ePerl hello.eper 359.8 21588 hits 60.00
>sec 0.002780 155 bytes
>HTML static hello.html 1195.2 50000 hits 41.83
>sec 0.000837 249 bytes
>HTML::Embperl hello.epl 510.8 30647 hits 60.00
>sec 0.001958 158 bytes
>HTML::Mason hello.mas 383.8 23030 hits 60.00
>sec 0.002605 134 bytes
>Template Toolkit hello.tt 553.6 33221 hits 60.01
>sec 0.001806 136 bytes
>mod_caucho JSP hello.jsp 859.9 50000 hits 58.15
>sec 0.001163 156 bytes
>mod_include SSI hello.shtm 1008.0 50000 hits 49.60
>sec 0.000992 136 bytes
>mod_perl handler hello.benc 886.3 50000 hits 56.42
>sec 0.001128 134 bytes
>mod_php PHP hello.php 750.8 45050 hits 60.00
>sec 0.001332 163 bytes
>
>As has been noted, my static html is probably slower than yours
>relatively. I have a dual CPU system & have most apache modules
>enabled by default, thus creating huge headers for static html.
>
>I think the dual CPU nature of my system means my system will
>spend more time waiting on SMP & network locking as the request
>rate gets faster, but I don't know much about these things, so if
>there is something to be gained here, please feel free to clarify
>how this might impact the results.
>
>--Josh
>
>_________________________________________________________________
>Joshua Chamas Chamas Enterprises Inc.
>NodeWorks >> free web link monitoring Huntington Beach, CA USA
>http://www.nodeworks.com 1-714-625-4051
__________________________________________________
Gunther Birznieks (gunther.birznieks@extropia.com)
eXtropia - The Web Technology Company
http://www.extropia.com/