You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Stas Bekman <sb...@iname.com> on 2000/02/16 17:27:07 UTC
choosing the right benchmark code
I've discovered a weird thing while working the perlformance section of
the guide. I wanted to show that methods are a little bit slower than
functions. I wrote the benchmark to prove this, but the results I've
received were reversed to my expectations -- the methods were faster than
the functions.
This is the code I've used:
use Benchmark;
use CGI qw(:standard);
$CGI::NO_DEBUG = 1;
my $q = new CGI;
my $x;
timethese
(10000,
{
'Method' => sub {$q->param('x',5); $x = $q->param('x'); },
'Function' => sub {param('x',5); $x = param('x'); },
});
In order to make the benchmarking as correct as possible, I've used
CGI.pm, since it provides both, the methods and functions API. I also made
all the initializations before the benchmark itself.
Do you have any ideas, what's wrong with the code I've used? The "method"
should be slower than "function" because of the class pointer derefence
and lookup of the method in the class.
The second mystery is that I've received different times while rerunning
the same benchmark.
Here are a few of them:
Function: 23 wallclock secs (14.63 usr + 1.32 sys = 15.95 CPU)
Method: 22 wallclock secs (13.54 usr + 1.14 sys = 14.68 CPU)
Function: 24 wallclock secs (14.38 usr + 1.28 sys = 15.66 CPU)
Method: 19 wallclock secs (13.10 usr + 1.09 sys = 14.19 CPU)
Function: 23 wallclock secs (14.77 usr + 1.29 sys = 16.06 CPU)
Method: 21 wallclock secs (13.70 usr + 1.10 sys = 14.80 CPU)
Function: 21 wallclock secs (14.02 usr + 1.39 sys = 15.41 CPU)
Method: 20 wallclock secs (13.32 usr + 1.12 sys = 14.44 CPU)
Isn't the total CPU clocks should be the same?
Thanks!
_______________________________________________________________________
Stas Bekman mailto:sbekman@iname.com http://www.stason.org/stas
Perl,CGI,Apache,Linux,Web,Java,PC http://www.stason.org/stas/TULARC
perl.apache.org modperl.sourcegarden.org perlmonth.com perl.org
single o-> + single o-+ = singlesheaven http://www.singlesheaven.com
yanf@nadym.ru
Posted by YANF WSCB <YA...@nadym.ru>.
unsubscribe
Re: choosing the right benchmark code
Posted by Doug MacEachern <do...@pobox.com>.
On Thu, 17 Feb 2000, Matt Sergeant wrote:
>
> It's also worth noting that this is _very_ much dependant on your perl
> version. Later versions of perl do much better method caching - I believe
> in the current devel version of perl method calls are only very slightly
> slower - rather than 2 times slower.
only a little better though, newer Perls make Foo->method() a bit faster
(some constant folding magic), but not Foo->$method(). and the
improvement does not address the @ISA lookup that still happens in either
case.
Re: choosing the right benchmark code
Posted by Autarch <au...@urth.org>.
On Thu, 17 Feb 2000, Matt Sergeant wrote:
> It's also worth noting that this is _very_ much dependant on your perl
> version. Later versions of perl do much better method caching - I believe
> in the current devel version of perl method calls are only very slightly
> slower - rather than 2 times slower.
At the risk of getting horrible completely off-topic:
9:46 ~ $ perl5.00503 -w t.pl
Benchmark: timing 1000000 iterations of function, method...
function: 3 wallclock secs ( 3.87 usr + 0.00 sys = 3.87 CPU)
method: 6 wallclock secs ( 5.70 usr + 0.00 sys = 5.70 CPU)
9:47 ~ $ perl5.00503 -w t.pl
Benchmark: timing 1000000 iterations of function, method...
function: 5 wallclock secs ( 3.85 usr + 0.00 sys = 3.85 CPU)
method: 6 wallclock secs ( 5.71 usr + 0.00 sys = 5.71 CPU)
9:47 ~ $ perl5.00503 -w t.pl
Benchmark: timing 1000000 iterations of function, method...
function: 3 wallclock secs ( 3.86 usr + 0.00 sys = 3.86 CPU)
method: 6 wallclock secs ( 5.70 usr + 0.00 sys = 5.70 CPU)
9:50 ~ $ /usr/local/perl5.5.65/bin/perl5.5.650 -w t.pl
Benchmark: timing 1000000 iterations of function, method...
function: 2 wallclock secs ( 1.67 usr + 0.01 sys = 1.68 CPU) @
595238.10/s (n=1000000)
method: 5 wallclock secs ( 5.62 usr + 0.00 sys = 5.62 CPU) @
177935.94/s (n=1000000)
9:51 ~ $ /usr/local/perl5.5.65/bin/perl5.5.650 -w t.pl
Benchmark: timing 1000000 iterations of function, method...
function: 0 wallclock secs ( 1.67 usr + 0.00 sys = 1.67 CPU) @
598802.40/s (n=1000000)
method: 5 wallclock secs ( 5.62 usr + 0.00 sys = 5.62 CPU) @
177935.94/s (n=1000000)
9:51 ~ $ /usr/local/perl5.5.65/bin/perl5.5.650 -w t.pl
Benchmark: timing 1000000 iterations of function, method...
function: 2 wallclock secs ( 1.68 usr + 0.00 sys = 1.68 CPU) @
595238.10/s (n=1000000)
method: 6 wallclock secs ( 5.61 usr + 0.00 sys = 5.61 CPU) @
178253.12/s (n=1000000)
Well, judging from this function calls have somehow been sped up by two
by a factor of 2 but methods are untouched. I wonder if this test is
somehow unrepresentative (I also wonder why I got 0 wallclock seconds for
a test that took 1.68 CPU).
-dave
/*==================
www.urth.org
We await the New Sun
==================*/
Re: choosing the right benchmark code
Posted by Matt Sergeant <ma...@sergeant.org>.
On Thu, 17 Feb 2000, Doug MacEachern wrote:
> On Wed, 16 Feb 2000, Stas Bekman wrote:
>
> > I've discovered a weird thing while working the perlformance section of
> > the guide. I wanted to show that methods are a little bit slower than
> > functions. I wrote the benchmark to prove this, but the results I've
> > received were reversed to my expectations -- the methods were faster than
> > the functions.
>
> for something like this, less code == better test, CGI.pm does lots of
> things under the covers.
>
> all you need is this:
>
> package Foo;
>
> sub bar { }
>
> use strict;
> use Benchmark;
>
> timethese(50_000, {
> method => sub {
> Foo->bar();
> },
> function => sub {
> Foo::bar('Foo');
> },
> });
>
> Benchmark: timing 50000 iterations of function, method...
> function: 1 wallclock secs ( 0.43 usr + 0.00 sys = 0.43 CPU)
> method: 2 wallclock secs ( 0.89 usr + 0.00 sys = 0.89 CPU)
It's also worth noting that this is _very_ much dependant on your perl
version. Later versions of perl do much better method caching - I believe
in the current devel version of perl method calls are only very slightly
slower - rather than 2 times slower.
--
<Matt/>
Details: FastNet Software Ltd - XML, Perl, Databases.
Tagline: High Performance Web Solutions
Web Sites: http://come.to/fastnet http://sergeant.org
Available for Consultancy, Contracts and Training.
Re: choosing the right benchmark code
Posted by Doug MacEachern <do...@pobox.com>.
On Wed, 16 Feb 2000, Stas Bekman wrote:
> I've discovered a weird thing while working the perlformance section of
> the guide. I wanted to show that methods are a little bit slower than
> functions. I wrote the benchmark to prove this, but the results I've
> received were reversed to my expectations -- the methods were faster than
> the functions.
for something like this, less code == better test, CGI.pm does lots of
things under the covers.
all you need is this:
package Foo;
sub bar { }
use strict;
use Benchmark;
timethese(50_000, {
method => sub {
Foo->bar();
},
function => sub {
Foo::bar('Foo');
},
});
Benchmark: timing 50000 iterations of function, method...
function: 1 wallclock secs ( 0.43 usr + 0.00 sys = 0.43 CPU)
method: 2 wallclock secs ( 0.89 usr + 0.00 sys = 0.89 CPU)