testing server response time

[Alan Raetz <al...@yahoo.com>]

I was trying to test the CGI response time of a
Perl/CGI script under two conditions: as multiple
modules versus a single big script. First I installed
IndigoPerl (from indigostar.com) on my Windows machine
(which has an integrated apache server with mod_perl
enabled by default). Then I ran this script locally
against the local server:

#!/usr/local/bin/perl

use strict;

use LWP::Simple;
use Time::HiRes;

my $script = $ARGV[0];

my $totalTime = 0;

my $i; for ($i=0; $i < 20; $i++) {

     my $before = Time::HiRes::gettimeofday();

     if ( $script eq 'big' ) {

          print "Running big: ";

          if ( !defined( my $content =
              
LWP::Simple::get('http://localhost/cgi-bin/bigtool.cgi')))
{
                  print "failed get\n";
          }

     } else {

          if ( !defined( my $content =
              
LWP::Simple::get('http://localhost/cgi-bin/webtool.cgi')))
{
                  print "failed get\n"; }

     }

     my $elapsed = Time::HiRes::gettimeofday() -
$before;

     print "did $i in $elapsed seconds.\n";

     $totalTime = $totalTime + $elapsed;
}

$totalTime = $totalTime / 20;

print "Average response time for 20 requests was:
$totalTime seconds.\n";

######## end of script #########

After I set up my app (webtool.cgi) and created the
single script version (bigtool.cgi), I ran this script
on my machine and it showed that the single file was
about 10-15% faster than the multiple modules.

My first question is, is the above script a valid test
of CGI response time? So, for example, should the
results reflect any improvements with mod_perl
enabled? Because what I found is that the response
time differed 
less than 5% between mod_perl-enabled and mod_perl
disabled configurations.

The CGI application I'm testing this on is about 4,100
lines of Perl, and 114k bytes.

I emailed support@indigostar, but it's been over two
days and no reply yet. I figure I'd just throw this
out
there for feedback. From other postings, it seems like
Windows mod_perl works great, and I should see a
significant speed-up.

btw, I'm running Windows 98 SE on a 500 MHz machine.

Here's another test I did: I ran my test script
remotely against my local IndigoPerl server
from my remote unix account on sourceforge (because
I thought maybe an interaction between the test
script and the server running on the same machine).
Basically, it shows the same results: enabling
mod_perl only improves performance about 0-5%. I
even tried this timing test against the simple
one-line
'hellocgi.pl' script that came with IndigoPerl, and
it's response is also similar with or without mod_perl
(0.47 sec versus 0.54 seconds).

Here's the thing I discoved in doing this though: when
mod_perl is enabled, the disk indicator on my machine
does not blink on when running the remote test script;
with mod_perl disabled, the light blinks! So this 
would indicate that mod_perl is getting enabled
and is 'working', not going to disk for requests...
unfortunately, it doesn't seem to improve the overall
response time much.

Comments? I'm certainly hoping that either this isn't
a valid test, or that IndigoPerl is just broken...

-alan


__________________________________________________
Do You Yahoo!?
Send FREE video emails in Yahoo! Mail!
http://promo.yahoo.com/videomail/

Re: testing server response time

[Alan Raetz <al...@yahoo.com>]

Perrin,

> You want something more like this:
> 
>  Alias /perl-bin/ "c:/IndigoPerl//perl-bin/"
>  PerlModule Apache::Registry
>  <Location /perl-bin>
>    SetHandler perl-script
>    PerlHandler Apache::Registry
>    Options ExecCGI
>  </Location>

Yup, this gets it working (It does need the line 
"LoadModule perl_module modules/mod_perl.so").
The response time for the one-liner script is 
reduced by a 10x factor after the first request,
according to my test script. The conf snippet I 
gave you was somehow corrupted when I copied it, 
but just to make sure, I reinstalled indigoperl 
from scratch, and the conf file from the original 
install does not work; "PerlModule Apache::Registry"
is not declared, and it uses "ScriptAlias /perl-bin/"
instead of "Alias /perl-bin/" and I verified that 
both of these lines cause problems. 

IndigoStar says that the server comes with mod_perl
enabled by default, and so I just assumed those
lines were correct. You saved me a lot of time,
thanks! I'll email IndigoStar this info...

-alan




__________________________________________________
Do You Yahoo!?
Send FREE video emails in Yahoo! Mail!
http://promo.yahoo.com/videomail/

Re: testing server response time

[Perrin Harkins <pe...@elem.com>]

> I was also thinking it would only make a small
> difference, but I see many perl/CGI scripts that boast
> 'all this functionality in a single script'

They probably don't know any better, but to me that translates to "giant
bloated unorganized mess of a script."

> # BEGIN MOD_PERL CONFIG
> #LoadModule perl_module modules/mod_perl.so
> #ScriptAlias /perl-bin/ "c:/IndigoPerl//perl-bin/"
> #PerlSendHeader On
> #SetHandler perl-script
> #Options ExecCGI
> #</Location>
> # END MOD_PERL CONFIG

That won't do it.  Check the docs for Apache::Registry.  That
ScriptAlias should be removed for mod_perl.

You want something more like this:

 Alias /perl-bin/ "c:/IndigoPerl//perl-bin/"
 PerlModule Apache::Registry
 <Location /perl-bin>
   SetHandler perl-script
   PerlHandler Apache::Registry
   Options ExecCGI
 </Location>

Not sure if you need PerlSendHeader or not.  It depends on your code.

- Perrin

Re: testing server response time

[Alan Raetz <al...@yahoo.com>]

Perrin,

Thanks for the response,

--- Perrin Harkins <pe...@elem.com> wrote:
> No offense, but your script must not have been doing
> much in this test.
> The difference between putting everything in one
> script vs. using
> modules is just the time it takes to open and read
> the files.  It's a
> tiny difference in most cases.  

I was also thinking it would only make a small
difference, but I see many perl/CGI scripts that boast
'all this functionality in a single script' as 
if multiple module was some sort of huge performance 
hit. My app has 13 separate modules, so I tried to 
test it. I'm sure both version had 
the same code, because they both worked fine. It's 
about 4000 lines of perl code (with CGI_Lite built 
in). So I was surprised it made such a big difference,
as well. You can see a demo of the app at 
http://chicodigital.com/demo.html

> 
> There could be a problem in your config.  How about
> posting the part you
> changed to enable mod_perl?

This is the lines in C:\indigoperl\conf\httpd.conf.
I either commented out and restarted the server to 
disable, or uncommented, stopped apache and restarted 
to enable:

# BEGIN MOD_PERL CONFIG
#LoadModule perl_module modules/mod_perl.so
#ScriptAlias /perl-bin/ "c:/IndigoPerl//perl-bin/"
#PerlSendHeader On
#SetHandler perl-script
#Options ExecCGI
#</Location>
# END MOD_PERL CONFIG

The application under test runs fine in either case.

thanks again,

-alan






__________________________________________________
Do You Yahoo!?
Send FREE video emails in Yahoo! Mail!
http://promo.yahoo.com/videomail/

Re: testing server response time

[Perrin Harkins <pe...@elem.com>]

> After I set up my app (webtool.cgi) and created the
> single script version (bigtool.cgi), I ran this script
> on my machine and it showed that the single file was
> about 10-15% faster than the multiple modules.

No offense, but your script must not have been doing much in this test.
The difference between putting everything in one script vs. using
modules is just the time it takes to open and read the files.  It's a
tiny difference in most cases.  Are you sure you were compiling all of
the same code in both cases?

> My first question is, is the above script a valid test
> of CGI response time?

It's not useless, but LWP isn't the fastest and it's a lot easier to
just use ab, or one of the other tools suggested in the guide.

> So, for example, should the
> results reflect any improvements with mod_perl
> enabled?

Yes.

> Because what I found is that the response
> time differed
> less than 5% between mod_perl-enabled and mod_perl
> disabled configurations.

There could be a problem in your config.  How about posting the part you
changed to enable mod_perl?

> From other postings, it seems like
> Windows mod_perl works great, and I should see a
> significant speed-up.

Not as much as on unix, but definitely a speed-up.

- Perrin