You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Stas Bekman <sb...@stason.org> on 2000/06/03 03:00:54 UTC
[RFC: performance] Preloading Perl Modules at Server Startup
Here is another one:
You can use the C<PerlRequire> and C<PerlModule> directives to load
commonly used modules such as C<CGI.pm>, C<DBI> and etc., when the
server is started. On most systems, server children will be able to
share the code space used by these modules. Just add the following
directives into I<httpd.conf>:
PerlModule CGI
PerlModule DBI
But an even better approach is to create a separate startup file
(where you code in plain perl) and put there things like:
use DBI;
use Carp;
Then you C<require()> this startup file in I<httpd.conf> with the
C<PerlRequire> directive, placing it before the rest of the mod_perl
configuration directives:
PerlRequire /path/to/start-up.pl
C<CGI.pm> is a special case. Ordinarily C<CGI.pm> autoloads most of
its functions on an as-needed basis. This speeds up the loading time
by deferring the compilation phase. When you use mod_perl, FastCGI or
another system that uses a persistent Perl interpreter, you will want
to precompile the functions at initialization time. To accomplish
this, call the package function compile() like this:
use CGI ();
CGI->compile(':all');
The arguments to C<compile()> are a list of method names or sets, and
are identical to those accepted by the C<use()> and C<import()>
operators. Note that in most cases you will want to replace C<':all'>
with the tag names that you actually use in your code, since generally
you only use a subset of them.
Let's conduct a memory usage test to prove that preloading, reduces
memory requirements.
In order to have an easy measurement we will use only one child
process, therefore we will use this setting:
MinSpareServers 1
MaxSpareServers 1
StartServers 1
MaxClients 1
MaxRequestsPerChild 100
We are going to use the C<Apache::Registry> script I<memuse.pl> which
consists of two parts: the first one preloads a bunch of modules (that
most of them aren't going to be used), the second part reports the
memory size and the shared memory size used by the single child
process that we start. and of course it prints the difference between
the two sizes.
memuse.pl
---------
use strict;
use CGI ();
use DB_File ();
use LWP::UserAgent ();
use Storable ();
use DBI ();
use GTop ();
my $r = shift;
$r->send_http_header('text/plain');
my $proc_mem = GTop->new->proc_mem($$);
my $size = $proc_mem->size;
my $share = $proc_mem->share;
my $diff = $size - $share;
printf "%10s %10s %10s\n", qw(Size Shared Difference);
printf "%10d %10d %10d (bytes)\n",$size,$share,$diff;
First we restart the server and execute this CGI script when none of
the above modules preloaded. Here is the result:
Size Shared Diff
4706304 2134016 2572288 (bytes)
Now we take all the modules:
use strict;
use CGI ();
use DB_File ();
use LWP::UserAgent ();
use Storable ();
use DBI ();
use GTop ();
and copy them into the startup script, so they will get preloaded.
The script remains unchanged. We restart the server and execute it
again. We get the following.
Size Shared Diff
4710400 3997696 712704 (bytes)
Let's put the two results into one table:
Preloading Size Shared Diff
Yes 4710400 3997696 712704 (bytes)
No 4706304 2134016 2572288 (bytes)
--------------------------------------------
Difference 4096 1863680 -1859584
You can clearly see that when the modules weren't preloaded the shared
memory pages size, were about 1864Kb smaller relative to the case
where the modules were preloaded.
Assuming that you have had 256M dedicated to the web server, if you
didn't preload the modules, you could have:
268435456 = X * 2572288 + 2134016
X = (268435456 - 2134016) / 2572288 = 103
103 servers.
Now let's calculate the same thing with modules preloaded:
268435456 = X * 712704 + 3997696
X = (268435456 - 3997696) / 712704 = 371
You can have almost 4 times more servers!!!
Remember that we have mentioned before that memory pages gets dirty
and the size of the shared memory gets smaller with time? So we have
presented the ideal case where the shared memory stays
intact. Therefore the real numbers will be a little bit different, but
not far from the numbers in our example.
Also it's obvious that in your case it's possible that the process
size will be bigger and the shared memory will be smaller, since you
will use different modules and a different code, so you won't get this
fantastic ratio, but this example is certainly helps to feel the
difference.
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://perl.org http://stason.org/TULARC
http://singlesheaven.com http://perlmonth.com http://sourcegarden.org
Re: [RFC: performance] Preloading Perl Modules at Server Startup
Posted by Stas Bekman <sb...@stason.org>.
On Mon, 5 Jun 2000, Vivek Khera wrote:
> >>>>> "SB" == Stas Bekman <sb...@stason.org> writes:
>
> SB> But an even better approach is to create a separate startup file
> SB> (where you code in plain perl) and put there things like:
>
> SB> use DBI;
> SB> use Carp;
>
> SB> Then you C<require()> this startup file in I<httpd.conf> with the
> SB> C<PerlRequire> directive, placing it before the rest of the mod_perl
> SB> configuration directives:
>
> SB> PerlRequire /path/to/start-up.pl
>
>
> You should recommend
>
> use DBI ();
> use Carp ();
>
> so that no symbols are imported into the name space of the start-up.pl
> script as it is unlikely to be needed there.
that's right.
> Also, I'd recommend using libapreq's Apache::Request if you don't need
> the content generating parts of CGI.pm... which leads to an
> enhancement
That's a topic of another section.
> I'd like to see Doug add to libapreq's functionality:
>
> Currently, you need to do a call like this if you're using
> Apache::Request inside a Registry script:
>
> my $ar = Apache::Request->new(Apache->request()) or die "Whoa Nelly!";
>
> I'd like to see the Apache::Request->new() method automatically call
> Apache->request() if no parameters are passed to it. Inside a
> Apache->handler, it is easy since you already have a Apache->request
> object passed to you, but in Registry, it is a lot of extra typing.
As Geoff has mentioned it works already with shift() ... seems ok to me.
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://perl.org http://stason.org/TULARC
http://singlesheaven.com http://perlmonth.com http://sourcegarden.org
Re: [RFC: performance] Preloading Perl Modules at Server Startup
Posted by Vivek Khera <kh...@kciLink.com>.
>>>>> "SB" == Stas Bekman <sb...@stason.org> writes:
SB> But an even better approach is to create a separate startup file
SB> (where you code in plain perl) and put there things like:
SB> use DBI;
SB> use Carp;
SB> Then you C<require()> this startup file in I<httpd.conf> with the
SB> C<PerlRequire> directive, placing it before the rest of the mod_perl
SB> configuration directives:
SB> PerlRequire /path/to/start-up.pl
You should recommend
use DBI ();
use Carp ();
so that no symbols are imported into the name space of the start-up.pl
script as it is unlikely to be needed there.
Also, I'd recommend using libapreq's Apache::Request if you don't need
the content generating parts of CGI.pm... which leads to an
enhancement I'd like to see Doug add to libapreq's functionality:
Currently, you need to do a call like this if you're using
Apache::Request inside a Registry script:
my $ar = Apache::Request->new(Apache->request()) or die "Whoa Nelly!";
I'd like to see the Apache::Request->new() method automatically call
Apache->request() if no parameters are passed to it. Inside a
Apache->handler, it is easy since you already have a Apache->request
object passed to you, but in Registry, it is a lot of extra typing.