[RFC: performance] Initializing CGI.pm

[Stas Bekman <sb...@stason.org>]

Here is another one. You should use Apache::Request instead, but just in
case you still use CGI.pm (me too :)

as usuals comments are welcome:

C<CGI.pm> is a big module that by default postpones the compilation of
its methods until they are actually needed, thus making it possible to
use it under slow mod_cgi handler without adding a big
overhead. That's not what we want under mod_perl and if you use
C<CGI.pm> you should precompile the methods that you are going to use
at the server startup in addition to preloading of the module.  Use
the compile method for that:

  use CGI;
  CGI->compile(':all');

where you should replace the tag group C<:all> with the real tags and
group tags that you are going to use if you want to optimize the
memory usage.

We are going to compare the shared memory foot print by using the
script which is back compatible with mod_cgi. You will see that you
can improve performance of this kind of scripts as well, but if you
really want a fast code think about porting it to use
C<Apache::Request> for CGI interface and some other module for HTML
generation.

So here is the C<Apache::Registry> script that we are going to use to
make the comparison:

  preload_cgi_pm.pl
  -----------------
  use strict;
  use CGI ();
  use GTop ();

  my $q = new CGI;
  print $q->header('text/plain');
  print join "\n", map {"$_ => ".$q->param($_) } $q->param;
  print "\n";
  
  my $proc_mem = GTop->new->proc_mem($$);
  my $size  = $proc_mem->size;
  my $share = $proc_mem->share;
  my $diff  = $size - $share;
  printf "%8s %8s %8s\n", qw(Size Shared Diff);
  printf "%8d %8d %8d (bytes)\n",$size,$share,$diff;

The script initializes the C<CGI> object, sends HTTP header and then
print all the arguments and values that were passed to the script if
at all. At the end as usual we print the memory usage.

As usual we are going to use a single child process, therefore we will
use this setting in I<httpd.conf>:

  MinSpareServers 1
  MaxSpareServers 1
  StartServers 1
  MaxClients 1
  MaxRequestsPerChild 100

We are going to run memory benchmarks on three different versions of
the I<startup.pl> file.  We always preload this module:

  use Gtop();

=over

=item option 1

Leave the file unmodified.

=item option 2

Preload C<CGI.pm>:

  use CGI ();

=item option 3

Preload C<CGI.pm> and pre-compile the methods that we are going to use
in the script:

  use CGI ();
  CGI->compile(qw(header param));

=back

The server was restarted before each new test.

So here are the results of the five tests that were conducted, sorted
by the I<Diff> column:

=over

=item 1

After the first request:

  Version     Size   Shared     Diff        Test type
  --------------------------------------------------------------------
        1  3321856  2146304  1175552  not preloaded
        2  3321856  2326528   995328  preloaded
        3  3244032  2465792   778240  preloaded & methods+compiled

=item 2

After the second request (all the subsequent request showed the same
results):

  Version     Size   Shared    Diff         Test type
  --------------------------------------------------------------------
        1  3325952  2134016  1191936 not preloaded
        2  3325952  2314240  1011712 preloaded
        3  3248128  2445312   802816 preloaded & methods+compiled

=back

The first version shows the results of the script execution when
C<CGI.pm> wasn't preloaded. The second version with module
preloaded. The third when it's both preloaded and the methods that are
going to be used are precompiled at the server startup.

By looking at the version one of the second table we can conclude
that, preloading adds about 20K of shared size. As we have mention at
the beginning of this section that's how C<CGI.pm> was implemented--to
reduce the load overhead. Which means that preloading CGI is almost
change a thing. But if we compare the second and the third versions we
will see a very significant difference of 207K (1011712-802816), and
we have used only a few methods (the I<header> method loads a few more
method transparently for a user). Imagine how much memory we are going
to save if we are going to precompile all the methods that we are
using in other scripts that use C<CGI.pm> and do a little bit more
than the script that we have used in the test.

But even in our very simple case using the same formula, what do we
see? (assuming that we have 256MB dedicated for mod_perl)

               RAM - largest_shared_size
  N_of Procs = -------------------------
                        Diff

                268435456 - 2134016
  (ver 1)  N =  ------------------- = 223
                      1191936

                268435456 - 2445312
  (ver 3)  N =  ------------------- = 331
                     802816

If we preload C<CGI.pm> and precompile a few methods that we use in
the test script, we can have 50% more child processes than when we
don't preload and precompile the methods that we are going to use.

META: I've heard that the 3.x generation will be less bloated, so
probably I'll have to rerun this using the new version.


_____________________________________________________________________
Stas Bekman              JAm_pH     --   Just Another mod_perl Hacker
http://stason.org/       mod_perl Guide  http://perl.apache.org/guide 
mailto:stas@stason.org   http://perl.org     http://stason.org/TULARC
http://singlesheaven.com http://perlmonth.com http://sourcegarden.org