You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Daniel Wilson <we...@animelab.com> on 2004/01/05 06:36:53 UTC
[mp1] modules not shared across children. preloading or fork() issue?
For referance the problem i am seeing, is that even though i have configured a load of modules to be preloaded, they are not being preloaded, or at least not being shared.
This happened after moving to a new server running the same redhat 9 but with different kernels and glibc's. I'm betting thats where the problem is. In the old server everything was shared for a good while.
Has anyone had similar problems with RedHat 9?
It appears to be one of two things;
1. mod_perl is not loading modules before the fork()'s.
2. fork() is not correctly identifing pages which can be shared.
How ever, all the code does appear to be loaded into the parent apache process its just seperated during the fork.
mod_perl / apache configured with
perl Makefile.PL APACHE_SRC=../apache_1.3.29/src USE_APACI=1 PERL_INIT=1 PERL_STACKED_HANDLERS=1 ALL_HOOKS=1 DO_HTTPD=1 APACI_ARGS='--prefix=/usr --activate-module=src/modules/gzip/mod_gzip.a --enable-module=env --activate-module=src/modules/php4/libphp4.a --activate-module=src/modules/perl/libperl.a --enable-module=log_config --disable-module=log_agent --disable-module=log_referer --enable-module=mime --enable-module=negotiation --enable-module=include --disable-module=autoindex --enable-module=dir --enable-module=cgi --enable-module=alias --enable-module=rewrite --enable-module=headers --enable-module=access --enable-module=auth --enable-module=expires --enable-module=setenvif --disable-module=userdir --disable-module=status --disable-module=info --disable-module=asis --enable-suexec --suexec-docroot=/var/www --suexec-caller=apache'
Useful stuff...
------- httpd.conf #####################
PerlModule Apache::Registry Apache::RegistryLoader DBD::mysql BSD::Resource DBI
PerlModule Storable Text::Kakasi Encode::Encoding Apache::DBI Carp
PerlModule IPC::Shareable Jcode IPC::SysV Encode Encode::Alias Lingua::JA::Romaji
PerlRequire "/var/www/startup.pl"
<Files anime.manga>
SetHandler perl-script
PerlHandler Apache::Registry
Options ExecCGI
PerlSendHeader On
</Files>
------- /var/www/startup.pl #####################
BEGIN {
use lib qw(/var/www/);
use strict;
#use Apache::Registry;
use lib::ModHTML ();
use IPC::Shareable ();
use drivers::index ();
.......
use drivers::anistuff ();
use Apache::DBI;
# use DBI;
use Apache::RegistryLoader;
Apache::RegistryLoader->new->handler("/anime.manga","/var/www/html/anime.manga");
Apache::DBI->connect_on_init("DBI:mysql:database=xxxxx", "xxxxx","xxxxxx",
{
PrintError => 1, # warn() on errors
RaiseError => 0, # don't die on error
AutoCommit => 1, # commit executes immediately
}
);
}
1;
}
------- TOP #####################
(apache was just started)
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
25270 apache 15 0 12772 12M 3152 S 0.0 2.5 0:00 0 httpd
25255 apache 23 0 12640 12M 3120 S 0.0 2.4 0:00 0 httpd
25284 apache 15 0 12620 12M 3144 S 1.1 2.4 0:00 0 httpd
25254 root 15 0 11196 10M 2804 S 1.0 2.2 0:05 0 httpd (parent)
------- pmap - parent#####################
(other junk same as below)
mapped: 15720 KB writable/private: 8904 KB shared: 72 KB
------- pmap - child #####################
httpd(25270)
08048000 (1600 KB) r-xp (03:03 131451) /usr/sbin/httpd
081d8000 (88 KB) rw-p (03:03 131451) /usr/sbin/httpd
081ee000 (9384 KB) rwxp (00:00 0)
40000000 (84 KB) r-xp (03:01 539705) /lib/ld-2.3.2.so
40015000 (4 KB) rw-p (03:01 539705) /lib/ld-2.3.2.so
40016000 (4 KB) rw-p (00:00 0)
40017000 (12 KB) r-xp (03:03 929343) /usr/lib/perl5/site_perl/5.8.2/i686-linux/auto/BSD/Resource/Resource.so
4001a000 (4 KB) rw-p (03:03 929343) /usr/lib/perl5/site_perl/5.8.2/i686-linux/auto/BSD/Resource/Resource.so
4001b000 (12 KB) r-xp (03:03 439898) /usr/lib/perl5/5.8.2/i686-linux/auto/Fcntl/Fcntl.so
4001e000 (4 KB) rw-p (03:03 439898) /usr/lib/perl5/5.8.2/i686-linux/auto/Fcntl/Fcntl.so
4001f000 (8 KB) r-xp (03:03 848151) /usr/lib/perl5/site_perl/5.8.2/i686-linux/auto/Text/Kakasi/Kakasi.so
40021000 (4 KB) rw-p (03:03 848151) /usr/lib/perl5/site_perl/5.8.2/i686-linux/auto/Text/Kakasi/Kakasi.so
40022000 (4 KB) rw-p (00:00 0)
40023000 (20 KB) r-xp (03:01 539626) /lib/libcrypt-2.3.2.so
40028000 (4 KB) rw-p (03:01 539626) /lib/libcrypt-2.3.2.so
40029000 (156 KB) rw-p (00:00 0)
40050000 (60 KB) r-xp (03:01 539648) /lib/libresolv-2.3.2.so
4005f000 (4 KB) rw-p (03:01 539648) /lib/libresolv-2.3.2.so
40060000 (8 KB) rw-p (00:00 0)
40062000 (132 KB) r-xp (03:01 637731) /lib/tls/libm-2.3.2.so
40083000 (4 KB) rw-p (03:01 637731) /lib/tls/libm-2.3.2.so
40084000 (12 KB) r-xp (03:01 539628) /lib/libdl-2.3.2.so
40087000 (4 KB) rw-p (03:01 539628) /lib/libdl-2.3.2.so
40088000 (72 KB) r-xp (03:01 539632) /lib/libnsl-2.3.2.so
4009a000 (4 KB) rw-p (03:01 539632) /lib/libnsl-2.3.2.so
4009b000 (8 KB) rw-p (00:00 0)
4009d000 (24 KB) r-xp (03:03 65209) /usr/lib/libgdbm.so.2.0.0
400a3000 (4 KB) rw-p (03:03 65209) /usr/lib/libgdbm.so.2.0.0
400a4000 (4 KB) rw-p (00:00 0)
400a5000 (932 KB) r-xp (03:03 945623) /usr/lib/perl5/5.8.2/i686-linux/CORE/libperl.so
4018e000 (40 KB) rw-p (03:03 945623) /usr/lib/perl5/5.8.2/i686-linux/CORE/libperl.so
40198000 (12 KB) rw-p (00:00 0)
4019b000 (8 KB) r-xp (03:01 539654) /lib/libutil-2.3.2.so
4019d000 (4 KB) rw-p (03:01 539654) /lib/libutil-2.3.2.so
4019e000 (4 KB) rw-p (00:00 0)
4019f000 (2048 KB) r--p (03:03 196819) /usr/lib/locale/locale-archive
4039f000 (88 KB) r-xp (03:03 115375) /usr/lib/perl5/site_perl/5.8.2/i686-linux/auto/DBI/DBI.so
403b5000 (4 KB) rw-p (03:03 115375) /usr/lib/perl5/site_perl/5.8.2/i686-linux/auto/DBI/DBI.so
403b6000 (60 KB) r-xp (03:03 929331) /usr/lib/perl5/site_perl/5.8.2/i686-linux/auto/DBD/mysql/mysql.so
403c5000 (8 KB) rw-p (03:03 929331) /usr/lib/perl5/site_perl/5.8.2/i686-linux/auto/DBD/mysql/mysql.so
403c7000 (116 KB) r-xp (03:03 49195) /usr/lib/mysql/libmysqlclient.so.12.0.0
403e4000 (16 KB) rw-p (03:03 49195) /usr/lib/mysql/libmysqlclient.so.12.0.0
403e8000 (48 KB) r-xp (03:03 65251) /usr/lib/libz.so.1.1.4
403f4000 (8 KB) rw-p (03:03 65251) /usr/lib/libz.so.1.1.4
403f6000 (56 KB) r-xp (03:03 848128) /usr/lib/perl5/5.8.2/i686-linux/auto/Storable/Storable.so
40404000 (4 KB) rw-p (03:03 848128) /usr/lib/perl5/5.8.2/i686-linux/auto/Storable/Storable.so
40405000 (32 KB) r-xp (03:03 65367) /usr/lib/libkakasi.so.2.1.0
4040d000 (32 KB) rw-p (03:03 65367) /usr/lib/libkakasi.so.2.1.0
40415000 (232 KB) rw-p (00:00 0)
4044f000 (24 KB) r-xp (03:03 945619) /usr/lib/perl5/5.8.2/i686-linux/auto/Encode/Encode.so
40455000 (4 KB) rw-p (03:03 945619) /usr/lib/perl5/5.8.2/i686-linux/auto/Encode/Encode.so
40456000 (12 KB) r-xp (03:03 782934) /usr/lib/perl5/5.8.2/i686-linux/auto/IPC/SysV/SysV.so
40459000 (4 KB) rw-p (03:03 782934) /usr/lib/perl5/5.8.2/i686-linux/auto/IPC/SysV/SysV.so
4045a000 (12 KB) r-xp (03:03 798327) /usr/lib/perl5/5.8.2/i686-linux/auto/Filter/Util/Call/Call.so
4045d000 (4 KB) rw-p (03:03 798327) /usr/lib/perl5/5.8.2/i686-linux/auto/Filter/Util/Call/Call.so
4045e000 (24 KB) r--s (03:03 407433) /usr/lib/gconv/gconv-modules.cache
40464000 (12 KB) r-xp (03:03 521487) /usr/lib/perl5/5.8.2/i686-linux/auto/Time/HiRes/HiRes.so
40467000 (4 KB) rw-p (03:03 521487) /usr/lib/perl5/5.8.2/i686-linux/auto/Time/HiRes/HiRes.so
40468000 (48 KB) rw-s (00:04 786432) /SYSV00000000
40474000 (44 KB) r-xp (03:01 539638) /lib/libnss_files-2.3.2.so
4047f000 (4 KB) rw-p (03:01 539638) /lib/libnss_files-2.3.2.so
4048c000 (16 KB) r-xp (03:01 539636) /lib/libnss_dns-2.3.2.so
40490000 (4 KB) rw-p (03:01 539636) /lib/libnss_dns-2.3.2.so
42000000 (1216 KB) r-xp (03:01 637971) /lib/tls/libc-2.3.2.so
42130000 (12 KB) rw-p (03:01 637971) /lib/tls/libc-2.3.2.so
42133000 (12 KB) rw-p (00:00 0)
bfff8000 (32 KB) rwxp (00:00 0)
mapped: 16972 KB writable/private: 10140 KB shared: 72 KB
------- Linked against ###################
libcrypt.so.1 => /lib/libcrypt.so.1 (0x40023000)
libresolv.so.2 => /lib/libresolv.so.2 (0x40050000)
libm.so.6 => /lib/tls/libm.so.6 (0x40062000)
libdl.so.2 => /lib/libdl.so.2 (0x40084000)
libnsl.so.1 => /lib/libnsl.so.1 (0x40088000)
libgdbm.so.2 => /usr/lib/libgdbm.so.2 (0x4009d000)
libperl.so => /usr/lib/perl5/5.8.2/i686-linux/CORE/libperl.so (0x400a5000)
libutil.so.1 => /lib/libutil.so.1 (0x4019b000)
libc.so.6 => /lib/tls/libc.so.6 (0x42000000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
Re: [mp1] modules not shared across children. preloading or fork()
issue?
Posted by Stas Bekman <st...@stason.org>.
Daniel Wilson wrote:
> For referance the problem i am seeing, is that even though i have
> configured a load of modules to be preloaded, they are not being preloaded,
> or at least not being shared.
[Daniel talks about the thread he has started earlier in Dec
http://marc.theaimsgroup.com/?t=107197605300005&r=1&w=2]
> This happened after moving to a new server running the same redhat 9 but
> with different kernels and glibc's. I'm betting thats where the problem is.
> In the old server everything was shared for a good while.
>
> Has anyone had similar problems with RedHat 9?
Have you asked this question on the glibc list? mod_perl doesn't do anything
special to enable sharing and simply relies on the OS.
Is it possible that top(1) gets the information incorrectly? Since you are on
Linux you can use Apache::VMonitor as an alternative and easier way to see the
stats on memory usage.
As Perrin suggested earlier could it be a bug in a new kernel that you moved
to? What happens if you downgrade/upgrade your kernel?
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
--
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
Re: [mp1] modules not shared across children. preloading or fork()issue?
Posted by Stas Bekman <st...@stason.org>.
Daniel Wilson wrote:
> Problem solved. Turned out to be some kernel patches redhat has applyed to
> their kernel. It works fine with a vanilla kernel, litteraly all memory is
> reported as being shared between apache processes now like it was before.
Great. May be you could do a great service to your fellow RH/mp users and
report this problem to those in charge at RH...
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
--
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
Re: [mp1] modules not shared across children. preloading or fork()issue?
Posted by Daniel Wilson <we...@animelab.com>.
Problem solved. Turned out to be some kernel patches redhat has applyed to
their kernel. It works fine with a vanilla kernel, litteraly all memory is
reported as being shared between apache processes now like it was before.
Thanks for your help.
Danni
----- Original Message -----
From: "Perrin Harkins" <pe...@elem.com>
To: "Daniel Wilson" <we...@animelab.com>
Cc: <mo...@perl.apache.org>
Sent: Tuesday, January 06, 2004 9:54 PM
Subject: Re: [mp1] modules not shared across children. preloading or
fork()issue?
> On Mon, 2004-01-05 at 00:36, Daniel Wilson wrote:
> > This happened after moving to a new server running the same redhat 9
> > but with different kernels and glibc's.
>
> Are you absolutely positive that nothing else changed? Not the Perl
> build, not the apache build, not your code, not your config files?
>
> > 1. mod_perl is not loading modules before the fork()'s.
>
> Easy to verify. Dump %INC at the end of your startup.pl and it will
> show you what has been loaded.
>
> > ------- httpd.conf #####################
> >
> > PerlModule Apache::Registry Apache::RegistryLoader DBD::mysql
> > BSD::Resource DBI
> > PerlModule Storable Text::Kakasi Encode::Encoding Apache::DBI Carp
> > PerlModule IPC::Shareable Jcode IPC::SysV Encode Encode::Alias
> > Lingua::JA::Romaji
>
> There used to be a limit on how many modules you could load with
> PerlModule. I'm not sure if there still is. Try moving all of these
> into startup.pl instead.
>
> - Perrin
>
>
> --
> Reporting bugs: http://perl.apache.org/bugs/
> Mail list info: http://perl.apache.org/maillist/modperl.html
--
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html
Re: [mp1] modules not shared across children. preloading or fork()
issue?
Posted by Perrin Harkins <pe...@elem.com>.
On Mon, 2004-01-05 at 00:36, Daniel Wilson wrote:
> This happened after moving to a new server running the same redhat 9
> but with different kernels and glibc's.
Are you absolutely positive that nothing else changed? Not the Perl
build, not the apache build, not your code, not your config files?
> 1. mod_perl is not loading modules before the fork()'s.
Easy to verify. Dump %INC at the end of your startup.pl and it will
show you what has been loaded.
> ------- httpd.conf #####################
>
> PerlModule Apache::Registry Apache::RegistryLoader DBD::mysql
> BSD::Resource DBI
> PerlModule Storable Text::Kakasi Encode::Encoding Apache::DBI Carp
> PerlModule IPC::Shareable Jcode IPC::SysV Encode Encode::Alias
> Lingua::JA::Romaji
There used to be a limit on how many modules you could load with
PerlModule. I'm not sure if there still is. Try moving all of these
into startup.pl instead.
- Perrin
--
Reporting bugs: http://perl.apache.org/bugs/
Mail list info: http://perl.apache.org/maillist/modperl.html