You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Kurt Hansen <kh...@charityweb.net> on 2005/02/11 16:57:18 UTC

[mp2] Memory not being shared

Hello,

I'm having problems getting modules loaded at Apache start to be shared 
across the children. Here's the output I get from Apache::VMonitor (I've 
editted down to just 2 children):

    Size  Share VSize   RSS   M Elapsed LastReq Srvd
     ----- ----- -----  ---- --  ------ ------ ----
12: 89.5M  4.2M 89.5M 70.9M _  0.000s  0.000s    0
 8: 89.5M  4.2M 89.5M 70.9M _  0.000s  0.000s    0

Total:     1501392K (1432M) size, 1435471K (1369M) approx real size 
(-shared)

Both the totals and the individual numbers imply very little memory is 
being shared. As I understand the output from VMonitor, only 4.2M out of 
89.5M are being shared. (Yes, Apache is awfully big, that's my next step 
in optimization.)

I'm loading the vast majority of my modules in a startup.pl file, so I 
don't understand why they aren't being shared. Obviously, I am missing 
something. Any pointers on where I should look next would be very 
helpful. Here is the information about my system:

Fedora Core 2
perl 5.8.5 (threaded)
Apache 2.0.52 (pre-fork -- config params below)
mp2 2.0.0-RC4 (DSO -- config params below)
apreq2 (2/10/05 version)

Relevant parts of httpd.conf (CWApache::Apache2Handler is use'd in 
startup.pl):

 LoadModule perl_module modules/mod_perl.so
 LoadModule apreq_module       modules/mod_apreq.so
 ...
 PerlRequire /usr/local/apache2/mp/conf/startup.pl
 ...
 <Location / >
       SetHandler perl-script
       PerlResponseHandler  CWApache::Apache2Handler
 </Location>

Config params for apache and mod_perl:

 ./configure --prefix=/usr/local/apache2/mp --with-mpm=prefork 
--disable-cgi --disable-userdir --enable-auth-dbm --enable-speling

 perl Makefile.PL MP_APXS=/usr/local/apache2/mp/bin/apxs MP_INST_APACHE2=1

Thank you for any guidance!

Take care,

Kurt Hansen
khansen@charityweb.net

Re: [mp2] Memory not being shared

Posted by "Richard F. Rebel" <rr...@whenu.com>.
Stas,

As far as the kernel issue goes, you are right there has been changes.
The meaning of 'SHARE' pages reported by the Linux kernel has changed
over time.  This does not necessarily mean your processes aren't sharing
via COW pages, just in how the kernel reports them.

I posted a response from the linux kernel mailing list a short time ago
that goes into detail.

Best,

Richard F. Rebel

On Fri, 2005-02-11 at 23:04 -0500, Stas Bekman wrote:
> I plugged vmonitor in the mp2 test suite (which loads a lot of modules) 
> and the conclusions are as following:
> 
> perl           Size Share VSize   Rss
> non-threaded: 24.8M  8.3M 24.8M 20.6M
> threaded    : 30.4M  8.6M 30.4M 26.2M
> 
> both running perl 5.8.6 under prefork mpm.
> 
> so you can see that while there is a difference is the total memory usage 
> (but not all modules are loaded for the non-threaded case), there is not 
> much difference between the two when it comes to shared memory.
> 
> While trying to shrink the test suite to find out where the unshared 
> memory comes from I've discovered that 10.2MB come from loading 
> Apache::TestRequest. I've verified that offline with:
> 
> perl-5.8.6-ithread bloat.pl 'use blib; use Apache::TestRequest'
> use blib; use Apache::TestRequest added 10.2M
> 
> so *all* the memory used by A-TR (and modules it has loaded) was 
> completely unshared. I don't know why that happens. A-TR is preloaded at 
> the server startup. It's probably the case where perl has tons of 
> variables which change, making all pages unshared.
> 
> I've run a quick test with mp1's test suite, and it seems to be quite 
> similar (not much sharing)
> 
> So it's quite possible that this is a kernel issue (Linux 
> rabbit.stason.org 2.6.8.1-12mdk here) or something that has to do with libc)
> 
-- 
Richard F. Rebel <rr...@whenu.com>
WhenU.com

Re: [mp2] Memory not being shared

Posted by Stas Bekman <st...@stason.org>.
I plugged vmonitor in the mp2 test suite (which loads a lot of modules) 
and the conclusions are as following:

perl           Size Share VSize   Rss
non-threaded: 24.8M  8.3M 24.8M 20.6M
threaded    : 30.4M  8.6M 30.4M 26.2M

both running perl 5.8.6 under prefork mpm.

so you can see that while there is a difference is the total memory usage 
(but not all modules are loaded for the non-threaded case), there is not 
much difference between the two when it comes to shared memory.

While trying to shrink the test suite to find out where the unshared 
memory comes from I've discovered that 10.2MB come from loading 
Apache::TestRequest. I've verified that offline with:

perl-5.8.6-ithread bloat.pl 'use blib; use Apache::TestRequest'
use blib; use Apache::TestRequest added 10.2M

so *all* the memory used by A-TR (and modules it has loaded) was 
completely unshared. I don't know why that happens. A-TR is preloaded at 
the server startup. It's probably the case where perl has tons of 
variables which change, making all pages unshared.

I've run a quick test with mp1's test suite, and it seems to be quite 
similar (not much sharing)

So it's quite possible that this is a kernel issue (Linux 
rabbit.stason.org 2.6.8.1-12mdk here) or something that has to do with libc)

-- 
__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com

Re: [mp2] Memory not being shared

Posted by Kurt Hansen <kh...@charityweb.net>.
Hello,

Richard F. Rebel wrote:

>I sent a message just prior to this that contained a forward from the
>linux kernel mailing list that explains this further, but:
>
>The method you are using to check changed with 2.6.10 and may readily
>account for the changes you are seeing in your figures.
>
>In the end, you should not be using '/proc/<pid>/statm' shared column
>derived figures to determine how shared apache is with it's children's
>via COW pages.  That figure in 2.4 was polluted with other data, and in
>2.6 it has changed.  In 2.6.10 and later, and for the "foreseeable"
>future it will NOT contain private pages shared with forked children.
>
>This leaves us at a loose end though.
>  
>
Thank you, Richard, Stas, and Perrin!!!

I find it absolutely incredible that, considering the depth of the 
problem, I got a definitive answer within a couple of days. And it is a 
definitive answer, i.e. don't use measures of shared memory to optimize 
on >= 2.6.10 Linux because memory shared between a parent and child is 
not counted. At least, for now.

I now know that down this path lies ghosts & time-eating demons. Do you 
know how many fruitless hours of work you have just saved me? And 
others? :-)

I think the speed with which this got answered is another example of the 
power of the Open Source methodology over a proprietary method of 
development.

I think there's a Degrees of Separation thing here, too, since getting 
over to the kernel network was key. And, the "Grokking Memory.." thread 
added to the mix. (Though, the "six degrees of separation" research was 
de-bunked a few years back...I think it is more than six.)

Take care,

Kurt
khansen@charityweb.net



Re: [mp2] Memory not being shared

Posted by "Richard F. Rebel" <rr...@whenu.com>.
Hi Kurt,

I sent a message just prior to this that contained a forward from the
linux kernel mailing list that explains this further, but:

The method you are using to check changed with 2.6.10 and may readily
account for the changes you are seeing in your figures.

In the end, you should not be using '/proc/<pid>/statm' shared column
derived figures to determine how shared apache is with it's children's
via COW pages.  That figure in 2.4 was polluted with other data, and in
2.6 it has changed.  In 2.6.10 and later, and for the "foreseeable"
future it will NOT contain private pages shared with forked children.

This leaves us at a loose end though.

Best,

On Fri, 2005-02-11 at 17:34 -0500, Kurt Hansen wrote:
> Perrin Harkins wrote:
> 
> >
> >I wonder if it has something to do with SMP.  It would be good to try it
> >on a kernel that is not compiled for SMP.  It might be worth compiling
> >your own kernel to test it.
> >
> >Running a non-threaded perl is always a good idea if you are not using
> >threads.  Performance will be better too.mercy 
> >  
> >
> I re-booted into the 2.6.10-1.12_FC2 (non-SMP) kernel. No difference.
> 
> Then, I tried going back to an earlier kernel, 2.6.8-1.521smp to be 
> exact. In that case, the shared memory jumped from 4M to 15M! That still 
> leaves 55M unshared, but the kernel seems to be part of the equation. 
> Thanks for the clue, Perrin! I'm not sure if there is a problem with 
> 2.6.10 or if 2.6.8 was the kernel on the machine when I originally 
> compiled perl.
> 
> Has anyone else seen problems with the 2.6.10 kernel?
> 
> I guess my next step is to re-compile perl and see where that leads me.
> 
> Take care,
> 
> Kurt Hansen
> khansen@charityweb.net
-- 
Richard F. Rebel <rr...@whenu.com>
WhenU.com

Re: [mp2] Memory not being shared

Posted by Kurt Hansen <kh...@charityweb.net>.
Perrin Harkins wrote:

>
>I wonder if it has something to do with SMP.  It would be good to try it
>on a kernel that is not compiled for SMP.  It might be worth compiling
>your own kernel to test it.
>
>Running a non-threaded perl is always a good idea if you are not using
>threads.  Performance will be better too.mercy 
>  
>
I re-booted into the 2.6.10-1.12_FC2 (non-SMP) kernel. No difference.

Then, I tried going back to an earlier kernel, 2.6.8-1.521smp to be 
exact. In that case, the shared memory jumped from 4M to 15M! That still 
leaves 55M unshared, but the kernel seems to be part of the equation. 
Thanks for the clue, Perrin! I'm not sure if there is a problem with 
2.6.10 or if 2.6.8 was the kernel on the machine when I originally 
compiled perl.

Has anyone else seen problems with the 2.6.10 kernel?

I guess my next step is to re-compile perl and see where that leads me.

Take care,

Kurt Hansen
khansen@charityweb.net

Re: [mp2] Memory not being shared

Posted by Perrin Harkins <pe...@elem.com>.
On Fri, 2005-02-11 at 16:23 -0500, Kurt Hansen wrote:
> What do y'all think would be my most fruitful next step, besides testing 
> out a new machine? Should I switch to static mp2? Non-threaded perl? 
> Eliminating perl modules to see if there is one causing the problem? Prayer?

I wonder if it has something to do with SMP.  It would be good to try it
on a kernel that is not compiled for SMP.  It might be worth compiling
your own kernel to test it.

Running a non-threaded perl is always a good idea if you are not using
threads.  Performance will be better too.

- Perrin


Re: [mp2] Memory not being shared

Posted by Kurt Hansen <kh...@charityweb.net>.
Perrin Harkins wrote:

>On Fri, 2005-02-11 at 12:54 -0500, Kurt Hansen wrote:
>  
>
>>Thanks for the help. I do preload almost all the modules in the 
>>startup.pl file. Here it is
>>    
>>
>
>This looks okay to me.  I have sometimes seen behavior like what you're
>experiencing on specific machines and not on others, while running
>identical code.  I never quite got to the bottom of it, but I assumed it
>had something to do with the kernel versions, or with the way top was
>reporting memory.  Can you try this on another machine?
>  
>
Thank you, Perrin, for your perspective. Though it is a bit 
disheartening. :-}

I can't try the exact setup on another machine quickly. But, I looked at 
the output from top on two other machines that have the same httpd.conf 
and startup.pl, but different versions of the kernel, mp2, apreq2, and 
hardware.

I'm seeing similar behavior on all three, though it is worst on the two 
systems with the same kernel. The two with linux 2.6.10-1.12_FC2smp both 
have about 4M shared. One with linux 2.6.8-1.521smp has 14M shared, 
still not as much as I would expect.

The main difference between the 4M systems is that one is using mp 
1.99_21 and the other mp 1.99_17. The 14M is using mp 1.99_17.

What do y'all think would be my most fruitful next step, besides testing 
out a new machine? Should I switch to static mp2? Non-threaded perl? 
Eliminating perl modules to see if there is one causing the problem? Prayer?

Take care,

Kurt Hansen
khansen@charityweb.net



Re: [mp2] Memory not being shared

Posted by Perrin Harkins <pe...@elem.com>.
On Fri, 2005-02-11 at 12:54 -0500, Kurt Hansen wrote:
> Thanks for the help. I do preload almost all the modules in the 
> startup.pl file. Here it is

This looks okay to me.  I have sometimes seen behavior like what you're
experiencing on specific machines and not on others, while running
identical code.  I never quite got to the bottom of it, but I assumed it
had something to do with the kernel versions, or with the way top was
reporting memory.  Can you try this on another machine?

- Perrin


Re: [mp2] Memory not being shared

Posted by Kurt Hansen <kh...@charityweb.net>.
Stas Bekman wrote:

> Kurt Hansen wrote:
>
>> I'm having problems getting modules loaded at Apache start to be 
>> shared across the children. Here's the output I get from 
>> Apache::VMonitor (I've editted down to just 2 children):
>>
>> Size Share VSize RSS M Elapsed LastReq Srvd
>> ----- ----- ----- ---- -- ------ ------ ----
>> 12: 89.5M 4.2M 89.5M 70.9M _ 0.000s 0.000s 0
>> 8: 89.5M 4.2M 89.5M 70.9M _ 0.000s 0.000s 0
>>
>> Total: 1501392K (1432M) size, 1435471K (1369M) approx real size 
>> (-shared)
>>
>> Both the totals and the individual numbers imply very little memory 
>> is being shared. As I understand the output from VMonitor, only 4.2M 
>> out of 89.5M are being shared. (Yes, Apache is awfully big, that's my 
>> next step in optimization.)
>>
>> I'm loading the vast majority of my modules in a startup.pl file, so 
>> I don't understand why they aren't being shared. Obviously, I am 
>> missing something. Any pointers on where I should look next would be 
>> very helpful. Here is the information about my system:
>>
>> Fedora Core 2
>> perl 5.8.5 (threaded)
>> Apache 2.0.52 (pre-fork -- config params below)
>> mp2 2.0.0-RC4 (DSO -- config params below)
>> apreq2 (2/10/05 version)
>>
>> Relevant parts of httpd.conf (CWApache::Apache2Handler is use'd in 
>> startup.pl):
>>
>> LoadModule perl_module modules/mod_perl.so
>> LoadModule apreq_module modules/mod_apreq.so
>> ...
>> PerlRequire /usr/local/apache2/mp/conf/startup.pl
>
>
> what's in that file? If you don't preload all your modules that 
> explains why you don't have the sharing.

Thanks for the help. I do preload almost all the modules in the 
startup.pl file. Here it is (I didn't include it at first because I 
thought it might be a bit long...):

use Apache2 ();

use lib qw(/www/comp/lib/perl5);

# enable if the mod_perl 1.0 compatibility is needed
use Apache::compat ();

# preload all mp2 modules
use ModPerl::MethodLookup;
ModPerl::MethodLookup::preload_all_modules();

use ModPerl::Util (); #for CORE::GLOBAL::exit

use Apache::RequestRec ();
use Apache::RequestIO ();
use Apache::RequestUtil ();

use Apache::ServerRec ();
use Apache::ServerUtil ();
use Apache::Connection ();
use Apache::Log ();

use APR::Table ();

use ModPerl::Registry ();

use Apache::Const -compile => qw(:common :http);
use APR::Const -compile => ':common';

use Apache::DBI (); #Added on 11/21/04 KH
$Apache::DBI::DEBUG = 1;

use Apache::AuthDBI ();
use Apache::Request ();
use Apache::Cookie ();
use Apache::Upload ();


use Apache::URI;
use Apache::Session;
use Apache::Session::File ();
use Crypt::HCE_MD5 ();
use HTTP::Request::Common ();
use LWP::UserAgent ();
use Crypt::RSA ();
use Crypt::RSA::Key ();
use PDF::API2;
use PDF::API2::CoreFont;
use Email::Valid ();
use Tie::IxHash ();

use Net::SSLeay qw(get_https post_https sslcat make_headers make_form);

use lib "/www/comp/lib/perl5";
use Stats::ResponseStats ();
use Page::FormField ();
use Convert::Cwebutils ();
use Email::Base ();
use Monitor::TransCheck ();
use Session::CW();
use Transaction::Process ();
use Request::CW ();
use Request::CWCookie ();

# New database and modules
use lib "/www/comp/lib/perl5/donor_modules";
use Transaction ();
use Database ();

# Added 2/10/05 -- KH
use Apache::Scoreboard ();
use Apache::VMonitor();
$Apache::VMonitor::Config{refresh} = 0;
$Apache::VMonitor::Config{verbose} = 0;
$Apache::VMonitor::Config{system} = 1;
$Apache::VMonitor::Config{apache} = 1;
$Apache::VMonitor::Config{procs} = 1;
$Apache::VMonitor::Config{mount} = 1;
$Apache::VMonitor::Config{fs_usage} = 1;
$Apache::VMonitor::Config{apache_sort_by} = 'size';

$Apache::VMonitor::PROC_REGEX = join "\|", qw(httpd mysql squid);

use CWApache::Apache2Handler ();

1;

>
>> <Location / >
>> SetHandler perl-script
>> PerlResponseHandler CWApache::Apache2Handler
>> </Location>
>>
>> Config params for apache and mod_perl:
>>
>> ./configure --prefix=/usr/local/apache2/mp --with-mpm=prefork 
>> --disable-cgi --disable-userdir --enable-auth-dbm --enable-speling
>>
>> perl Makefile.PL MP_APXS=/usr/local/apache2/mp/bin/apxs 
>> MP_INST_APACHE2=1
>
>
> for the future, the bug report guidelines can be found here:
> http://perl.apache.org/bugs/


Oh, OK. I didn't perceive this to be a bug in mod_perl. More of a bug in 
the way I'm using mod_perl. :-) In any case, here is the bug report:

-------------8<---------- Start Bug Report ------------8<----------
1. Problem Description:

I'm having problems getting modules loaded at Apache start to be shared 
across the children. Here's the output I get from Apache::VMonitor (I've 
editted down tojust 2 children):

Size Share VSize RSS M Elapsed LastReq Srvd
----- ----- ----- ---- -- ------- ------- ----
12: 89.5M 4.2M 89.5M 70.9M _ 0.000s 0.000s 0
8: 89.5M 4.2M 89.5M 70.9M _ 0.000s 0.000s 0

Total: 1501392K (1432M) size, 1435471K (1369M) approx real size (-shared)

Both the totals and the individual numbers imply very little memory is 
being shared. As I understand the output from VMonitor, only 4.2M out of 
89.5M are being shared. (Yes, Apache is awfully big, that's my next step 
in optimization.)

I'm loading the vast majority of my modules in a startup.pl file, so I 
don't understand why they aren't being shared.

2. Used Components and their Configuration:

*** mod_perl version 1.999021

*** using /usr/local/src/mod_perl-2.0.0-RC4/lib/Apache/BuildConfig.pm

*** Makefile.PL options:
MP_APR_LIB => aprext
MP_APXS => /usr/local/apache2/mp/bin/apxs
MP_COMPAT_1X => 1
MP_GENERATE_XS => 1
MP_INST_APACHE2 => 1
MP_LIBNAME => mod_perl
MP_USE_DSO => 1


*** /usr/local/apache2/mp/bin/httpd -V
Server version: Apache/2.0.52
Server built: Feb 10 2005 11:14:27
Server's Module Magic Number: 20020903:9
Architecture: 32-bit
Server compiled with....
-D APACHE_MPM_DIR="server/mpm/prefork"
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D HTTPD_ROOT="/usr/local/apache2/mp"
-D SUEXEC_BIN="/usr/local/apache2/mp/bin/suexec"
-D DEFAULT_PIDLOG="logs/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_LOCKFILE="logs/accept.lock"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"


*** (apr|apu)-config linking info

-L/usr/local/apache2/mp/lib -lapr-0 -lrt -lm -lcrypt -lnsl -lpthread -ldl
-L/usr/local/apache2/mp/lib -laprutil-0 -lgdbm -ldb-4.2 -lexpat



*** /usr/local/bin/perl -V
Summary of my perl5 (revision 5 version 8 subversion 5) configuration:
Platform:
osname=linux, osvers=2.6.7-1.494.2.2smp, archname=i686-linux-thread-multi-ld
uname='linux pax.charityweb.net 2.6.7-1.494.2.2smp #1 smp tue aug 3 
09:59:49 edt 2004 i686 i686 i386 gnulinux '
config_args=''
hint=previous, useposix=true, d_sigaction=define
usethreads=define use5005threads=undef useithreads=define 
usemultiplicity=define
useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=define
usemymalloc=y, bincompat5005=undef
Compiler:
cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS 
-fno-strict-aliasing -pipe -I/usr/local/include -march=i686 -mcpu=i686 
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
optimize='-O2',
cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS 
-fno-strict-aliasing -pipe -I/usr/local/include -march=i686 -mcpu=i686 
-I/usr/include/gdbm -D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS 
-fno-strict-aliasing -pipe -I/usr/local/include -march=i686 -mcpu=i686 
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm'
ccversion='', gccversion='3.3.3 20040412 (Red Hat Linux 3.3.3-7)', 
gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='long double', nvsize=12, Off_t='off_t', 
lseeksize=8
alignbytes=4, prototype=define
Linker and Libraries:
ld='gcc', ldflags =' -L/usr/local/lib'
libpth=/lib /usr/lib /usr/local/lib
libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
libc=/lib/libc-2.3.3.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.3.3'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'


Characteristics of this binary (from libperl):
Compile-time options: MULTIPLICITY USE_ITHREADS USE_LONG_DOUBLE 
USE_LARGE_FILES PERL_IMPLICIT_CONTEXT
Built under linux
Compiled at Aug 11 2004 12:15:15
%ENV:
PERL_LWP_USE_HTTP_10="1"
@INC:
/usr/local/lib/perl5/5.8.5/i686-linux-thread-multi-ld
/usr/local/lib/perl5/5.8.5
/usr/local/lib/perl5/site_perl/5.8.5/i686-linux-thread-multi-ld
/usr/local/lib/perl5/site_perl/5.8.5
/usr/local/lib/perl5/site_perl
.

*** Packages of interest status:

Apache::Request: 2.05
CGI : 3.05
LWP : 5.800
mod_perl : 1.999021


3. This is the core dump trace: (if you get a core dump):

Not applicable

This report was generated by t/REPORT on Fri Feb 11 17:34:42 2005 GMT.

-------------8<---------- End Bug Report --------------8<----------


Re: [mp2] Memory not being shared

Posted by Stas Bekman <st...@stason.org>.
Kurt Hansen wrote:
> Hello,
> 
> I'm having problems getting modules loaded at Apache start to be shared 
> across the children. Here's the output I get from Apache::VMonitor (I've 
> editted down to just 2 children):
> 
>    Size  Share VSize   RSS   M Elapsed LastReq Srvd
>     ----- ----- -----  ---- --  ------ ------ ----
> 12: 89.5M  4.2M 89.5M 70.9M _  0.000s  0.000s    0
> 8: 89.5M  4.2M 89.5M 70.9M _  0.000s  0.000s    0
> 
> Total:     1501392K (1432M) size, 1435471K (1369M) approx real size 
> (-shared)
> 
> Both the totals and the individual numbers imply very little memory is 
> being shared. As I understand the output from VMonitor, only 4.2M out of 
> 89.5M are being shared. (Yes, Apache is awfully big, that's my next step 
> in optimization.)
> 
> I'm loading the vast majority of my modules in a startup.pl file, so I 
> don't understand why they aren't being shared. Obviously, I am missing 
> something. Any pointers on where I should look next would be very 
> helpful. Here is the information about my system:
> 
> Fedora Core 2
> perl 5.8.5 (threaded)
> Apache 2.0.52 (pre-fork -- config params below)
> mp2 2.0.0-RC4 (DSO -- config params below)
> apreq2 (2/10/05 version)
> 
> Relevant parts of httpd.conf (CWApache::Apache2Handler is use'd in 
> startup.pl):
> 
> LoadModule perl_module modules/mod_perl.so
> LoadModule apreq_module       modules/mod_apreq.so
> ...
> PerlRequire /usr/local/apache2/mp/conf/startup.pl

what's in that file? If you don't preload all your modules that explains 
why you don't have the sharing.

> <Location / >
>       SetHandler perl-script
>       PerlResponseHandler  CWApache::Apache2Handler
> </Location>
> 
> Config params for apache and mod_perl:
> 
> ./configure --prefix=/usr/local/apache2/mp --with-mpm=prefork 
> --disable-cgi --disable-userdir --enable-auth-dbm --enable-speling
> 
> perl Makefile.PL MP_APXS=/usr/local/apache2/mp/bin/apxs MP_INST_APACHE2=1

for the future, the bug report guidelines can be found here:
http://perl.apache.org/bugs/


-- 
__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com