You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Bill Moseley <mo...@hank.org> on 2000/11/10 01:24:31 UTC
Re: pre-loaded modules on Solaris
Hi Mike,
I've cc'd the mod_perl list for other Solaris users to consider.
At 10:49 AM 11/01/00 -0800, Michael Blakeley wrote:
>I saw a significant benefit from pre-loading modules. Let's take a
>test case where we share via startup.pl:
>
>Without preloading:
>
># tellshared.pl web
> count vsz rss kB %
> 8 146304 67784 78520 54
>
>With the "use" section above:
>
># tellshared.pl web
> count vsz rss kB %
> 8 132672 17032 115640 87
>
>'rss' is the resident set size - that is, the amount of actual RAM in
>use. The 'vsz' tells the size of the virtual address space - the swap
>in use. The kB column shows the difference (ie, the "saved RAM" via
>page sharing) and the % shows the %-shared.
I'm not clear you can measure the shared memory space that way. I don't
really understand the memory system much, but here's a paper that I found
interesting:
http://www.sun.com/solutions/third-party/global/SAS/pdf/vmsizing.pdf
"The virtual address size of a process often bares no resemblance to the
amount of memory a process is using because it contains all of the
unallocated memory, libraries, shared memory and sometimes hardware devices
(in the case of XSun).
"The RSS figure is a measure of the amount of physical memory mapped into a
process, but often there is more than one copy of the process running, and
a large proportion of a process is shared with another.
"MemTool provides a mechanism for getting a detailed look at a processes
memory utilization. MemTool can show how much memory is in-core, how much
of that is shared, and hence how much private memory a process has. The
pmem command (or /usr/proc/bin/pmap -x in Solaris 2.6) can be used to show
the memory utilization of a single process."
Now, with a simple "hello world" mod_perl handler that loaded a bunch of
modules I did see that pre-loading modules reduces memory usage -- both in
looking at ps output, and with the pmap program, even after a number of
requests. This is consistent with what you commented on above.
I'm repeating, but I found with a real-world application that sharing the
modules ended up using quite a bit more "private" memory. I don't know if
that's only an issue with my specific OS, or with how my specific
application is running.
Here's ps output with pre-loaded modules. On these tests I'm running
Apache 1.3.12 with mod_perl 1.24 static, but everything else is DSO. I've
got maxclients set to one so there's only the parent and one child.
I'm pre-loading modules in a <perl> section here:
S USER PID PPID %CPU %MEM VSZ RSS STIME TIME COMMAND
S lii 318 1 0.0 0.3 8376 5464 15:50:38 0:00 httpd.mo
S lii 319 318 0.8 1.1 24720 21288 15:50:38 0:05 httpd.mo
And now without pre-loaded modules:
S USER PID PPID %CPU %MEM VSZ RSS STIME TIME COMMAND
S lii 1260 1 0.0 0.2 4392 3552 15:56:25 0:00 httpd.mo
S lii 1261 1260 0.9 0.6 14592 12304 15:56:25 0:05 httpd.mo
And here's comparing the totals returned by the pmap program that should
detail shared and private memory (according to the paper cited above).
Address Kbytes Resident Shared Private
-------- ------ -------- ------ -------
total Kb 24720 22720 3288 19432 <<< pre-loaded modules
total Kb 14592 12976 3096 9880 <<< not pre-loaed modules.
Indeed there's a tiny bit more shared memory in the pre-loaded Apache, but
the amount of "private" memory is significantly higher, too. Ten megs a
child will add up. It doesn't really make sense to me, but that's what
pmap is showing.
Maybe this isn't that interesting. Anyway, I'll try a non DSO Apache and
see if it makes a difference, and also try with an Apache that forks off
more clients than just one, but I can't imagine that making a difference.
Later,
Bill Moseley
mailto:moseley@hank.org
Re: pre-loaded modules on Solaris
Posted by John Siracusa <si...@mindspring.com>.
On 11/10/00 10:15 AM, John Hughes wrote:
> A guess: When pmap says "shared" it means stuff that realy shared.
> I.E. it's not counting stuff that is marked copy-on-write, which is
> shared UNTIL YOU TRY TO MODIFY IT.
Is there any way to measure the size of the shared copy-on-write pages on
Solaris? It seems insane that everyone just "assumes" that all this
copy-on-write goodness is happening, despite (seemingly) no way to measure
it. I'll admit, it's always been my assumption too, but I'd really like
some hard numbers to back me up.
-John
RE: pre-loaded modules on Solaris
Posted by John Hughes <jo...@Calva.COM>.
> At 06:11 AM 11/10/00 -0500, barries wrote:
> >> Address Kbytes Resident Shared Private
> >> -------- ------ -------- ------ -------
> >> total Kb 24720 22720 3288 19432 <<< pre-loaded modules
> >> total Kb 14592 12976 3096 9880 <<< not pre-loaed modules.
>
> When looking at pmap it looks like the main difference in "private" memory
> usage is in the heap. I'm not clear why the heap would end up so much
> bigger when pre-loading modules.
A guess: When pmap says "shared" it means stuff that realy shared.
I.E. it's not counting stuff that is marked copy-on-write, which is
shared UNTIL YOU TRY TO MODIFY IT.
Using pre-loaded modules you will get some things (XS) mmaped in as
shared, read-only and a whole bunch of stuff loaded into the heap.
this is "shared" when you fork, until you change it.
The trick is that most of the perl "bytecode" doesn't get modified after
it's loaded, so it'll stay "shared".
--
John Hughes <jo...@Calva.COM>,
CalvaEDI SA. Tel: +33-1-4313-3131
66 rue du Moulin de la Pointe, Fax: +33-1-4313-3139
75013 PARIS.
Re: pre-loaded modules on Solaris
Posted by Stas Bekman <st...@stason.org>.
> At 06:11 AM 11/10/00 -0500, barries wrote:
> >> Address Kbytes Resident Shared Private
> >> -------- ------ -------- ------ -------
> >> total Kb 24720 22720 3288 19432 <<< pre-loaded modules
> >> total Kb 14592 12976 3096 9880 <<< not pre-loaed modules.
>
> >Stupid question, probably, but when running the non-pre-loaded version,
> >are you sure all the same modules are being loaded?
>
> Yes. According to perl-status, anyway. Some modules are loaded into the
> parent, of course, because of mod_perl. But when not pre-loading I start
> the server, look at perl-status and then made some requests and looked
> again to see what was loaded. The difference is what modules I'm use'ing
> in my test.
>
> >I'm wondering if there's
> >some set of modules that, for some reason, isn't being loaded by the
> >sequence of requests you're firing against all of your httpds to
> >get the servers "warmed up" to represent real-life state.
>
> When looking at pmap it looks like the main difference in "private" memory
> usage is in the heap. I'm not clear why the heap would end up so much
> bigger when pre-loading modules.
>
> Unfortunately, Linux doesn't seem to have the same reporting abilities as
> Solaris, but using /proc/<pid>/statm to show shared and private memory
> under these same test showed that pre-loading was a big win. So it seems
> like a Solaris issue.
Uhm, http://perl.apache.org/guide/performance.html#Sharing_Memory which
uses extensively libgtop library, which is available for Solaris...
>
>
>
>
> Bill Moseley
> mailto:moseley@hank.org
>
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://apachetoday.com http://jazzvalley.com
http://singlesheaven.com http://perlmonth.com perl.org apache.org
Re: pre-loaded modules on Solaris
Posted by Bill Moseley <mo...@hank.org>.
At 06:11 AM 11/10/00 -0500, barries wrote:
>> Address Kbytes Resident Shared Private
>> -------- ------ -------- ------ -------
>> total Kb 24720 22720 3288 19432 <<< pre-loaded modules
>> total Kb 14592 12976 3096 9880 <<< not pre-loaed modules.
>Stupid question, probably, but when running the non-pre-loaded version,
>are you sure all the same modules are being loaded?
Yes. According to perl-status, anyway. Some modules are loaded into the
parent, of course, because of mod_perl. But when not pre-loading I start
the server, look at perl-status and then made some requests and looked
again to see what was loaded. The difference is what modules I'm use'ing
in my test.
>I'm wondering if there's
>some set of modules that, for some reason, isn't being loaded by the
>sequence of requests you're firing against all of your httpds to
>get the servers "warmed up" to represent real-life state.
When looking at pmap it looks like the main difference in "private" memory
usage is in the heap. I'm not clear why the heap would end up so much
bigger when pre-loading modules.
Unfortunately, Linux doesn't seem to have the same reporting abilities as
Solaris, but using /proc/<pid>/statm to show shared and private memory
under these same test showed that pre-loading was a big win. So it seems
like a Solaris issue.
Bill Moseley
mailto:moseley@hank.org
Re: pre-loaded modules on Solaris
Posted by barries <ba...@slaysys.com>.
On Thu, Nov 09, 2000 at 04:24:31PM -0800, Bill Moseley wrote:
> And here's comparing the totals returned by the pmap program that should
> detail shared and private memory (according to the paper cited above).
>
> Address Kbytes Resident Shared Private
> -------- ------ -------- ------ -------
> total Kb 24720 22720 3288 19432 <<< pre-loaded modules
> total Kb 14592 12976 3096 9880 <<< not pre-loaed modules.
>
> Indeed there's a tiny bit more shared memory in the pre-loaded Apache, but
> the amount of "private" memory is significantly higher, too. Ten megs a
> child will add up. It doesn't really make sense to me, but that's what
> pmap is showing.
Stupid question, probably, but when running the non-pre-loaded version,
are you sure all the same modules are being loaded? I'm wondering if there's
some set of modules that, for some reason, isn't being loaded by the
sequence of requests you're firing against all of your httpds to
get the servers "warmed up" to represent real-life state.
- Barrie