You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Torsten Foertsch <to...@gmx.net> on 2005/05/11 13:27:52 UTC
Apache::SizeLimit?
Hi,
as I have learned from the "Debugging memory allocation" thread
Apache::SizeLimit can be used to kill a child if it's unshared part grows to
big. To measure this unshared part on linux /proc/PID/statm is used. I was in
doubt that the simple difference between the total number of pages and the
shared pages really reflect the unshared part. So, I tried out some things:
First, in httpd.conf I allocate a noticeable amount of memory:
<Perl>
$My::Klaus="x"x10000000;
</Perl>
This memory is shared among all children due to linux' copy on write.
Now /proc/PID/statm for the master and a child say:
master: 5254 4190 2089 71 0 3094 0
child: 5272 4208 2107 71 0 3094 0
When I write to $My::Klaus from within a child I expect the total size to
remain the same and the shared portion to drop considerably (actually ~2500
pages which is not possible since it is only 2107).
My first try was to assign simply a new string: $My::Klaus="y"x10000000;
Now /proc/PID/statm shows for this child:
7749 6718 2107 71 0 5571 0
Not what I have expected. The shared portion remains but the total size grows
by 2477 pages. Well, perl allocates simply a new string. Hence the total
process size grows.
My second try was to modify the string in place: $My::Klaus=~tr/x/y/;
Now /proc/PID/statm shows:
5307 4276 2107 71 0 3129 0
Nothing has changed much. Particularly the number of shared pages remains
exactly the same.
To check that $My::Klaus is really copied on write I have checked the total
used and free space shown in /proc/meminfo. The figures show exactly what is
to be expected. The used memory grows by the amount needed to hold a second
copy of $My::Klaus.
Hence, killing an apache child when the unshared part grows beyond a certain
limit do on linux NOT account the copy on write pages that get unshared.
As I have heard libgtop also only provides the information from /proc. That
means do also not account the copy on write effect. Is that information
right?
Is there a way on linux to get the amount of pages shared among processes by
copy on write?
Torsten
Re: Apache2::Status, B::TerseSize
Posted by Stas Bekman <st...@stason.org>.
Matthew Berk wrote:
> Trying to track down a bit o' memory bloat using the Symbol Table dump
> in /perl-status and B::TerseSize. Running into some problems....
>
> Error: [Wed May 11 15:38:54 2005] [error] [client ::1] Use of
> uninitialized value in length at /Library/Perl/5.8.6/darwin-thread-
> multi-2level/Apache2/Status.pm line 562.\n, referer: http://localhost/
> perl-status/?symdump
>
> (The page actually starts with "Memory Usage for package main\n
> \nTotals: 67848 bytes | 0 OPs", but then there's a server error.)
Yup, we know about this issue, Matthew. I suspect it's an issue with
B::TerseSize. I didn't have a chance to look at it yet. If you have the
time, please investigate why does it fail. I'd start by checking whether
this Apache::Status feature works under mp1 with the same perl version.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Apache2::Status, B::TerseSize
Posted by Matthew Berk <ma...@openlist.com>.
Trying to track down a bit o' memory bloat using the Symbol Table
dump in /perl-status and B::TerseSize. Running into some problems....
Error: [Wed May 11 15:38:54 2005] [error] [client ::1] Use of
uninitialized value in length at /Library/Perl/5.8.6/darwin-thread-
multi-2level/Apache2/Status.pm line 562.\n, referer: http://localhost/
perl-status/?symdump
(The page actually starts with "Memory Usage for package main\n
\nTotals: 67848 bytes | 0 OPs", but then there's a server error.)
Perl Version: 5.8.6
Apache: 2.0.54
OS: SuSe
MP: 2.0.0-RC6
Setup in httpd.conf is:
PerlModule Apache2::compat
<Location /perl-status>
SetHandler perl-script
PerlHandler Apache2::Status
</Location>
PerlSetVar StatusOptionsAll On
PerlSetVar StatusTerse On
PerlSetVar StatusTerseSize On
PerlSetVar StatusTerseSizeMainSummary On
PerlModule B::TerseSize
Any thoughts?
Thanks,
Matthew
Re: Apache::SizeLimit?
Posted by Stas Bekman <st...@stason.org>.
Perrin Harkins wrote:
> On Wed, 2005-05-11 at 13:20 -0400, Stas Bekman wrote:
>
>>It's not that SizeLimit and GTop no longer work, it's the recent linux
>>kernels that have stopped maitaining certain information about shared
>>memory. So the sharing happens, but there is no way to measure it.
>
>
> My impression was that our measurements of shared memory were never
> really as accurate as we thought for copy-on-write, and with the newer
> kernels they're now much worse. I'm falling back to using just the
> process size for the moment, and hope to take total free memory on the
> machine into account at some point to figure out sharing.
When I was adding this feature to Apache::VMonitor, I did the comparison
testing to see when things started to swap and it was giving me the
correct numbers. But it was long time ago. Must be some early 2.4 linux
kernel.
But as you say a process size should do the trick for those who can't rely
on the shared memory information. I remember someone said that *BSDs
report it correctly.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: Apache::SizeLimit?
Posted by Perrin Harkins <pe...@elem.com>.
On Wed, 2005-05-11 at 13:20 -0400, Stas Bekman wrote:
> It's not that SizeLimit and GTop no longer work, it's the recent linux
> kernels that have stopped maitaining certain information about shared
> memory. So the sharing happens, but there is no way to measure it.
My impression was that our measurements of shared memory were never
really as accurate as we thought for copy-on-write, and with the newer
kernels they're now much worse. I'm falling back to using just the
process size for the moment, and hope to take total free memory on the
machine into account at some point to figure out sharing.
- Perrin
Re: Apache::SizeLimit?
Posted by Stas Bekman <st...@stason.org>.
Perrin Harkins wrote:
> On Wed, 2005-05-11 at 13:27 +0200, Torsten Foertsch wrote:
>
>>Is there a way on linux to get the amount of pages shared among processes by
>>copy on write?
>
>
> No. There was a lengthy discussion about this on the list a few months
> back. The methods used by SizeLimit and GTop no longer work, if they
> ever did. They do calculate size correctly, but not shared size.
It's not that SizeLimit and GTop no longer work, it's the recent linux
kernels that have stopped maitaining certain information about shared
memory. So the sharing happens, but there is no way to measure it.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: Apache::SizeLimit?
Posted by Perrin Harkins <pe...@elem.com>.
On Wed, 2005-05-11 at 13:27 +0200, Torsten Foertsch wrote:
> Is there a way on linux to get the amount of pages shared among processes by
> copy on write?
No. There was a lengthy discussion about this on the list a few months
back. The methods used by SizeLimit and GTop no longer work, if they
ever did. They do calculate size correctly, but not shared size.
- Perrin