You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Torsten Foertsch <to...@gmx.net> on 2005/08/07 20:37:34 UTC
[PATCH] Apache2::SizeLimit on Linux
Hi,
the attached patch allows Apache2::SizeLimit to use the new /proc/PID/smaps
instead of /proc/PID/statm. This takes into account copy-on-write pages when
counting shared memory.
The patch looks if /proc/PID/smaps exists and if Linux::Smaps is installed. If
not it uses the old /proc/PID/statm. Hence, it is compatible with old
linuxes.
/proc/PID/smaps exists by now in the mm-series of the linux kernel. I hope the
patch will take its way into the vanilla kernel in the not so distant future.
The following handler shows the effect:
<Perl>
require Apache2::SizeLimit;
package X;
use strict;
use Apache2::RequestRec ();
use Apache2::RequestIO ();
use Apache2::Const -compile=>qw(OK);
my $x="a"x(1024*1024);
sub handler {
my $r=shift;
my $args=$r->args;
my ($size, $shared)=$Apache2::SizeLimit::HOW_BIG_IS_IT->();
$x=~tr/a/b/ if( $args );
my ($size2, $shared2)=$Apache2::SizeLimit::HOW_BIG_IS_IT->();
$r->content_type('text/plain');
$r->print("1: size=$size shared=$shared\n");
$r->print("2: size=$size2 shared=$shared2\n");
return Apache2::Const::OK;
}
</Perl>
<Location /X>
SetHandler modperl
PerlResponseHandler X
</Location>
The parent apache allocates a megabyte for the string in $x. The tr-command
then overwrites all "a" with "b" if the handler is called with an argument.
This write is done in place, thus, the process size doesn't change. Only $x is
not shared anymore by means of COW between the parent and the child.
When Smaps are available curl shows:
r2@s93:~/work/mp2> curl http://localhost:8181/X?1
1: size=13452 shared=7456
2: size=13452 shared=6432
The shared memory has lost 1024 kB.
Without Smaps it says:
r2@s93:~/work/mp2> curl http://localhost:8181/X?1
1: size=13052 shared=3628
2: size=13052 shared=3636
One can see the kernel lies about the shared memory. It simply doesn't count
COW pages as shared.
Both experiments are done with a Suse9.3, apache 2.0.54, mod_perl 2.0.1 and
Linux::Smaps 0.01.
The first one uses linux 2.6.13-rc4-mm1 (with a little patch that will go into
the next mm-version, see
http://marc.theaimsgroup.com/?l=linux-kernel&m=112335328222533&w=2), the
second with Suses default kernel (2.6.11.4-20a-default).
Torsten
Re: [PATCH] Apache2::SizeLimit on Linux
Posted by Torsten Foertsch <to...@gmx.net>.
On Friday 19 August 2005 18:59, Philip M. Gollucci wrote:
> Can you combine your 2 e-mail patches into one patch set in one e-mail
> to make the life of who ever does easier? At least I think this is the
> second one.
Here it is.
Torsten
Re: [PATCH] Apache2::SizeLimit on Linux
Posted by "Philip M. Gollucci" <pg...@p6m7g8.com>.
Torsten Foertsch wrote:
> On Monday 08 August 2005 01:30, Stas Bekman wrote:
> It corrects some podchecker errors and warnings as well.
Hi,
Great work.
I'm not going to commit it as I don't have a linux bed readily available
to test on... Pretty sad for a guy that works with LAMP stacks :)
I'm +1 for it though.
Can you combine your 2 e-mail patches into one patch set in one e-mail
to make the life of who ever does easier? At least I think this is the
second one.
Thanks!
--
END
------------------------------------------------------------
What doesn't kill us can only make us stronger.
Nothing is impossible.
Philip M. Gollucci (pgollucci@p6m7g8.com) 301.254.5198
Consultant / http://p6m7g8.net/Resume/
Senior Developer / Liquidity Services, Inc.
http://www.liquidityservicesinc.com
http://www.liquidation.com
http://www.uksurplus.com
http://www.govliquidation.com
http://www.gowholesale.com
Re: [PATCH] Apache2::SizeLimit on Linux
Posted by Torsten Foertsch <to...@gmx.net>.
On Monday 08 August 2005 01:30, Stas Bekman wrote:
> Torsten Foertsch wrote:
> > Hi,
> >
> > the attached patch allows Apache2::SizeLimit to use the new
> > /proc/PID/smaps instead of /proc/PID/statm. This takes into account
> > copy-on-write pages when counting shared memory.
>
> Torsten++!
>
> Could you please submit a doc patch for Apache2/SizeLimit.pod
> http://svn.apache.org/viewcvs.cgi/perl/modperl/docs/trunk/src/docs/2.0/api/
>Apache2/SizeLimit.pod?rev=159831&view=log explaining the fact that shared
> memory calculation is normally wrong under linux kernel 2.6 and how to fix
> that situation? (below)
see attachment
It corrects some podchecker errors and warnings as well.
> > The patch looks if /proc/PID/smaps exists and if Linux::Smaps is
> > installed. If not it uses the old /proc/PID/statm. Hence, it is
> > compatible with old linuxes.
> >
> > /proc/PID/smaps exists by now in the mm-series of the linux kernel. I
> > hope the patch will take its way into the vanilla kernel in the not so
> > distant future.
>
> [...]
>
> > The first one uses linux 2.6.13-rc4-mm1 (with a little patch that will go
> > into the next mm-version, see
> > http://marc.theaimsgroup.com/?l=linux-kernel&m=112335328222533&w=2), the
> > second with Suses default kernel (2.6.11.4-20a-default).
>
> Do you know when the mm-series will become a part of the mainstream branch?
MM (Andrew Morton) says: "Yes, I expect we'll merge it up.". Since he is the
maintainer of the 2.6 vanilla linux I think it can be expected rather sooner
than later.
Torsten
Re: [PATCH] Apache2::SizeLimit on Linux
Posted by Stas Bekman <st...@stason.org>.
Torsten Foertsch wrote:
> Hi,
>
> the attached patch allows Apache2::SizeLimit to use the new /proc/PID/smaps
> instead of /proc/PID/statm. This takes into account copy-on-write pages when
> counting shared memory.
Torsten++!
Could you please submit a doc patch for Apache2/SizeLimit.pod
http://svn.apache.org/viewcvs.cgi/perl/modperl/docs/trunk/src/docs/2.0/api/Apache2/SizeLimit.pod?rev=159831&view=log
explaining the fact that shared memory calculation is normally wrong under
linux kernel 2.6 and how to fix that situation? (below)
> The patch looks if /proc/PID/smaps exists and if Linux::Smaps is installed. If
> not it uses the old /proc/PID/statm. Hence, it is compatible with old
> linuxes.
>
> /proc/PID/smaps exists by now in the mm-series of the linux kernel. I hope the
> patch will take its way into the vanilla kernel in the not so distant future.
[...]
> The first one uses linux 2.6.13-rc4-mm1 (with a little patch that will go into
> the next mm-version, see
> http://marc.theaimsgroup.com/?l=linux-kernel&m=112335328222533&w=2), the
> second with Suses default kernel (2.6.11.4-20a-default).
Do you know when the mm-series will become a part of the mainstream branch?
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: [PATCH] Apache2::SizeLimit on Linux
Posted by "Philip M. Gollucci" <pg...@p6m7g8.com>.
Torsten Foertsch wrote:
> --- mod_perl-2.0.1/lib/Apache2/SizeLimit.pm~ 2005-04-26
20:58:44.000000000 +0200
> +++ mod_perl-2.0.1/lib/Apache2/SizeLimit.pm 2005-08-07 17:56:00.691361040 +0200
> @@ -56,7 +56,11 @@
>
> } elsif (LINUX) {
>
> - $HOW_BIG_IS_IT = \&linux_size_check;
> + if ( eval { require Linux::Smaps } and Linux::Smaps->new($$) ) {
> + $HOW_BIG_IS_IT = \&linux_smaps_size_check;
> + } else {
> + $HOW_BIG_IS_IT = \&linux_size_check;
> + }
>
> } elsif (BSD_LIKE) {
>
> @@ -85,6 +89,11 @@
> }
> }
>
> +sub linux_smaps_size_check {
> + my $s=Linux::Smaps->new($$)->all;
> + return ($s->size, $s->shared_clean+$s->shared_dirty);
> +}
> +
> # return process size (in KB)
> sub linux_size_check {
> my($size, $resident, $share) = (0, 0, 0);
It looks sane to me, but I don't have a Linux comp to test on yet. I
might in a week or so; but I wouldn't be able to commit it until then.
--
END
------------------------------------------------------------
What doesn't kill us can only make us stronger.
Nothing is impossible.
Philip M. Gollucci (pgollucci@p6m7g8.com) 301.254.5198
Consultant / http://p6m7g8.net/Resume/
Senior Developer / Liquidity Services, Inc.
http://www.liquidityservicesinc.com
http://www.liquidation.com
http://www.uksurplus.com
http://www.govliquidation.com
http://www.gowholesale.com
Re: [PATCH] Apache2::SizeLimit on Linux
Posted by Perrin Harkins <pe...@elem.com>.
On Sun, 2005-08-07 at 20:37 +0200, Torsten Foertsch wrote:
> the attached patch allows Apache2::SizeLimit to use the new /proc/PID/smaps
> instead of /proc/PID/statm. This takes into account copy-on-write pages when
> counting shared memory.
Thanks, Torsten. I saw your module hit CPAN and was thinking we should
incorporate this into SizeLimit, and now here's the patch!
- Perrin