You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Perrin Harkins <pe...@elem.com> on 2005/02/14 23:39:05 UTC
Re: measuring shared memory on linux (was [Fwd: Re: /proc/*/statm,
exactly what does "shared" mean?)]
On Sat, 2005-02-12 at 11:43 -0500, Stas Bekman wrote:
> Richard F. Rebel wrote:
> > Hello Perrin and other modperl users,
> >
> > I have forwarded an email from Hugh Dickens as a follow up on recent
> > discussion of mod_perl+apache forked children and copy-on-write pages.
>
> Thanks for pursuing that issue at the linux kernel list, Richard. As you
> have suggested it doesn't look very good.
I want to add my thanks too.
> At the same time the actual sharing is the same as before, so you still
> win from preloading things. It's just that you can't quantify it anymore.
This is especially unfortunate because preloading appears to make the
SIZE measurement increase, as does the use of mmap in things like
Cache::FastMmap. Before, we were assuming you could see how much memory
was truly being used by subtracting SHARE from SIZE, but it seems that
won't work.
> Once we get some "official" resolution we should update the tools like
> Apache-VMonitor to reflect that discovery, in order to not mislead users.
> More importantly it affects Apache::SizeLimit and Apache::GTopSizeLimit,
> which as the things are now, aren't working properly when someone limits
> on shared memory.
Or on anything at all, since the size measurements they use include
things that really shouldn't be counted as part of the process size.
It seems that to tune properly you would need to send in requests with a
benchmarking tool and keep adjusting MaxClients upward while watching
the free memory on the box. That's really ugly. Maybe there's a way to
count memory used by each process slowly but accurately which we could
use during tuning at least.
- Perrin
Re: measuring shared memory on linux (was [Fwd: Re: /proc/*/statm,
exactly what does "shared" mean?)]
Posted by "Richard F. Rebel" <rr...@whenu.com>.
On Mon, 2005-02-14 at 17:39 -0500, Perrin Harkins wrote:
> > Thanks for pursuing that issue at the linux kernel list, Richard. As you
> > have suggested it doesn't look very good.
>
> I want to add my thanks too.
You are welcome. I wish I wasn't the bearer of such bad news tho.
> Or on anything at all, since the size measurements they use include
> things that really shouldn't be counted as part of the process size.
>
> It seems that to tune properly you would need to send in requests with a
> benchmarking tool and keep adjusting MaxClients upward while watching
> the free memory on the box. That's really ugly. Maybe there's a way to
> count memory used by each process slowly but accurately which we could
> use during tuning at least.
>
> - Perrin
That is an ugly way, but probably the only way we have at this time.
I received further information on the linux kernel mailing list.
Basically we can apply a 'patch', okay, a *hack* that is very unlikely
to ever end up in the vanilla kernel and start maintaining it.
The way I see it, unless we patch/hack the kernel to count the
information we are interested in, we are SOL. If we do add it, expect a
significant performance hit and don't expect it to be in the vanilla
kernel.
I am going to *try* and take the code sample I was given and wedge it
into the right place on a machine sometime this week. If it works, I'll
let you know. Maybe we can convince someone on lkml that if we make a
way to turn this *feature* on or off with a sysctl and put it somewhere
else in /proc/<pid/* that they'll put it in the vanilla kernel. Then
someone with more of a clue than I have can maintain it :)
Best,
Richard
--
Richard F. Rebel
cat /dev/null > `tty`
Re: measuring shared memory on linux (was [Fwd: Re: /proc/*/statm,
exactly what does "shared" mean?)]
Posted by Stas Bekman <st...@stason.org>.
Perrin Harkins wrote:
> On Mon, 2005-02-14 at 18:45 -0500, Stas Bekman wrote:
>
>>that approach is not very practical if change your code base constantly.
>>Since you will have to retune things every time you change your code.
>
>
> I know, it's terrible, but it's all I've come up with so far. Maybe we
> need to rethink how the size-limiting modules work so that they can use
> something like the total amount of free physical RAM instead. I think
> that's sort of what you were thinking of too.
>
>
>>I'd rather see the kernel providing a new feature which tells us the exact
>>amount of memory used by a group of forked processes (swapped in and out).
>
>
> Me too, but I'm not holding my breath. We can find out how much total
> memory is free though, at least on Linux.
That particular information is far from being useful, since there is the
cache. On my (linux) machine I have 0MB of free memory and 400-500MB of
cached buffers. So on linux one needs to combine the two to get the
answer. (see the output of free(1) on linux to see what I'm talking about)
So what you are saying is that whatever the technique is to get that
remaining free memory, if we have it we could just set a limit on how much
"free" memory is available and start killing Apache procs, when that limit
is passed. I think that might just work. The roughest approach will be to
just kill the current process. The fine tuned one will be to maintain a
table of proc sizes as I've suggested before.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: measuring shared memory on linux (was [Fwd: Re: /proc/*/statm,
exactly what does "shared" mean?)]
Posted by Salve J Nilsen <sj...@pvv.org>.
Suddenly, Perrin Harkins uttered:
> On Mon, 2005-02-14 at 18:45 -0500, Stas Bekman wrote:
>> that approach is not very practical if change your code base constantly.
>> Since you will have to retune things every time you change your code.
>
> I know, it's terrible, but it's all I've come up with so far.
Perhaps the CSA (Comprehensive System Accounting) and ELSA (Enhanced Linux
System Accounting) projects can do something for us? These provide patches
to the Linux kernel in order to improve process accounting.
CSA: http://oss.sgi.com/projects/csa/
ELSA: http://elsa.sourceforge.net/
Haven't looked at them closely, so this may be a shot in the dark... :-\
- Salve
--
#!/usr/bin/perl
sub AUTOLOAD{$AUTOLOAD=~/.*::(\d+)/;seek(DATA,$1,0);print# Salve Joshua Nilsen
getc DATA}$"="'};&{'";@_=unpack("C*",unpack("u*",':4@,$'.# <sj...@foo.no>
'2!--"5-(50P%$PL,!0X354UC-PP%/0\`'."\n"));eval "&{'@_'}"; __END__ is near! :)
Re: measuring shared memory on linux (was [Fwd: Re: /proc/*/statm,
exactly what does "shared" mean?)]
Posted by Perrin Harkins <pe...@elem.com>.
On Mon, 2005-02-14 at 18:45 -0500, Stas Bekman wrote:
> that approach is not very practical if change your code base constantly.
> Since you will have to retune things every time you change your code.
I know, it's terrible, but it's all I've come up with so far. Maybe we
need to rethink how the size-limiting modules work so that they can use
something like the total amount of free physical RAM instead. I think
that's sort of what you were thinking of too.
> I'd rather see the kernel providing a new feature which tells us the exact
> amount of memory used by a group of forked processes (swapped in and out).
Me too, but I'm not holding my breath. We can find out how much total
memory is free though, at least on Linux.
- Perrin
Re: measuring shared memory on linux (was [Fwd: Re: /proc/*/statm,
exactly what does "shared" mean?)]
Posted by Stas Bekman <st...@stason.org>.
Perrin Harkins wrote:
[...]
> It seems that to tune properly you would need to send in requests with a
> benchmarking tool and keep adjusting MaxClients upward while watching
> the free memory on the box. That's really ugly. Maybe there's a way to
> count memory used by each process slowly but accurately which we could
> use during tuning at least.
that approach is not very practical if change your code base constantly.
Since you will have to retune things every time you change your code.
I'd rather see the kernel providing a new feature which tells us the exact
amount of memory used by a group of forked processes (swapped in and out).
If we have this number we are gold. just maintain a table of absolute
process sizes and kill the biggest processes, when that total memory usage
number is bigger than the limit. i.e. limiting all procs at once, rather
on the process by process basis. So this will still provide a solid
protection from swapping.
--
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com