You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl-cvs@perl.apache.org by Stas Bekman <sb...@iil.intel.com> on 1999/08/14 19:39:40 UTC
rewrite of Apache::SizeLimit
Hi,
I rewrote Apache::SizeLimit to use GTop perl interface to libgtop
(http://home-of-linux.org/gnome/libgtop/) a crossplatform lib to access
system resources (currenlty supports linux, *BSD, Solaris)
Using this library opens up the opportunities to limit the processes on
many different criterions (cpu, inodes, total memory (shared,rss, size and
etc), process memory, process cpu and much more - check out the URL from
above).
So first I've changed the name from Apache::SizeLimit to Apache::Limit,
since it's going to be a much wider range of limits, than just Size...
As of this moment I've implemented the MIN_PROCESS_SHARED_SIZE in addition
to the original MAX_PROCESS_SIZE.
Of course the question that follows, what is an overhead added? You will
see the benchmarks below, that were optimized to measure the margins added
by Apache::(Size)?Limit handlers relative to a regular configuration.
The results are that both Apache::(Size)?Limit handlers counted the same
results, so there is no more overhead added by making Apache::SizeLimit
obsolete. And there is a very little overhead about 15% related to the
regular run (with no Apache::(Size)?Limit handler enabled), which is
actually much smaller in a real application and not "Hello" print.
So here are the benchmarks - try the lib, tell me whether it works for
you. Later I'll add the missing pods and upload it to CVS.
BTW, Am I correct to rename it?
Environment:
* Both testing clients and server were run on the same machine (worse
testing)
* Machine: Linux RH6.0, 200Mb RAM, Pentium 200MMX
* Server Software: Apache/1.3.9-dev (mod_perl 1.22-dev)
* server params:
MinSpareServers 20
MaxSpareServers 20
StartServers 20
MaxClients 50
MaxRequestsPerChild 300
Benchmarks:
the script:
----------
print "Content-type: text/html\n\n";
print "Hello $$\n";
% ab -n 100 -c 10 http://www.stas.com/perl/limit/hello.pl
1. no limits handler
Concurrency Level: 10
Time taken for tests: 1.457 seconds
Complete requests: 100
Failed requests: 0
Total transferred: 21500 bytes
HTML transferred: 1100 bytes
Requests per second: 68.63
Transfer rate: 14.76 kb/s received
Connnection Times (ms)
min avg max
Connect: 0 0 3
Processing: 122 143 175
Total: 122 143 178
2. Apache::Limit
Concurrency Level: 10
Time taken for tests: 1.780 seconds
Complete requests: 100
Failed requests: 0
Total transferred: 21500 bytes
HTML transferred: 1100 bytes
Requests per second: 56.18
Transfer rate: 12.08 kb/s received
Connnection Times (ms)
min avg max
Connect: 0 0 3
Processing: 158 176 255
Total: 158 176 258
3. Apache::SizeLimit
Concurrency Level: 10
Time taken for tests: 1.808 seconds
Complete requests: 100
Failed requests: 0
Total transferred: 21500 bytes
HTML transferred: 1100 bytes
Requests per second: 55.31
Transfer rate: 11.89 kb/s received
Connnection Times (ms)
min avg max
Connect: 0 0 3
Processing: 155 178 238
Total: 155 178 241
_______________________________________________________________________
Stas Bekman mailto:sbekman@iil.intel.com www.singlesheaven.com/stas
Perl,CGI,Apache,Linux,Web,Java,PC at www.singlesheaven.com/stas/TULARC
www.apache.org & www.perl.com == www.modperl.com || perl.apache.org
single o-> + single o-+ = singlesheaven http://www.singlesheaven.com
Re: rewrite of Apache::SizeLimit
Posted by Stas Bekman <sb...@iil.intel.com>.
> LibGTop currently does not have any support for Solaris, and
> it will never have until some volunteer writes the code for
> it. I can't do this myself since I do not have any machine
> to test it on.
>
> -- http://home-of-linux.org/gnome/libgtop/plattform-solaris.shtml
>
> so please dont cvs rm SizeLimit.pm :)
First, I'm not going to remove SizeLimit.pm for back compatibility. That's
why I've renamed it to Limit.pm
Second, take a look at: http://gdev.hom.net/changelogs/libgtop.html
As I understand it does builds on Solaris, while it states that it
doesn't. I guess this will change on the next release.
>
> On Sat, 14 Aug 1999, Stas Bekman wrote:
>
> > Hi,
> >
> > I rewrote Apache::SizeLimit to use GTop perl interface to libgtop
> > (http://home-of-linux.org/gnome/libgtop/) a crossplatform lib to access
> > system resources (currenlty supports linux, *BSD, Solaris)
> >
> > Using this library opens up the opportunities to limit the processes on
> > many different criterions (cpu, inodes, total memory (shared,rss, size and
> > etc), process memory, process cpu and much more - check out the URL from
> > above).
> >
> > So first I've changed the name from Apache::SizeLimit to Apache::Limit,
> > since it's going to be a much wider range of limits, than just Size...
> >
> > As of this moment I've implemented the MIN_PROCESS_SHARED_SIZE in addition
> > to the original MAX_PROCESS_SIZE.
> >
> > Of course the question that follows, what is an overhead added? You will
> > see the benchmarks below, that were optimized to measure the margins added
> > by Apache::(Size)?Limit handlers relative to a regular configuration.
> >
> > The results are that both Apache::(Size)?Limit handlers counted the same
> > results, so there is no more overhead added by making Apache::SizeLimit
> > obsolete. And there is a very little overhead about 15% related to the
> > regular run (with no Apache::(Size)?Limit handler enabled), which is
> > actually much smaller in a real application and not "Hello" print.
> >
> > So here are the benchmarks - try the lib, tell me whether it works for
> > you. Later I'll add the missing pods and upload it to CVS.
> >
> > BTW, Am I correct to rename it?
> >
> > Environment:
> > * Both testing clients and server were run on the same machine (worse
> > testing)
> > * Machine: Linux RH6.0, 200Mb RAM, Pentium 200MMX
> > * Server Software: Apache/1.3.9-dev (mod_perl 1.22-dev)
> > * server params:
> > MinSpareServers 20
> > MaxSpareServers 20
> > StartServers 20
> > MaxClients 50
> > MaxRequestsPerChild 300
> >
> > Benchmarks:
> >
> > the script:
> > ----------
> > print "Content-type: text/html\n\n";
> > print "Hello $$\n";
> >
> > % ab -n 100 -c 10 http://www.stas.com/perl/limit/hello.pl
> >
> > 1. no limits handler
> >
> > Concurrency Level: 10
> > Time taken for tests: 1.457 seconds
> > Complete requests: 100
> > Failed requests: 0
> > Total transferred: 21500 bytes
> > HTML transferred: 1100 bytes
> > Requests per second: 68.63
> > Transfer rate: 14.76 kb/s received
> >
> > Connnection Times (ms)
> > min avg max
> > Connect: 0 0 3
> > Processing: 122 143 175
> > Total: 122 143 178
> >
> > 2. Apache::Limit
> >
> > Concurrency Level: 10
> > Time taken for tests: 1.780 seconds
> > Complete requests: 100
> > Failed requests: 0
> > Total transferred: 21500 bytes
> > HTML transferred: 1100 bytes
> > Requests per second: 56.18
> > Transfer rate: 12.08 kb/s received
> >
> > Connnection Times (ms)
> > min avg max
> > Connect: 0 0 3
> > Processing: 158 176 255
> > Total: 158 176 258
> >
> > 3. Apache::SizeLimit
> >
> > Concurrency Level: 10
> > Time taken for tests: 1.808 seconds
> > Complete requests: 100
> > Failed requests: 0
> > Total transferred: 21500 bytes
> > HTML transferred: 1100 bytes
> > Requests per second: 55.31
> > Transfer rate: 11.89 kb/s received
> >
> > Connnection Times (ms)
> > min avg max
> > Connect: 0 0 3
> > Processing: 155 178 238
> > Total: 155 178 241
> >
> >
> >
> > _______________________________________________________________________
> > Stas Bekman mailto:sbekman@iil.intel.com www.singlesheaven.com/stas
> > Perl,CGI,Apache,Linux,Web,Java,PC at www.singlesheaven.com/stas/TULARC
> > www.apache.org & www.perl.com == www.modperl.com || perl.apache.org
> > single o-> + single o-+ = singlesheaven http://www.singlesheaven.com
> >
>
>
_______________________________________________________________________
Stas Bekman mailto:sbekman@iil.intel.com www.singlesheaven.com/stas
Perl,CGI,Apache,Linux,Web,Java,PC at www.singlesheaven.com/stas/TULARC
www.apache.org & www.perl.com == www.modperl.com || perl.apache.org
single o-> + single o-+ = singlesheaven http://www.singlesheaven.com
Re: rewrite of Apache::SizeLimit
Posted by doctor jest <ix...@maz.org>.
LibGTop currently does not have any support for Solaris, and
it will never have until some volunteer writes the code for
it. I can't do this myself since I do not have any machine
to test it on.
-- http://home-of-linux.org/gnome/libgtop/plattform-solaris.shtml
so please dont cvs rm SizeLimit.pm :)
On Sat, 14 Aug 1999, Stas Bekman wrote:
> Hi,
>
> I rewrote Apache::SizeLimit to use GTop perl interface to libgtop
> (http://home-of-linux.org/gnome/libgtop/) a crossplatform lib to access
> system resources (currenlty supports linux, *BSD, Solaris)
>
> Using this library opens up the opportunities to limit the processes on
> many different criterions (cpu, inodes, total memory (shared,rss, size and
> etc), process memory, process cpu and much more - check out the URL from
> above).
>
> So first I've changed the name from Apache::SizeLimit to Apache::Limit,
> since it's going to be a much wider range of limits, than just Size...
>
> As of this moment I've implemented the MIN_PROCESS_SHARED_SIZE in addition
> to the original MAX_PROCESS_SIZE.
>
> Of course the question that follows, what is an overhead added? You will
> see the benchmarks below, that were optimized to measure the margins added
> by Apache::(Size)?Limit handlers relative to a regular configuration.
>
> The results are that both Apache::(Size)?Limit handlers counted the same
> results, so there is no more overhead added by making Apache::SizeLimit
> obsolete. And there is a very little overhead about 15% related to the
> regular run (with no Apache::(Size)?Limit handler enabled), which is
> actually much smaller in a real application and not "Hello" print.
>
> So here are the benchmarks - try the lib, tell me whether it works for
> you. Later I'll add the missing pods and upload it to CVS.
>
> BTW, Am I correct to rename it?
>
> Environment:
> * Both testing clients and server were run on the same machine (worse
> testing)
> * Machine: Linux RH6.0, 200Mb RAM, Pentium 200MMX
> * Server Software: Apache/1.3.9-dev (mod_perl 1.22-dev)
> * server params:
> MinSpareServers 20
> MaxSpareServers 20
> StartServers 20
> MaxClients 50
> MaxRequestsPerChild 300
>
> Benchmarks:
>
> the script:
> ----------
> print "Content-type: text/html\n\n";
> print "Hello $$\n";
>
> % ab -n 100 -c 10 http://www.stas.com/perl/limit/hello.pl
>
> 1. no limits handler
>
> Concurrency Level: 10
> Time taken for tests: 1.457 seconds
> Complete requests: 100
> Failed requests: 0
> Total transferred: 21500 bytes
> HTML transferred: 1100 bytes
> Requests per second: 68.63
> Transfer rate: 14.76 kb/s received
>
> Connnection Times (ms)
> min avg max
> Connect: 0 0 3
> Processing: 122 143 175
> Total: 122 143 178
>
> 2. Apache::Limit
>
> Concurrency Level: 10
> Time taken for tests: 1.780 seconds
> Complete requests: 100
> Failed requests: 0
> Total transferred: 21500 bytes
> HTML transferred: 1100 bytes
> Requests per second: 56.18
> Transfer rate: 12.08 kb/s received
>
> Connnection Times (ms)
> min avg max
> Connect: 0 0 3
> Processing: 158 176 255
> Total: 158 176 258
>
> 3. Apache::SizeLimit
>
> Concurrency Level: 10
> Time taken for tests: 1.808 seconds
> Complete requests: 100
> Failed requests: 0
> Total transferred: 21500 bytes
> HTML transferred: 1100 bytes
> Requests per second: 55.31
> Transfer rate: 11.89 kb/s received
>
> Connnection Times (ms)
> min avg max
> Connect: 0 0 3
> Processing: 155 178 238
> Total: 155 178 241
>
>
>
> _______________________________________________________________________
> Stas Bekman mailto:sbekman@iil.intel.com www.singlesheaven.com/stas
> Perl,CGI,Apache,Linux,Web,Java,PC at www.singlesheaven.com/stas/TULARC
> www.apache.org & www.perl.com == www.modperl.com || perl.apache.org
> single o-> + single o-+ = singlesheaven http://www.singlesheaven.com
>
Re: rewrite of Apache::SizeLimit
Posted by Doug MacEachern <do...@cp.net>.
On Sun, 15 Aug 1999, Stas Bekman wrote:
>
> Ok, So what is the consensus:
> * Apache::GTopLimit
> * Apache::GTop::Limit
> * Apache::Limit::GTop
I would go with:
Apache::GTopLimit
Apache::GTopStatus
I don't like three-level deep packages, unless the user never has to know
about them. besides, extra package == extra size :)
-Doug
Re: rewrite of Apache::SizeLimit
Posted by Stas Bekman <sb...@iil.intel.com>.
> > So first I've changed the name from Apache::SizeLimit to Apache::Limit,
> > since it's going to be a much wider range of limits, than just Size...
>
> great work Stas! I suggested Apache::GTopLimit to you before (still in
> my .sent-mail):
This is weird, for it's a second time you resend me an email that has
never reached me :( I think there is at least one another email that
wasn't answered from you
>
> ---
> I wonder if you should just start a new Apache::GTopLimit module? and
> leave SizeLimit as is. for two reasons,
> 1) since libgtop is not yet ported everywhere / not installed everywhere
> 2) since we are trying to limit size, better sense to have a slimmer
> module
>
> in fact, I might consider writing a mod_sizelimit in C, based on your Perl
> version :)
> I think "Apache::Limit" is far to broad a name.
Ok, So what is the consensus:
* Apache::GTopLimit
* Apache::GTop::Limit
* Apache::Limit::GTop
with the second choice, we could have later:
Apache::GTop::Status - since GTop is perfect for making system status
presentation. One of the things I want Apache::GTop::Status to be is the
same as mod_info, but also showing off the SHARED, which is pretty
important.
GTop provides also info for netstat and other resources.
BTW, I want to go further and make Apache::GTop::Limit limit on total
memory usage and not just per process, since I don't care if my process is
big, while I have enough total free memory and its a waste to kill it off.
Killing on 2 factors - shortage of total free memory and process's size
seems to me a much better approach (with a little more overhead ofcourse)
_______________________________________________________________________
Stas Bekman mailto:sbekman@iil.intel.com www.singlesheaven.com/stas
Perl,CGI,Apache,Linux,Web,Java,PC at www.singlesheaven.com/stas/TULARC
www.apache.org & www.perl.com == www.modperl.com || perl.apache.org
single o-> + single o-+ = singlesheaven http://www.singlesheaven.com
Re: rewrite of Apache::SizeLimit
Posted by Ask Bjoern Hansen <as...@valueclick.com>.
On Sat, 14 Aug 1999, Doug MacEachern wrote:
...
> I think "Apache::Limit" is far to broad a name.
I agree. A lot of access handles f.x. also limits stuff.
Someone on modperl@apache.org suggested splitting modulenames into sub
names for each handler, like Apache::Access::Whatever,
Apache::Trans::Whatnot. A lot of modules wouldn't fit into this, but we
should come up with some solution to the namespace problems we will get in
the future otherwise. :-/
- ask
--
ask bjoern hansen - <http://www.netcetera.dk/~ask/>
more than 30M impressions per day, <http://valueclick.com>
Re: rewrite of Apache::SizeLimit
Posted by Doug MacEachern <do...@cp.net>.
On Sat, 14 Aug 1999, Stas Bekman wrote:
> So first I've changed the name from Apache::SizeLimit to Apache::Limit,
> since it's going to be a much wider range of limits, than just Size...
great work Stas! I suggested Apache::GTopLimit to you before (still in
my .sent-mail):
---
I wonder if you should just start a new Apache::GTopLimit module? and
leave SizeLimit as is. for two reasons,
1) since libgtop is not yet ported everywhere / not installed everywhere
2) since we are trying to limit size, better sense to have a slimmer
module
in fact, I might consider writing a mod_sizelimit in C, based on your Perl
version :)
---
I think "Apache::Limit" is far to broad a name.
-Doug