You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Stas Bekman <st...@stason.org> on 2000/06/20 18:23:52 UTC
[RFC] Swapping Prevention
I've rewritten the guide's swapping section now delving into quite
technical details. Please take a moment to review this section and comment
on it.
I have tried to stay on focus to deliver as much as possible relevant
details without actually delving into the fine details of memory
management techniques like virtual memory, sharing and alike, therefore
this version is a simplified one. I hope that it'll be still educating and
clear for folks with no CS background. The goal is to helps to understand
the reasoning behind desired swapping prevention under mod_perl.
Submitting of the additional techniques/details and possible
problems/solutions that I've not covered or covered incorrectly is very
welcome. Remember that I'm not trying to reproduce the operating systems
course but provide only the relevant details. Thanks a lot!
So here we go:
=head2 Swapping Prevention
Before we delve into swapping process details, let's refresh our
memory management knowledge.
The computer memory is called RAM, which stands for Random Access
Memory. Reading and writing to RAM is by a few orders faster than
doing the same operations with a hard disk, the former uses the
non-movable memory cells, while the latter uses the rotating magnetic
media.
On most operating systems a swap memory is used as an extension for
RAM and not as a duplication of it. So if your OS is one of those, if
you have 128MB of RAM and 256MB swap partition, you have a total of
384MB of memory available for OS. You should never count this extra
memory when you decide on the maximum number of processes to be run,
and we will show why in the moment.
The swapping memory can be built of a number of hard disk partitions
and swap files formatted to be used as a swap memory. When you need
more swap memory you can always extend it on demand as long as you
have some free disk space (for more information see the I<mkswap> and
I<swapon> manpages).
System memory is quantified in units called memory pages. Usually the
size of the memory page is 4KB or 8KB. So if you have 256MB of RAM
installed on your machine and the page size is 4KB your system has
64,000 memory pages to work with. When the system is started all
memory pages are available for use by the programs (processes).
When a process asks for a specific memory page, the system checks
whether this page is already loaded in memory and if it's not, the
I<page fault> event occurs, which requires the system to allocate a
free memory page and load this page into memory.
Each program asks a system for a certain number of the RAM memory
pages when it started. For example if a program needs 512KB of memory
when it starts and the memory page size is 4KB, the system will have
to allocate 128 memory pages for this program. Note that if some of
the pages can be shared with other processes it's possible that less
pages will be needed. For examples pages that are occupied by all
kind of I<libc> libraries are generally shared between the processes that
use these libraries.
After the process was started it might ask the OS for additional
memory pages when it needs them. While there are memory pages
available, the OS allocates these demanded memory pages and delivers
them to the process that asks for them. Most programs, particularly
Perl programs, on most modern OSs don't return memory pages while they
are running to improve the performance. If some of the memory gets
freed it's reused when needed by the process, without creating an
additional overhead by asking the system to allocate new memory pages.
That's why you can observe that Perl programs are usually growing in
size and almost never shrink.
When the process quits it returns its memory pages to the pool of
freely available pages for other processes to reuse.
The more processes are running the more memory pages are used, the
less available memory pages remain. At some point when a process asks
for memory pages the system cannot find any available, since all of
them are in use. That's the moment when the system starts the swapping
process which uses the swapping memory.
When there is no free memory pages available, in order to gain the
required number of memory pages requested by the process the system
(kernel) has to page out (i.e. move to a hard disk's swap partition)
exactly the same number of pages. The kernel pages out memory pages
using the LRU (least recently used) or a similar algorithm to decide
which pages to take out, in attempt to prevent the situation where the
next process which is going to get CPU will need exactly the pages
that were just paged out, and therefore prevent the page fault (when
the page is not found in RAM). Because if the page faults occur, more
pages will have to be swapped out in order to free the memory for the
pages that get swapped in (loaded from swap into RAM).
When the CPU has to page memory pages in and out, the system slows
down, and not serving the process as fast as before. This leads to
accumulation of the processes waiting for CPU, which causes processing
demands to go up, which in turn slows down the system even more as
more memory is required. This ever worsening spiral will lead the
machine to halt, unless the resource demand suddenly drops down and
allows the processes to catch up with their tasks and go back to
normal memory usage.
This scenario is certainly educating, and it should be now obvious
that your system that runs the web server should never swap. It's
absolutely normal for your desktop to start swapping. You will see it
immediately since things will slow down and sometimes the system will
freeze for a short periods. But as we just mentioned, you can stop
starting new programs and can quit some, thus allowing the system to
catch up with the load and come back to use the RAM. In the case of
the web server you have much less control since it's users who load
your machine. Therefore you should configure the server, so that the
maximum number of possible processes will be small enough using the
C<MaxClients> directive. This will ensure that at the peak hours the
system won't swap. Remember that swap space is an emergency pool, not
a resource to be used routinely. If you are low on memory and you
badly need it, buy it or reduce a number of processes to prevent
swapping.
However sometimes due to the faulty code, some process might start
spinning in some unconstrained loop, consuming all the available RAM
and starting to heavily use the swap memory. In such a situation it
helps when you have a big emergency pool (i.e. lots of swap
memory). But you have to resolve this problem as soon as possible since
this pool
won't last for a long time. In the meanwhile the C<Apache::Resource>
module can be handy.
Sometimes calling an undefined subroutine in a module can cause a
tight loop that consumes all the available memory. Here is a way to
catch such errors. Define an C<AUTOLOAD> subroutine:
sub UNIVERSAL::AUTOLOAD {
my $class = shift;
warn "$class can't \$UNIVERSAL::AUTOLOAD!\n";
}
This will produce a nice error in I<error_log>, giving the line number
of the call and the name of the undefined subroutine.
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://perl.org http://stason.org/TULARC
http://singlesheaven.com http://perlmonth.com http://sourcegarden.org
Re: [RFC] Swapping Prevention
Posted by Barrie Slaymaker <rb...@telerama.com>.
A nit: the distinction between paging and swapping doesn't seem clear to me.
You describe the paging process, then talk about how you should never swap.
Or maybe that's too detailed for your intended audience.
- Barrie
[RFC] (corrected) Swapping Prevention
Posted by Stas Bekman <st...@stason.org>.
Here is yet another almost complete rewrite of the section. Thanks to
Barrie and Ed for the comments.
I've taken time to re-read long time ago read linux kernel memory
management system overview, so I hope now the section carries no mistakes.
Remember that many details, like virtual memory explanation, are skipped.
Comments are welcome.
=head2 Swapping Prevention
Before we delve into swapping process details, let's refresh our
knowledge of memory components and memory management
The computer memory is called RAM, which stands for Random Access
Memory. Reading and writing to RAM is, by a few orders, faster than
doing the same operations on a hard disk, the former uses non-movable
memory cells, while the latter uses rotating magnetic media.
On most operating systems swap memory is used as an extension for RAM
and not as a duplication of it. So if your OS is one of those, if you
have 128MB of RAM and 256MB swap partition, you have a total of 384MB
of memory available. You should never count the extra memory when you
decide on the maximum number of processes to be run, and we will show
why in the moment.
The swapping memory can be built of a number of hard disk partitions
and swap files formatted to be used as swap memory. When you need
more swap memory you can always extend it on demand as long as you
have some free disk space (for more information see the I<mkswap> and
I<swapon> manpages).
System memory is quantified in units called memory pages. Usually the
size of a memory page is between 1KB and 8KB. So if you have 256MB of
RAM installed on your machine and the page size is 4KB your system has
64,000 main memory pages to work with and these pages are fast. If
you have 256MB swap partition the system can use yet another 64,000
memory pages, but they are much slower.
When the system is started all memory pages are available for use by
the programs (processes).
Unless the program is really small, the process running this program
uses only a few segments of the program, each segment mapped onto its
own memory page. Therefore only a few memory pages are required to be
loaded into the memory.
When the process needs an additional program's segment to be loaded
into the memory, it asks the system whether the page containing this
segment is already loaded in the memory. If the page is not found--an
event know as a I<page fault> occurs, which requires the system to
allocate a free memory page, go to the disk, read and load the
requested program's segment into the allocated memory page.
If a process needs to bring a new page into physical memory and there
are no free physical pages available, the operating system must make
room for this page by discarding another page from physical memory.
If the page to be discarded from physical memory came from an image or
data file and has not been written to then the page does not need to
be saved. Instead it can be discarded and if the process needs that
page again it can be brought back into memory from the image or data
file.
However, if the page has been modified, the operating system must
preserve the contents of that page so that it can be accessed at a
later time. This type of page is known as a I<dirty page> and when it
is removed from memory it is saved in a special sort of file called
the swap file. This process is referred to as a I<swapping out>.
Accesses to the swap file are very long relative to the speed of the
processor and physical memory and the operating system must juggle the
need to write pages to disk with the need to retain them in memory to
be used again.
In order to improve the swapping out process, to decrease the
possibility that the page that has just been swapped out, will be
needed at the next moment, the LRU (least recently used) or a similar
algorithm is used.
To summarize the two swapping scenarios, read-only pages discarding
incurs no overhead in contrast with the discarding scenario of the
data pages that have been written to, since in the latter case the
pages have to be written to a swap partition located on the slow disk.
Therefore your machine's overall performance will be much better if
there will be less memory pages that can become dirty.
But the problem is, Perl is a language with no strong data types,
which means that both, the program code and the program data are seen
as a data pages by OS since both mapped to the same memory
pages. Therefore a big chunk of your Perl code becomes dirty when its
variables are modified and when the pages need to be discarded they
have to be written to the swap partition.
This leads us to two important conclusions about swapping and Perl.
=over
=item *
Running your system when there is no free main memory available
hinders performance, because processes memory pages should be
discarded and then reread from disk again and again.
=item *
Since a majority of the running code is a Perl code, in addition to
the overhead of reading the previously discarded pages in, the
overhead of saving the dirty pages to the swap partition is occurring.
=back
When the system has to swap memory pages in and out, the system slows
down, not serving the processes as fast as before. This leads to an
accumulation of processes waiting for their turn to run, which further
causes processing demands to go up, which in turn slows down the
system even more as more memory is required. This ever worsening
spiral will lead the machine to halt, unless the resource demand
suddenly drops down and allows the processes to catch up with their
tasks and go back to normal memory usage.
In addition it's important to know that for a better performance, most
programs, particularly programs written in Perl, on most modern OSs
don't return memory pages while they are running. If some of the
memory gets freed it's reused when needed by the process, without
creating the additional overhead of asking the system to allocate new
memory pages. That's why you will observe that Perl programs grow in
size as they run and almost never shrink.
When the process quits it returns its memory pages to the pool of
freely available pages for other processes to use.
This scenario is certainly educating, and it should be now obvious
that your system that runs the web server should never swap. It's
absolutely normal for your desktop to start swapping. You will see it
immediately since things will slow down and sometimes the system will
freeze for a short periods. But as we just mentioned, you can stop
starting new programs and can quit some, thus allowing the system to
catch up with the load and come back to use the RAM.
In the case of the web server you have much less control since it's
users who load your machine by issuing requests to your server.
Therefore you should configure the server, so that the maximum number
of possible processes will be small enough using the C<MaxClients>
directive. This will ensure that at peak hours the system won't
swap. Remember that swap space is an emergency pool, not a resource to
be used routinely. If you are low on memory and you badly need it,
buy it or reduce the number of processes to prevent swapping.
However sometimes, due to the faulty code, some process might start
spinning in an unconstrained loop, consuming all the available RAM and
starting to heavily use swap memory. In such a situation it helps when
you have a big emergency pool (i.e. lots of swap memory). But you have
to resolve this problem as soon as possible since this pool won't last
for a long time. In the meanwhile the C<Apache::Resource> module can
be handy.
Sometimes calling an undefined subroutine in a module can cause a
tight loop that consumes all the available memory. Here is a way to
catch such errors. Define an C<AUTOLOAD> subroutine:
sub UNIVERSAL::AUTOLOAD {
my $class = shift;
warn "$class can't \$UNIVERSAL::AUTOLOAD!\n";
}
This will produce a nice error in I<error_log>, giving the line number
of the call and the name of the undefined subroutine.
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://perl.org http://stason.org/TULARC
http://singlesheaven.com http://perlmonth.com http://sourcegarden.org
Re: [RFC] Swapping Prevention
Posted by Stas Bekman <st...@stason.org>.
On Tue, 20 Jun 2000, Joshua Chamas wrote:
> > your machine. Therefore you should configure the server, so that the
> > maximum number of possible processes will be small enough using the
> > C<MaxClients> directive. This will ensure that at the peak hours the
> > system won't swap. Remember that swap space is an emergency pool, not
> > a resource to be used routinely. If you are low on memory and you
> > badly need it, buy it or reduce a number of processes to prevent
> > swapping.
>
> One common mistake that people make is to not load
> test against a server to trigger the full MaxClients
> in production. In order to prevent swapping, one must
> simulate the server at its point of greatest RAM stress,
> and one can start to do this by running ab against
> a program so that each of the MaxClient httpd processes
> goes through its MaxRequests.
>
> So lets say MaxRequests is set to 1000 and MaxClients
> is set to 100, then in order to see the system go
> through a full cycle fully maxed out in RAM, one might
>
> ab -c 100 -n 100000 http://yoursite.loadtest/modperl/ramhog
>
> Then fire up top, sit back, and enjoy the show!
>
> In summary this aids in swapping prevention, by load testing
> the server under highest RAM stress preproduction, and seeing
> if it swaps. Then one can tune their parameters to avoid
> swapping under this highest load, by decreasing MaxClients
> or MaxRequests.
Hmm, good points Joshua. But just running the above example is not
enough. Remember that when running the production server code become
unshared, more memory is used, therefore the above test won't protect
ensure that swapping won't happen.
Unless you can reproduce a production server load, including different
queries that will lead to the very close to real simulation of the server
usage your advice is definitely helps but not as much as one would want.
I think that it'd be really nice to have a post processing handler that
will log all the requests including the query data and feed it into a
database on the constant basis, merging with the previously identicall
inputs and bumping out the count. So when you will want to reproduce the
real load of your server, the other program will analyze this database and
will generate the output like this:
input %
----------------------
URL1?query1 46.1%
URL1?query2 12.1%
URL2?query3 7.2%
URL2?query4 4.2%
...........
URLN?queryN 0.2%
-------------------
100%
so this input can be fed into the program that runs ab. So if you want to
simulate 10000 requests, you will run 10000*0.461 URLs of the request in
the first row above, 10000*0.121 of the second, etc...
The you will get to tune your server to the maximum performance given that
the behavior of the users is not about to change. The older your database
is the more correct statistics you will get.
Cith help of real usage statistics, in addition ability to fine tune the
MaxClient directive, you will learn what sections of code are mostly used
and therefore optimize them.
_____________________________________________________________________
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:stas@stason.org http://perl.org http://stason.org/TULARC
http://singlesheaven.com http://perlmonth.com http://sourcegarden.org
Re: [RFC] Swapping Prevention
Posted by Tom Brown <tb...@baremetal.com>.
On Tue, 20 Jun 2000, Joshua Chamas wrote:
> > your machine. Therefore you should configure the server, so that the
> > maximum number of possible processes will be small enough using the
> > C<MaxClients> directive. This will ensure that at the peak hours the
> > system won't swap. Remember that swap space is an emergency pool, not
> > a resource to be used routinely. If you are low on memory and you
> > badly need it, buy it or reduce a number of processes to prevent
> > swapping.
>
> One common mistake that people make is to not load
> test against a server to trigger the full MaxClients
> in production. In order to prevent swapping, one must
> simulate the server at its point of greatest RAM stress,
> and one can start to do this by running ab against
> a program so that each of the MaxClient httpd processes
> goes through its MaxRequests.
attached is a little perl script (3.5k) that I use to "replay" a log file
as fast as it will go... the script uses Sys-5 message queues and forks
off the number of children specified by -n to read these messages and make
the given request... Since we use it for benchmarking machines hosting
lots of virtual servers, the input file format is actually
host uri status size
(space separated) and then it whines about responses that don't match the
status or have a vastly differing size...
This version isn't as polished as the previous one was... but I left that
under /tmp/ for too long :-(
Anyhow, my beef with ab is that it just pounds on one page (but it's
very good at that).
Anyhow, I haven't generally used it for testing MaxClients settings, but
that would just mean increasing the -n parameter... it is
_simple_ and doesn't deal with stuff like keep alives, and it's
not the fastest client, so it would be plain stupid to try to use
this to benchmark the number of GIFs per second a faster CPU
could serve. Basically, we generally use for correctness testing rather
than all out performance.
-Tom
p.s. Options...
# -n = num threads;
# -v = verbose
# -d = debug
# -s = allowed size variance (bytes)
# -p = allowed size variance (%)
# -w = warn if response takes more than this seconds
Re: [RFC] Swapping Prevention
Posted by Joshua Chamas <jo...@chamas.com>.
> your machine. Therefore you should configure the server, so that the
> maximum number of possible processes will be small enough using the
> C<MaxClients> directive. This will ensure that at the peak hours the
> system won't swap. Remember that swap space is an emergency pool, not
> a resource to be used routinely. If you are low on memory and you
> badly need it, buy it or reduce a number of processes to prevent
> swapping.
One common mistake that people make is to not load
test against a server to trigger the full MaxClients
in production. In order to prevent swapping, one must
simulate the server at its point of greatest RAM stress,
and one can start to do this by running ab against
a program so that each of the MaxClient httpd processes
goes through its MaxRequests.
So lets say MaxRequests is set to 1000 and MaxClients
is set to 100, then in order to see the system go
through a full cycle fully maxed out in RAM, one might
ab -c 100 -n 100000 http://yoursite.loadtest/modperl/ramhog
Then fire up top, sit back, and enjoy the show!
In summary this aids in swapping prevention, by load testing
the server under highest RAM stress preproduction, and seeing
if it swaps. Then one can tune their parameters to avoid
swapping under this highest load, by decreasing MaxClients
or MaxRequests.
-- Joshua
_________________________________________________________________
Joshua Chamas Chamas Enterprises Inc.
NodeWorks >> free web link monitoring Huntington Beach, CA USA
http://www.nodeworks.com 1-714-625-4051
Stas Bekman wrote:
>
> I've rewritten the guide's swapping section now delving into quite
> technical details. Please take a moment to review this section and comment
> on it.
>
> I have tried to stay on focus to deliver as much as possible relevant
> details without actually delving into the fine details of memory
> management techniques like virtual memory, sharing and alike, therefore
> this version is a simplified one. I hope that it'll be still educating and
> clear for folks with no CS background. The goal is to helps to understand
> the reasoning behind desired swapping prevention under mod_perl.
>
> Submitting of the additional techniques/details and possible
> problems/solutions that I've not covered or covered incorrectly is very
> welcome. Remember that I'm not trying to reproduce the operating systems
> course but provide only the relevant details. Thanks a lot!
>
> So here we go:
>
> =head2 Swapping Prevention
>
> Before we delve into swapping process details, let's refresh our
> memory management knowledge.
>
> The computer memory is called RAM, which stands for Random Access
> Memory. Reading and writing to RAM is by a few orders faster than
> doing the same operations with a hard disk, the former uses the
> non-movable memory cells, while the latter uses the rotating magnetic
> media.
>
> On most operating systems a swap memory is used as an extension for
> RAM and not as a duplication of it. So if your OS is one of those, if
> you have 128MB of RAM and 256MB swap partition, you have a total of
> 384MB of memory available for OS. You should never count this extra
> memory when you decide on the maximum number of processes to be run,
> and we will show why in the moment.
>
> The swapping memory can be built of a number of hard disk partitions
> and swap files formatted to be used as a swap memory. When you need
> more swap memory you can always extend it on demand as long as you
> have some free disk space (for more information see the I<mkswap> and
> I<swapon> manpages).
>
> System memory is quantified in units called memory pages. Usually the
> size of the memory page is 4KB or 8KB. So if you have 256MB of RAM
> installed on your machine and the page size is 4KB your system has
> 64,000 memory pages to work with. When the system is started all
> memory pages are available for use by the programs (processes).
>
> When a process asks for a specific memory page, the system checks
> whether this page is already loaded in memory and if it's not, the
> I<page fault> event occurs, which requires the system to allocate a
> free memory page and load this page into memory.
>
> Each program asks a system for a certain number of the RAM memory
> pages when it started. For example if a program needs 512KB of memory
> when it starts and the memory page size is 4KB, the system will have
> to allocate 128 memory pages for this program. Note that if some of
> the pages can be shared with other processes it's possible that less
> pages will be needed. For examples pages that are occupied by all
> kind of I<libc> libraries are generally shared between the processes that
> use these libraries.
>
> After the process was started it might ask the OS for additional
> memory pages when it needs them. While there are memory pages
> available, the OS allocates these demanded memory pages and delivers
> them to the process that asks for them. Most programs, particularly
> Perl programs, on most modern OSs don't return memory pages while they
> are running to improve the performance. If some of the memory gets
> freed it's reused when needed by the process, without creating an
> additional overhead by asking the system to allocate new memory pages.
> That's why you can observe that Perl programs are usually growing in
> size and almost never shrink.
>
> When the process quits it returns its memory pages to the pool of
> freely available pages for other processes to reuse.
>
> The more processes are running the more memory pages are used, the
> less available memory pages remain. At some point when a process asks
> for memory pages the system cannot find any available, since all of
> them are in use. That's the moment when the system starts the swapping
> process which uses the swapping memory.
>
> When there is no free memory pages available, in order to gain the
> required number of memory pages requested by the process the system
> (kernel) has to page out (i.e. move to a hard disk's swap partition)
> exactly the same number of pages. The kernel pages out memory pages
> using the LRU (least recently used) or a similar algorithm to decide
> which pages to take out, in attempt to prevent the situation where the
> next process which is going to get CPU will need exactly the pages
> that were just paged out, and therefore prevent the page fault (when
> the page is not found in RAM). Because if the page faults occur, more
> pages will have to be swapped out in order to free the memory for the
> pages that get swapped in (loaded from swap into RAM).
>
> When the CPU has to page memory pages in and out, the system slows
> down, and not serving the process as fast as before. This leads to
> accumulation of the processes waiting for CPU, which causes processing
> demands to go up, which in turn slows down the system even more as
> more memory is required. This ever worsening spiral will lead the
> machine to halt, unless the resource demand suddenly drops down and
> allows the processes to catch up with their tasks and go back to
> normal memory usage.
>
> This scenario is certainly educating, and it should be now obvious
> that your system that runs the web server should never swap. It's
> absolutely normal for your desktop to start swapping. You will see it
> immediately since things will slow down and sometimes the system will
> freeze for a short periods. But as we just mentioned, you can stop
> starting new programs and can quit some, thus allowing the system to
> catch up with the load and come back to use the RAM. In the case of
> the web server you have much less control since it's users who load
> your machine. Therefore you should configure the server, so that the
> maximum number of possible processes will be small enough using the
> C<MaxClients> directive. This will ensure that at the peak hours the
> system won't swap. Remember that swap space is an emergency pool, not
> a resource to be used routinely. If you are low on memory and you
> badly need it, buy it or reduce a number of processes to prevent
> swapping.
>
> However sometimes due to the faulty code, some process might start
> spinning in some unconstrained loop, consuming all the available RAM
> and starting to heavily use the swap memory. In such a situation it
> helps when you have a big emergency pool (i.e. lots of swap
> memory). But you have to resolve this problem as soon as possible since
> this pool
> won't last for a long time. In the meanwhile the C<Apache::Resource>
> module can be handy.
>
> Sometimes calling an undefined subroutine in a module can cause a
> tight loop that consumes all the available memory. Here is a way to
> catch such errors. Define an C<AUTOLOAD> subroutine:
>
> sub UNIVERSAL::AUTOLOAD {
> my $class = shift;
> warn "$class can't \$UNIVERSAL::AUTOLOAD!\n";
> }
>
> This will produce a nice error in I<error_log>, giving the line number
> of the call and the name of the undefined subroutine.
>
> _____________________________________________________________________
> Stas Bekman JAm_pH -- Just Another mod_perl Hacker
> http://stason.org/ mod_perl Guide http://perl.apache.org/guide
> mailto:stas@stason.org http://perl.org http://stason.org/TULARC
> http://singlesheaven.com http://perlmonth.com http://sourcegarden.org