You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Paul Querna <ch...@force-elite.com> on 2008/10/29 19:59:06 UTC
MPMs, COW vs Child Process Spawning
Is COW ability of fork important enough with modern memory and operating
systems, to maintain two significantly different code paths for spawning
children processes?
Background:
One of the things I would like to do on the Simple MPM is unify how
child processes are created on win32 and unix.
On Win32, there is no fork, so roughly speaking what the current winnt
MPM creates a new process, and feeds the configuration over a pipe to
the new child.
On Unix, all of the current MPMs use fork, and do not execute a new
process, but instead then drop privileges and continue running.
What I would like to do, is change Unix to use the same pattern as on
Windows.
Doing this could also solves some issues with on some platforms[2].
It is common on most unix platforms for fork() to use copy on write[1]
for memory pages.
This means when spawning a child it uses much less 'real' memory,
compared to spawning a completely new process.
[1] - http://en.wikipedia.org/wiki/Copy-on-write
[2] - https://trac.macports.org/ticket/16340
Re: MPMs, COW vs Child Process Spawning
Posted by Mladen Turk <mt...@apache.org>.
Paul Querna wrote:
> Is COW ability of fork important enough with modern memory and operating
> systems, to maintain two significantly different code paths for spawning
> children processes?
>
> Background:
>
> One of the things I would like to do on the Simple MPM is unify how
> child processes are created on win32 and unix.
>
> On Win32, there is no fork, so roughly speaking what the current winnt
> MPM creates a new process, and feeds the configuration over a pipe to
> the new child.
>
> On Unix, all of the current MPMs use fork, and do not execute a new
> process, but instead then drop privileges and continue running.
>
> What I would like to do, is change Unix to use the same pattern as on
> Windows.
>
I'm currently working on a portable apr process shared data
using shared memory (See apr-dev list about the proposal)
http://mail-archives.apache.org/mod_mbox/apr-dev/200804.mbox/%3C4809F5F9.4060006@apache.org%3E
(Didn't get any replies to that, so it's either OK to proceed
or it was totally non understandable ;)
This could help unifying the platforms that need some sort of
passing data between parent and child(s)
Regards
--
^(TM)
Re: MPMs, COW vs Child Process Spawning
Posted by Torsten Foertsch <to...@gmx.net>.
On Wed 29 Oct 2008, Paul Querna wrote:
> Is COW ability of fork important enough with modern memory and
> operating systems, to maintain two significantly different code paths
> for spawning children processes?
Don't know if it matters but mod_perl heavily relies on COW. Why drop a
really valuable optimization?
Torsten
--
Need professional mod_perl support?
Just hire me: torsten.foertsch@gmx.net
Re: MPMs, COW vs Child Process Spawning
Posted by Dirk-Willem van Gulik <di...@webweaving.org>.
On 29 Oct 2008, at 19:59, Paul Querna wrote:
> It is common on most unix platforms for fork() to use copy on
> write[1] for memory pages.
Over the years - I've become completely addicted to both copy on write
(for larger sites with high loads) - and also note that the
optimisation (in hardware even!) of some unixes around exactly this
pattern is extreme.
So this is something which really would need to prove itself in that
space,
Dw
Re: MPMs, COW vs Child Process Spawning
Posted by Tim Bray <Ti...@Sun.COM>.
On Oct 30, 2008, at 2:16 AM, Graham Leggett wrote:
>> I don't think the intention would be to use a single process on
>> Unix -- we would still spawn multiple children -- they would just
>> be created using fork+exec of the httpd binary, rather than just
>> fork.
>
> This doesn't make sense - fork is far more efficient than fork+exec,
> we shouldn't be supporting the less efficient one for everybody just
> because it means less code.
>
> Instead we should try and fork, and if fork fails (eg Windows), or
> where configured (eg Leopard until Apple fixes it), it should fall
> back gracefully to fork+exec.
+1
I'd think that most people who'd done process-wrangling in unix-like
systems and other alternatives (VMS, Windows) would have come to
appreciate the many virtues of fork(). This feels like a
pessimization. -T
Re: MPMs, COW vs Child Process Spawning
Posted by Graham Leggett <mi...@sharp.fm>.
Paul Querna wrote:
> I don't think the intention would be to use a single process on Unix --
> we would still spawn multiple children -- they would just be created
> using fork+exec of the httpd binary, rather than just fork.
This doesn't make sense - fork is far more efficient than fork+exec, we
shouldn't be supporting the less efficient one for everybody just
because it means less code.
Instead we should try and fork, and if fork fails (eg Windows), or where
configured (eg Leopard until Apple fixes it), it should fall back
gracefully to fork+exec.
Regards,
Graham
--
Re: MPMs, COW vs Child Process Spawning
Posted by Jim Jagielski <ji...@jaguNET.com>.
On Oct 29, 2008, at 5:36 PM, Paul Querna wrote:
> Akins, Brian wrote:
>> One of the things I've noticed on a practical level, is that forked
>> children
>> in worker wind up being "linearized" within themselves to some
>> extent.
>> Think of how the buffered logs work in mod_log_config (one buffer
>> protected
>> by a mutex). (I can't think of the correct term for it right now.
>> Basically
>> every thread is waiting on the same mutex eventually.) This is
>> seen in
>> other modules...
>> The fork gets around this somewhat, as there are multiple mutexes
>> independently locked/unlocked.
>
> I don't think the intention would be to use a single process on
> Unix -- we would still spawn multiple children -- they would just be
> created using fork+exec of the httpd binary, rather than just fork.
>
Why? The most painful part is the actually forking, so why
make it worse with tagging an exec with it? IMO, when Apache needs
to fork (except for during startup), the assumption should be
Because We Are Busy, and should be as efficient and quick as
possible...
Re: MPMs, COW vs Child Process Spawning
Posted by Paul Querna <ch...@force-elite.com>.
Akins, Brian wrote:
> One of the things I've noticed on a practical level, is that forked children
> in worker wind up being "linearized" within themselves to some extent.
> Think of how the buffered logs work in mod_log_config (one buffer protected
> by a mutex). (I can't think of the correct term for it right now. Basically
> every thread is waiting on the same mutex eventually.) This is seen in
> other modules...
>
> The fork gets around this somewhat, as there are multiple mutexes
> independently locked/unlocked.
I don't think the intention would be to use a single process on Unix --
we would still spawn multiple children -- they would just be created
using fork+exec of the httpd binary, rather than just fork.
-Paul
Re: MPMs, COW vs Child Process Spawning
Posted by "Akins, Brian" <Br...@turner.com>.
One of the things I've noticed on a practical level, is that forked children
in worker wind up being "linearized" within themselves to some extent.
Think of how the buffered logs work in mod_log_config (one buffer protected
by a mutex). (I can't think of the correct term for it right now. Basically
every thread is waiting on the same mutex eventually.) This is seen in
other modules...
The fork gets around this somewhat, as there are multiple mutexes
independently locked/unlocked.
If we have one multi-threaded process, this will force us to think about the
performance ramifications of these uses of single mutexes. For
mod_log_config, for example, a queue with a single writer thread may be the
answer.
--
Brian Akins
Chief Operations Engineer
Turner Digital Media Technologies
Re: MPMs, COW vs Child Process Spawning
Posted by Paul Querna <ch...@force-elite.com>.
Paul Querna wrote:
....
> Doing this could also solves some issues with on some platforms[2].
...
> [2] - https://trac.macports.org/ticket/16340
See also:
https://issues.apache.org/bugzilla/show_bug.cgi?id=44250
http://bugs.python.org/issue1676
Re: MPMs, COW vs Child Process Spawning
Posted by Joe Orton <jo...@redhat.com>.
On Wed, Oct 29, 2008 at 11:59:06AM -0700, Paul Querna wrote:
> Is COW ability of fork important enough with modern memory and operating
> systems, to maintain two significantly different code paths for spawning
> children processes?
I looked at a stock 2.2 install (x86_64) with most modules built as
DSOs, and it looks like there's about 1-2MB of anon heap allocated.
(look at pmap or /proc/$PID/maps if anybody else wants to compare)
So... I would guess the answer is, "it depends", if that is a reasonable
estimate at the fork benefit. If you expect people to be doing those
insane 50K process tricks as are possible with prefork, then 50K*1MB is
a huge additional overhead, yes. If not, then I'd guess it's lost in
the noise.
I think the far bigger difference between Windows and Unix on this front
is fd/HANDLE inheritance though. Unless you want to get into passing
fds over AF_UNIX sockets on Unix (please no) then there is going to
remain a major disparity in how an MPM operates between the two
platforms, and you might as well keep the stuff Unix does well, no?
Regards, Joe
Re: MPMs, COW vs Child Process Spawning
Posted by Jorge Schrauwen <jo...@gmail.com>.
On Wed, Oct 29, 2008 at 9:56 PM, Akins, Brian <Br...@turner.com>wrote:
> On 10/29/08 3:40 PM, "Paul Querna" <ch...@force-elite.com> wrote:
>
> > Of course, maybe that is a good thing, since then they would be ported
> > forward and work on win32 too.
>
> Can you delete open files in win32 yet?? If not, most of my stuff will not
> port...
>
Yes you can... although you need to force it.
Way back I use a library (for got the name) that had all kinds of winapi
stuff that was useful. Unlocking a file was one of them.
so it was unlock the file, then remove it. Down site is if the file is open
by some process all output will be lost.
~Jorge
>
> --
> Brian Akins
> Chief Operations Engineer
> Turner Digital Media Technologies
>
>
Re: MPMs, COW vs Child Process Spawning
Posted by Ruediger Pluem <rp...@apache.org>.
On 10/29/2008 08:40 PM, Paul Querna wrote:
> Jorge Schrauwen wrote:
>> I think for a large configurations, the copy on write is a
>> significant optimisation - you can have large numbers of processes,
>> and a large configuration, and get away with it, as practically the
>> configuration is only memory resident once.
>>
>> I think it would be important to still support both, but certainly
>> the code to support both shouldn't be at all complex. Perhaps the
>> choice of which to choose could be configurable, so that Leopard
>> users could choose "config via pipe".
>>
>>
>> If you are going to implement both methodes...
>> fork() should be used and exec and pipe the config as a fallback.
>
>
> Well, part of the problem is that many current apache modules assume
> that the parent and child are fork'ed from one another -- this is a
> wrong assumption of course, since,it doesn't work on win32 that way --
> but most of those modules are win32 only of course. If we do this on
> unix, we will break all of those modules.
>
> Of course, maybe that is a good thing, since then they would be ported
> forward and work on win32 too.
I guess this would be a huge problem for making simple MPM the default MPM
in 2.4 as this might require significant changes to the modules that go far
beyond recompiling.
So I would say that we should implement both and have it configurable (ideally
at config time). Why? This allows to make performance tests and compare the
difference between fork and non fork in different configurations.
If it turns out that keeping both implementations is a PITA and that there
is no notable gain in using fork we can drop fork off in the next major
release after 2.4. This also gives the module authors more time to adjust
themselves to a non fork MPM even if they don't work under Windows.
Regards
Rüdiger
Re: MPMs, COW vs Child Process Spawning
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Akins, Brian wrote:
> On 10/29/08 3:40 PM, "Paul Querna" <ch...@force-elite.com> wrote:
>
>> Of course, maybe that is a good thing, since then they would be ported
>> forward and work on win32 too.
>
> Can you delete open files in win32 yet?? If not, most of my stuff will not
> port...
No. Delete-on-close yes. But that entails random name assignments to avoid
any conflict while the file persists.
Re: MPMs, COW vs Child Process Spawning
Posted by "Akins, Brian" <Br...@turner.com>.
On 10/29/08 3:40 PM, "Paul Querna" <ch...@force-elite.com> wrote:
> Of course, maybe that is a good thing, since then they would be ported
> forward and work on win32 too.
Can you delete open files in win32 yet?? If not, most of my stuff will not
port...
--
Brian Akins
Chief Operations Engineer
Turner Digital Media Technologies
Re: MPMs, COW vs Child Process Spawning
Posted by Paul Querna <ch...@force-elite.com>.
Jorge Schrauwen wrote:
> I think for a large configurations, the copy on write is a
> significant optimisation - you can have large numbers of processes,
> and a large configuration, and get away with it, as practically the
> configuration is only memory resident once.
>
> I think it would be important to still support both, but certainly
> the code to support both shouldn't be at all complex. Perhaps the
> choice of which to choose could be configurable, so that Leopard
> users could choose "config via pipe".
>
>
> If you are going to implement both methodes...
> fork() should be used and exec and pipe the config as a fallback.
Well, part of the problem is that many current apache modules assume
that the parent and child are fork'ed from one another -- this is a
wrong assumption of course, since,it doesn't work on win32 that way --
but most of those modules are win32 only of course. If we do this on
unix, we will break all of those modules.
Of course, maybe that is a good thing, since then they would be ported
forward and work on win32 too.
-Paul
Re: MPMs, COW vs Child Process Spawning
Posted by Jorge Schrauwen <jo...@gmail.com>.
On Wed, Oct 29, 2008 at 8:28 PM, Graham Leggett <mi...@sharp.fm> wrote:
> Paul Querna wrote:
>
> One of the things I would like to do on the Simple MPM is unify how child
>> processes are created on win32 and unix.
>>
>> On Win32, there is no fork, so roughly speaking what the current winnt MPM
>> creates a new process, and feeds the configuration over a pipe to the new
>> child.
>>
>> On Unix, all of the current MPMs use fork, and do not execute a new
>> process, but instead then drop privileges and continue running.
>>
>> What I would like to do, is change Unix to use the same pattern as on
>> Windows.
>>
>
> Hmmm.
>
> I think for a large configurations, the copy on write is a significant
> optimisation - you can have large numbers of processes, and a large
> configuration, and get away with it, as practically the configuration is
> only memory resident once.
>
> I think it would be important to still support both, but certainly the code
> to support both shouldn't be at all complex. Perhaps the choice of which to
> choose could be configurable, so that Leopard users could choose "config via
> pipe".
>
If you are going to implement both methodes...
fork() should be used and exec and pipe the config as a fallback.
Then I know nothing about mpm's so I'll keep quite now.
>
> Although the MPM should be simple, it should ideally not be so simplified
> to become simplistic.
>
> Regards,
> Graham
> --
~ Jorge
Re: MPMs, COW vs Child Process Spawning
Posted by Graham Leggett <mi...@sharp.fm>.
Paul Querna wrote:
> One of the things I would like to do on the Simple MPM is unify how
> child processes are created on win32 and unix.
>
> On Win32, there is no fork, so roughly speaking what the current winnt
> MPM creates a new process, and feeds the configuration over a pipe to
> the new child.
>
> On Unix, all of the current MPMs use fork, and do not execute a new
> process, but instead then drop privileges and continue running.
>
> What I would like to do, is change Unix to use the same pattern as on
> Windows.
Hmmm.
I think for a large configurations, the copy on write is a significant
optimisation - you can have large numbers of processes, and a large
configuration, and get away with it, as practically the configuration is
only memory resident once.
I think it would be important to still support both, but certainly the
code to support both shouldn't be at all complex. Perhaps the choice of
which to choose could be configurable, so that Leopard users could
choose "config via pipe".
Although the MPM should be simple, it should ideally not be so
simplified to become simplistic.
Regards,
Graham
--