You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Sascha Schumann <sa...@schumann.cx> on 2000/05/03 10:32:01 UTC
Moving things around
Hi,
attached is a first draft of a flowchart of a important part of
the Apache 2.0 build system. Your input is appreciated.
Short summary:
Select the best MPM which is supported by the target platform,
based on checks for so-called feature sets.
Now some more text:
The MPM determines how Apache and subsequent libraries will be
built. Before we can make any qualified assumptions about the
build process, we must know which MPM gets chosen.
We need to know the requirements (feature sets) each MPM needs to
operate. At the beginning, we check which feature sets are
available on the target platform.
We then select the final MPM based on what feature sets are
available. The system gives user-selected MPMs a higher priority.
By doing this, the system transparently selects the best MPM
supported on the target platform.
After enabling the feature set (i.e. updating CFLAGS), we perform
various feature checks and continue with executing the
module-specific checks.
- Sascha
Re: Moving things around
Posted by Ben Laurie <be...@algroup.co.uk>.
Sascha Schumann wrote:
>
> > We don't. If the user specifies disable-threads, then the prefork MPM
> > _must_ be chosen, coz its the only one that will work.
>
> Just because prefork is currently the only non-threaded MPM does
> not mean that there will be no other one with the same
> requirements.
I know. But it doesn't affect my point.
> > > It makes much more sense to me to let users choose whatever MPM
> > > they want. Then the MPM tells APR what it needs to operate, and
> > > not the other way around.
> >
> > But then modules that use other APR features would be screwed. Which is
> > why APR must support everything available, not merely what the current
> > MPM needs.
>
> Define "other APR features."
Err, threading, for example?
> We are basically only talking about enabling/disabling threads in
> APR depending on what the MPM requires. Why should some other
> module care about threading issues?
Because some other module may need threading!
> > MPM is really just another module. Its needs are not tbe be-all and
> > end-all of Apache's needs.
>
> While this is true from a programmer's perspective, the
> requirements of the MPM affect the build process drastically.
> In that way, MPMs are considered special.
Yes, but I can still see no rational reason to exclude functionality
from APR simply because an MPM doesn't need it.
Cheers,
Ben.
--
http://www.apache-ssl.org/ben.html
Re: Moving things around
Posted by rb...@covalent.net.
On Sun, 7 May 2000, Sascha Schumann wrote:
> > We don't. If the user specifies disable-threads, then the prefork MPM
> > _must_ be chosen, coz its the only one that will work.
>
> Just because prefork is currently the only non-threaded MPM does
> not mean that there will be no other one with the same
> requirements.
So, when there are more non-threaded MPM's, there will be a default and
other options. This has nothing to do with this discussion though. If
APR doesn't have threads, some non-threaded MPM will be chosen.
> > > It makes much more sense to me to let users choose whatever MPM
> > > they want. Then the MPM tells APR what it needs to operate, and
> > > not the other way around.
> >
> > But then modules that use other APR features would be screwed. Which is
> > why APR must support everything available, not merely what the current
> > MPM needs.
>
> Define "other APR features."
>
> We are basically only talking about enabling/disabling threads in
> APR depending on what the MPM requires. Why should some other
> module care about threading issues?
Because at some point, we are going to have layered or filtered I/O, and
every single implementation we have spoken about so far has used some form
of threading to make this work for modules like mod_include. Just because
the prefork MPM doesn't want threads doesn't mean no other module will
want threads. We have a lot of modules right now that want to use threads
in 1.3. The prefork MPM provides the most reliability, because if a
process goes does you only lose one connection. This says nothing at all
about threading abilities of the server.
> > MPM is really just another module. Its needs are not tbe be-all and
> > end-all of Apache's needs.
>
> While this is true from a programmer's perspective, the
> requirements of the MPM affect the build process drastically.
> In that way, MPMs are considered special.
No. This is the wrong way to think about it, IMHO.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Moving things around
Posted by Tony Finch <do...@dotat.at>.
dean gaudet <dg...@arctic.org> wrote:
>
>a 500 process prefork server with an extra thread per process "because APR
>lets me do it" consumes 1000 kernel-schedulable-objects.
All the world is not Linux :-) but otherwise your point is valid.
Tony.
--
f.a.n.finch fanf@demon.net dot@dotat.at
343 high-strung herald of the haphazard
Re: Moving things around
Posted by rb...@covalent.net.
On Sun, 7 May 2000, Sascha Schumann wrote:
> > If I choose the prefork MPM, then APR should *not* have its threads
> > disabled. I might have a module that spins up a thread to do some work.
> > Note that some of our I/O filtering designs will spin up a thread.
>
> That's a good example and one which makes me reconsidering my
> point.
>
> So, provided that the APR configuration on one platform is always
> the same, dynamically loadable MPMs are possible.
No. MPM's do affect how the compiler works. Some of the stuff in MPM's
can and should be inlin'ed, this can't be done if the MPM is not linked
in. There are other optimizations possible with the MPM code.
Dynamically loadable MPM's have always been _possible_, it just isn't
_advisable_.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Moving things around
Posted by Sascha Schumann <sa...@schumann.cx>.
> If I choose the prefork MPM, then APR should *not* have its threads
> disabled. I might have a module that spins up a thread to do some work.
> Note that some of our I/O filtering designs will spin up a thread.
That's a good example and one which makes me reconsidering my
point.
So, provided that the APR configuration on one platform is always
the same, dynamically loadable MPMs are possible.
- Sascha
Re: Moving things around
Posted by dean gaudet <dg...@arctic.org>.
On Sat, 6 May 2000, Greg Stein wrote:
> If I choose the prefork MPM, then APR should *not* have its threads
> disabled. I might have a module that spins up a thread to do some work.
> Note that some of our I/O filtering designs will spin up a thread.
no. this is wrong.
it is in nobody's best interest to support every possible permutation of
threads and processes and fibers and whathaveyous in the same
executable/dll/so/shared-memory-space. this violates code modularity so
grossly i'm not even sure where to begin to explain.
we do *not* want to encourage people to build complex server software
which is multiprocess but might use a thread here and there. do you
realise how hard this would be to debug? let alone link?
a 500 process prefork server with an extra thread per process "because APR
lets me do it" consumes 1000 kernel-schedulable-objects.
if you choose prefork MPM then you are choosing an APR that does not
support threads, and none of your modules can use threads.
if that's a problem, then maybe you need to split up your service into
multithreaded portions and multiprocess portions, and use proxy technology
to tie them together.
the few percentage points in performance you lose from making a
multi-service solution such as this will be more than worth it when it
comes time to debug and scale the system.
-dean
Re: Moving things around
Posted by Greg Stein <gs...@lyra.org>.
On Sun, 7 May 2000, Sascha Schumann wrote:
>...
> > > It makes much more sense to me to let users choose whatever MPM
> > > they want. Then the MPM tells APR what it needs to operate, and
> > > not the other way around.
> >
> > But then modules that use other APR features would be screwed. Which is
> > why APR must support everything available, not merely what the current
> > MPM needs.
>
> Define "other APR features."
>
> We are basically only talking about enabling/disabling threads in
> APR depending on what the MPM requires. Why should some other
> module care about threading issues?
If I choose the prefork MPM, then APR should *not* have its threads
disabled. I might have a module that spins up a thread to do some work.
Note that some of our I/O filtering designs will spin up a thread.
The choice of the MPM should have no bearing on the APR configuration. If
the user wants to optimize APR and disable threads, then they can. But
Apache shouldn't be doing it merely as a side-effect of choosing a
particular MPM.
> > MPM is really just another module. Its needs are not tbe be-all and
> > end-all of Apache's needs.
>
> While this is true from a programmer's perspective, the
> requirements of the MPM affect the build process drastically.
> In that way, MPMs are considered special.
I disagree. The MPM does not necessarily alter the build. Especially to do
something like disable threads in APR.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: Moving things around
Posted by Sascha Schumann <sa...@schumann.cx>.
> We don't. If the user specifies disable-threads, then the prefork MPM
> _must_ be chosen, coz its the only one that will work.
Just because prefork is currently the only non-threaded MPM does
not mean that there will be no other one with the same
requirements.
> > It makes much more sense to me to let users choose whatever MPM
> > they want. Then the MPM tells APR what it needs to operate, and
> > not the other way around.
>
> But then modules that use other APR features would be screwed. Which is
> why APR must support everything available, not merely what the current
> MPM needs.
Define "other APR features."
We are basically only talking about enabling/disabling threads in
APR depending on what the MPM requires. Why should some other
module care about threading issues?
> MPM is really just another module. Its needs are not tbe be-all and
> end-all of Apache's needs.
While this is true from a programmer's perspective, the
requirements of the MPM affect the build process drastically.
In that way, MPMs are considered special.
- Sascha
Re: Moving things around
Posted by Ben Laurie <be...@algroup.co.uk>.
Sascha Schumann wrote:
>
> On Fri, May 05, 2000 at 07:42:38AM -0400, rbb@covalent.net wrote:
> >
> > > Let me clarify why that won't work.
> > >
> > > Everything (including APR and MM) depends on which MPM gets chosen,
> > > because it can change *everything* depending on what feature set
> > > is associated with the MPM.
> >
> > You are going on the assumption that APR only gets what the MPM needs. We
> > decided a few weeks ago that this was the wrong approach. APR gets what
> > the machine provides, and the MPM decision is based on what APR was
> > compiled with and what the user specified. For the prefork MPM, there are
> > two ways to specify this desire to Apache:
> >
> > 1) When configuring APR --disable-threads
> > 2) When configuring Apache --with-mpm=prefork
> >
> > The first method means there is no extra code in APR for threading. The
> > second means APR has threading code but Apache ignores it.
>
> I don't see how this can work. Why the hell do you expect the
> user to know that he must specify --disable-threads, so that he
> can use the prefork MPM?
We don't. If the user specifies disable-threads, then the prefork MPM
_must_ be chosen, coz its the only one that will work.
> It makes much more sense to me to let users choose whatever MPM
> they want. Then the MPM tells APR what it needs to operate, and
> not the other way around.
But then modules that use other APR features would be screwed. Which is
why APR must support everything available, not merely what the current
MPM needs.
MPM is really just another module. Its needs are not tbe be-all and
end-all of Apache's needs.
Cheers,
Ben.
--
http://www.apache-ssl.org/ben.html
Re: Moving things around
Posted by Greg Stein <gs...@lyra.org>.
On Sat, 6 May 2000, Sascha Schumann wrote:
> On Fri, May 05, 2000 at 07:42:38AM -0400, rbb@covalent.net wrote:
>...
> > You are going on the assumption that APR only gets what the MPM needs. We
> > decided a few weeks ago that this was the wrong approach. APR gets what
> > the machine provides, and the MPM decision is based on what APR was
> > compiled with and what the user specified. For the prefork MPM, there are
> > two ways to specify this desire to Apache:
> >
> > 1) When configuring APR --disable-threads
> > 2) When configuring Apache --with-mpm=prefork
> >
> > The first method means there is no extra code in APR for threading. The
> > second means APR has threading code but Apache ignores it.
>
> I don't see how this can work. Why the hell do you expect the
> user to know that he must specify --disable-threads, so that he
> can use the prefork MPM?
Those options are exclusive; the user doesn't have to do both.
1) The user has forced APR to not use threads. During the Apache
configuration, it sees that threads are not available, so it resorts to
the prefork MPM by default.
2) APR does get configured with threads, but Apache ignores that fact and
just uses the prefork.
> It makes much more sense to me to let users choose whatever MPM
> they want. Then the MPM tells APR what it needs to operate, and
> not the other way around.
But it makes sense to me to have APR configure as it pleases. Apache then
uses whatever functionality is available, or follows what the user has
told it to do (e.g. select a specific MPM).
IMO, Apache doesn't even need its own thread detection: it should be
possible to determine that from the resulting APR configuration.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: Moving things around
Posted by Sascha Schumann <sa...@schumann.cx>.
On Fri, May 05, 2000 at 07:42:38AM -0400, rbb@covalent.net wrote:
>
> > Let me clarify why that won't work.
> >
> > Everything (including APR and MM) depends on which MPM gets chosen,
> > because it can change *everything* depending on what feature set
> > is associated with the MPM.
>
> You are going on the assumption that APR only gets what the MPM needs. We
> decided a few weeks ago that this was the wrong approach. APR gets what
> the machine provides, and the MPM decision is based on what APR was
> compiled with and what the user specified. For the prefork MPM, there are
> two ways to specify this desire to Apache:
>
> 1) When configuring APR --disable-threads
> 2) When configuring Apache --with-mpm=prefork
>
> The first method means there is no extra code in APR for threading. The
> second means APR has threading code but Apache ignores it.
I don't see how this can work. Why the hell do you expect the
user to know that he must specify --disable-threads, so that he
can use the prefork MPM?
It makes much more sense to me to let users choose whatever MPM
they want. Then the MPM tells APR what it needs to operate, and
not the other way around.
- Sascha
Re: Moving things around
Posted by Sascha Schumann <sa...@schumann.cx>.
> you. My favorite example is you can select a threaded MPM if
> your platform doesn't have threads (and there's no way to configure
> APR to use threads, since it's smart enough to know that threads
> no be here).
That's the good part of using a macro package.
The feature set checks (like POSIX threads) are already available
in macro form.
That means we can "call" APR by using macros. If you look at my
patch, we are doing exactly that to figure out whether Pthreads
are available.
- Sascha
Re: Moving things around
Posted by Sascha Schumann <sa...@schumann.cx>.
On Fri, May 05, 2000 at 07:24:21PM -0700, Greg Stein wrote:
> On Fri, 5 May 2000, dean gaudet wrote:
> > On Fri, 5 May 2000, Sascha Schumann wrote:
> > > Everything (including APR and MM) depends on which MPM gets chosen,
> > > because it can change *everything* depending on what feature set
> > > is associated with the MPM.
> >
> > FWIW, i agree with sascha.
> >
> > it requires somewhat of our old style of configuration to know what MPMs
> > are available on a platform.
>
> I'm with Ryan on this one. APR should be independently configured. It sets
> up a base level of functionality. If Apache and its MPM doesn't use that
> functionality, well TFB.
>
> Remember: APR is an independent entity. It should not rely on outside
> influences during its configuration.
I see APR as a helper library for Apache, compiled each time when
Apache is build.
> Sascha: why doesn't Ryan's description of configuration work? (e.g. config
> APR, then config Apache based on APR capabilities, MPM selected, and other
> config switches) I don't recall seeing a good reason *against* this
> process.
A threaded APR will not work with non-threaded MPMs.
If we are going the route Ryan described, then a user has to:
* choose MPM
* figure out what requirements the MPM has
* configure APR appropiately
I propose:
* choose MPM
* the MPM knows what it needs and configures APR
I don't see any advantage in adding complexity. But perhaps I'm
missing something?
- Sascha
Re: Moving things around
Posted by Greg Stein <gs...@lyra.org>.
On Fri, 5 May 2000, dean gaudet wrote:
> On Fri, 5 May 2000, Sascha Schumann wrote:
> > Everything (including APR and MM) depends on which MPM gets chosen,
> > because it can change *everything* depending on what feature set
> > is associated with the MPM.
>
> FWIW, i agree with sascha.
>
> it requires somewhat of our old style of configuration to know what MPMs
> are available on a platform.
I'm with Ryan on this one. APR should be independently configured. It sets
up a base level of functionality. If Apache and its MPM doesn't use that
functionality, well TFB.
Remember: APR is an independent entity. It should not rely on outside
influences during its configuration.
Sascha: why doesn't Ryan's description of configuration work? (e.g. config
APR, then config Apache based on APR capabilities, MPM selected, and other
config switches) I don't recall seeing a good reason *against* this
process.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: Moving things around
Posted by dean gaudet <dg...@arctic.org>.
On Fri, 5 May 2000, Sascha Schumann wrote:
> Everything (including APR and MM) depends on which MPM gets chosen,
> because it can change *everything* depending on what feature set
> is associated with the MPM.
FWIW, i agree with sascha.
it requires somewhat of our old style of configuration to know what MPMs
are available on a platform.
-dean
Re: Moving things around
Posted by rb...@covalent.net.
> Let me clarify why that won't work.
>
> Everything (including APR and MM) depends on which MPM gets chosen,
> because it can change *everything* depending on what feature set
> is associated with the MPM.
You are going on the assumption that APR only gets what the MPM needs. We
decided a few weeks ago that this was the wrong approach. APR gets what
the machine provides, and the MPM decision is based on what APR was
compiled with and what the user specified. For the prefork MPM, there are
two ways to specify this desire to Apache:
1) When configuring APR --disable-threads
2) When configuring Apache --with-mpm=prefork
The first method means there is no extra code in APR for threading. The
second means APR has threading code but Apache ignores it.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: Moving things around
Posted by Sascha Schumann <sa...@schumann.cx>.
On Wed, May 03, 2000 at 08:08:22AM -0400, Jim Jagielski wrote:
> Sascha Schumann wrote:
> >
> > Hi,
> >
> > attached is a first draft of a flowchart of a important part of
> > the Apache 2.0 build system. Your input is appreciated.
>
> This is nice!
>
> > Now some more text:
> >
> > The MPM determines how Apache and subsequent libraries will be
> > built. Before we can make any qualified assumptions about the
> > build process, we must know which MPM gets chosen.
> >
> > We need to know the requirements (feature sets) each MPM needs to
> > operate. At the beginning, we check which feature sets are
> > available on the target platform.
>
> This implies that APR is configured before the MPM selection
> process is done... I know that we had talked about the order
> of the configures, with, IIRC, the general agreement being it
> should be the reverse of what is is (ie: it should be MM ->
> APR -> Apache) :)
Let me clarify why that won't work.
Everything (including APR and MM) depends on which MPM gets chosen,
because it can change *everything* depending on what feature set
is associated with the MPM.
So the order must be:
Choose MPM
Apache's feature checks (this cannot be dropped unless APR
provides a complete compatibility layer. I don't think that's
necessary, the current system works quite well.)
APR's configure, considering data (i.e. ac_cv_enable_threads)
passed in from the application using APR.
APR will determine whether shared memory support is needed and
runs mm's configure accordingly.
I've attached a patch which reorders some things. In particular:
* MPM's are now considered special modules, so they are treated
before any library/header/function checks.
* mpm/config.m4 was cleaned up to avoid some code redundancy in
the config code of MPMs. Utilizes feature sets now.
* The env variable MPM_HINT is now supported. It should be set in
hints.m4 to the preferred MPM of the target platform.
* APR's threads check has been cleaned up
* Pthread.h is recognized now on OpenBSD. And we can finally
build on FreeBSD 2.2.x. Yeah! :)
- Sascha