You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Rasmus Lerdorf <ra...@lerdorf.on.ca> on 1999/03/01 06:54:20 UTC
Apache 2 mutiple pools feature request
I have brought this up before. But since development is fast and furious
right now, I thought I would restate it.
Many sites are running multiple httpd pools as different user id's these
days. Many because I advised them to do so because it is the best way to
run an Apache/PHP or Apache/mod_perl server where you want to make sure
that different users writing PHP or mod_perl scripts can not step all over
each other. Running PHP or Perl as a cgi under suExec is of course one
way, but you lose all the performance the module version. PHP also has a
safe_mode feature, but it can be restrictive and confusing and I wouldn't
bet the farm on it. This is one area where Alex Belits' fhttpd server is
very nice (www.fhttpd.org).
It would be nice if this could be specified from a single httpd.conf file
as opposed to forcing people to maintain multiple configuration files.
Something like:
User nobody
MinSpareServers 10
MaxSpareServers 20
StartServers 10
ThreadsPerServer 5
MaxClients 256
<Pool web1>
User web1
MinSpareServers 1
MaxSpareServers 2
StartServers 1
ThreadsPerServer 100
</Pool>
<Pool web2>
User web2
</Pool>
<VirtualHost My.Host>
ServerName My.Host
Pool web1
</VirtualHost>
Or I suppose you can just specify different user id's in the VirtualHost
blocks and figure it out from there, but it might be cleaner to have
distinct pool definition blocks.
-Rasmus
Re: Apache 2 mutiple pools feature request
Posted by Tony Finch <do...@dotat.at>.
Brian Behlendorf <br...@hyperreal.org> wrote:
>
>Formalizing this as something that Apache does makes some sense to me;
>e.g., having a standard interface to back-end daemons for launching (as
>root, setting to a particular UID) / restarting / killing, distributing
>requests, etc. Maybe even a standard back-end protocol, like what jserv
>uses today.
FastCGI?
Tony.
--
f.a.n.finch dot@dotat.at fanf@demon.net
Re: Apache 2 mutiple pools feature request
Posted by Greg Stein <gs...@lyra.org>.
Cliff Skolnick wrote:
>...
> With 5000 web sites on a large server, I would have to run 5000 processes in
> your model all the time. I would much rather the server see that this vhost
> gets it's own address space (process pool) and looks to see if there is an
> active process handling the pool. If there is a process hand off the
> request, and if not fork() a process to handle the request. The ability to
> fork multiple processes for a busy pool, and to reap inactive pools to zero
> processes would need to be a goal.
How about an adaptive method? All vhosts are in threads until a thread
kills the process. The monitor process restarts the process, but signals
that the vhost that killed the process should be process-based.
(this assumes that the status area is immune to process crashes)
Hmm. I guess you couldn't really tell which of the requests killed the
process. I'd say mark them all. Certainly, the monitor should log the
situation. After a pattern develops, the system administrator could
narrow down which vhost (and script!) is causing the damage.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
RE: Apache 2 mutiple pools feature request
Posted by Cliff Skolnick <cl...@steam.com>.
>
> Rather than trying to spawn the different httpd "pools" from a single
> invocation of the server, would having the single config file but with
> multiple invocations be good enough?
It would depend... I think a major issue here is to be able to give a
virtual host it's own address space as apache moves to a multi-threaded
model. As the use of modules like mod_php and mod_perl grows there would be
a risk of a bug that would bring down a whole bunch of threads and their
associated requests. This would not be acceptable for a large ISP running a
shared server. Even IIS4.0 will let you run certain sites in their own
address space, probably because of the real possibility of a buggy .asp page
taking down multiple requests.
With 5000 web sites on a large server, I would have to run 5000 processes in
your model all the time. I would much rather the server see that this vhost
gets it's own address space (process pool) and looks to see if there is an
active process handling the pool. If there is a process hand off the
request, and if not fork() a process to handle the request. The ability to
fork multiple processes for a busy pool, and to reap inactive pools to zero
processes would need to be a goal.
Cliff
> If so, I think <IfDefine> would do the job
>
> On Mon, Mar 01, 1999 at 12:54:20AM -0500, Rasmus Lerdorf wrote:
> [major snippage all over the place]
> > Something like:
> > <Pool web1>
> > User web1
> > </Pool>
> >
> > <Pool web2>
> > User web2
> > </Pool>
> >
> > <VirtualHost My.Host>
> > ServerName My.Host
> > Pool web1
> > </VirtualHost>
>
> This could be done using:
>
> <IfDefine web1>
> Listen 10.0.0.1:80
> <VirtualHost My.Host>
> ServerName My.Host
> User web1
> </VirtualHost>
> </IfDefine>
>
> <IfDefine web2>
> Listen 10.0.0.2:80
> <VirtualHost The.Other.Guys.Host>
> ServerName The.Other.Guys.Host
> User web2
> </VirtualHost>
> </IfDefine>
>
> Then you run:
>
> httpd -Dweb1
> httpd -Dweb2
>
> Does this work? Naturally, you still don't get support for different
> pools listening on the same socket.
>
> --
> Manoj Kasichainula - manojk at io dot com - http://www.io.com/~manojk/
> "How can one live in this age and not be curious?" -- Charles Krauthammer
>
Re: Apache 2 mutiple pools feature request
Posted by Manoj Kasichainula <ma...@io.com>.
Rather than trying to spawn the different httpd "pools" from a single
invocation of the server, would having the single config file but with
multiple invocations be good enough?
If so, I think <IfDefine> would do the job
On Mon, Mar 01, 1999 at 12:54:20AM -0500, Rasmus Lerdorf wrote:
[major snippage all over the place]
> Something like:
> <Pool web1>
> User web1
> </Pool>
>
> <Pool web2>
> User web2
> </Pool>
>
> <VirtualHost My.Host>
> ServerName My.Host
> Pool web1
> </VirtualHost>
This could be done using:
<IfDefine web1>
Listen 10.0.0.1:80
<VirtualHost My.Host>
ServerName My.Host
User web1
</VirtualHost>
</IfDefine>
<IfDefine web2>
Listen 10.0.0.2:80
<VirtualHost The.Other.Guys.Host>
ServerName The.Other.Guys.Host
User web2
</VirtualHost>
</IfDefine>
Then you run:
httpd -Dweb1
httpd -Dweb2
Does this work? Naturally, you still don't get support for different
pools listening on the same socket.
--
Manoj Kasichainula - manojk at io dot com - http://www.io.com/~manojk/
"How can one live in this age and not be curious?" -- Charles Krauthammer
Re: Apache 2 mutiple pools feature request
Posted by "Michael H. Voase" <mv...@midcoast.com.au>.
Can I make a segestion at this point in the discussion .
The topic of running multiple interpreters using seperately
configured http servers was something that spurred me into
kludging mod_cgi into mod_cgisock . When I looked into the
situation of running mod_perl with apache , it sturck me
that seperating out the interpreter from the http server
would prevent the interpreter from wandering around in
apache's memory space possibly causing problems . Also if
one is using a language that is capable of implementing
unix domain sockets ( tcl and java notable exceptions ) one
can write seperate deamons to service http cgi requests .
Before anyone bites my head off , I am aware that cgisock
places a lot of limitations on what a cgi programmer can
and cannot do with their script but it does provide an
insulating layer between apache and the interpreter such
that it is difficult for a wayward script to cause
apache much grief . I am currently working on a cgisock
server wrapper for a perl interpreter written in C to
hide the more gruesome details of cgisock services from the
cgi programmer if they want that , however it is not a far
stretch of the imagination to implement a simmilar wrapper
for tcl,java,PHP,basic or whatever . The end result is that
one can run a http server without the need to
configure dynamically loaded modules or seperate builds
of apache and still retain a fairly modest memory footprint .
( the other bonus is a finer grain of control of how many
threads / processes per URL you want , independent of how
many http servers one is running )
While I have your ear for the moment , I am also looking for
comments on the concept of an interactive http connection that
can serve multiple sequential requests overs the single http
connection . It is an idea I am still trying to determine wether
it is worth persuing or dropping . The idea was raised a couple
of weeks ago on apache-modules and it is something that I think
can be implemented using using code from mod_cgi , mod_perl ,
mod_cgisock
and/or http_core . The original segestion was for an interactive
'shell' and I am wary of having a http connection wait around
for user input , however the idea of a browser polling serveral
requests down the same socket could provide interesting results .
If anyone has any comments on this topic or wishes to tell me
I am 're-inventing' the wheel again , all comments would be
gratefully welcome .
just a couple of thoughts anyway..
Cheers Mik Voase.
Brian Behlendorf wrote:
>
> For what it's worth, the jserv developers have the same problem. Of
> course, they have an advantage, in that their model is to run a backend
> daemon for java requests, and it's not difficult to configure more than
> one daemon running as different UID's.
>
> Formalizing this as something that Apache does makes some sense to me;
> e.g., having a standard interface to back-end daemons for launching (as
> root, setting to a particular UID) / restarting / killing, distributing
> requests, etc. Maybe even a standard back-end protocol, like what jserv
> uses today.
>
> I'm sure the Perl folks would prefer to write their daemon in
> multithreaded Perl, and of course the jserv developers are already using a
> JVM for this, so I don't think we'd need to go so far as create daemon
> stub code or anything. This would be a real boon to those people being
> asked to run PHP *and* jserv *and* mod_perl on their systems (like me at
> taz :) - instead of a mod_perl, mod_php and mod_jserv, we just have
> mod_relay which relays a compiled version of the request (or subrequest)
> to the backend daemon based on request parameters, and awaits the
> response.
>
> Yes, you have the IPC overhead, but on modern architectures that's less
> significant. Or so I've been led to believe - the benchmarks out of
> mod_jserv are pretty nice.
>
> This is still relevant when we go multithreaded in 2.0, too. It'll be
> nice to insulate the raw HTTP engine from dangerous code in dynamic
> content engines which may be less robust.
>
> Rasmus, what do you think? I bet you could get started by writing
> something using the jserv protocol already :)
>
> Brian
>
--
----------------------------------------------------------------------------
/~\ /~\ CASTLE INDUSTRIES PTY. LTD.
| |_____| | Incorporated 1969. in N.S.W., Australia
| | Phone +612 6562 1345 Fax +612 6567 1449
| /~\ | Web http://www.midcoast.com.au/~mvoase
| [ ] | Michael H. Voase. Director.
~~~~~~~~~~~~~~ I wouldn't have Windoze for *nix .
----------------------------------------------------------------------------
Re: Apache 2 mutiple pools feature request
Posted by "Michael H. Voase" <mv...@midcoast.com.au>.
> > Things like the ability to do a sub-request,
> > per-directory or per-location configuration directives, access to the
> > various ap_table lists, apache_note passing to the logging module and
> > other custom modules along with inheriting the log file descriptors and
> > being able to write to Apache's log files plus a bunch of other stuff I
> > can't think of right now. I suppose an IPC protocol could be built to do
> > pass all this around along with all the logic to control the number of
> > instances of the daemon, but it seems like a lot of duplicated work when
> > this has all been done in the Apache code already.
>
Keep 'em coming , Im listening to this so just keep
rattling on with what you want and I'll look at what I
can do . Remeber that cgisock is an IPC protocol and
can supply most of the things you are asking without
the incumberances of FastCGGI . Just keep the segestions
rolling in and look out for cgisock V 0.2.0
Cheers Mik Voase.
--
----------------------------------------------------------------------------
/~\ /~\ CASTLE INDUSTRIES PTY. LTD.
| |_____| | Incorporated 1969. in N.S.W., Australia
| | Phone +612 6562 1345 Fax +612 6567 1449
| /~\ | Web http://www.midcoast.com.au/~mvoase
| [ ] | Michael H. Voase. Director.
~~~~~~~~~~~~~~ Cause Linux Flies and Windoze Dies ... 'nuf
said.
----------------------------------------------------------------------------
Re: Apache 2 mutiple pools feature request
Posted by Costin Manolache <co...@tdiinc.com>.
> > Having IIOP as wire protocol and a good API that allows access to the full
> > Apache API will solve a lot of problems ( and you get for free many years of
> > research and many existing implementations ).
>
> I've not had much experience with Corba, but those who have say it's too
> slow and complex for server-side apps concerned about speed. I think it'd
> be great to see if that was due to the protocol itself, or simply due to
> poor implementations. But it'll be tough to beat a custom protocol.
Again, I don't want to argue with people knowing more than I do.
Simpler doesn't always means faster - any custom protocol ( including AJP )
will need a lot of time to reach the optimizations done currently in IIOP
( where a lot of people had a lot of fun).
Plus the protocol is at least designed with speed in mind: byteorder negotiation,
can be used over network or via direct calls, ORBs can optimized the
calling path, etc.
Sooner or later you will need authentication, encryption, etc - all this is
available in the corba design ( and sometimes in free implementations).
So I think it will be tough for a custom protocol to beat corba - many smart
people spent lots of time designing it.
And you will be able to write "modules" in any language with an ORB available.
( some early Perl or Tcl Orbs had few 100 lines of code - of course no
optimization
and only basic support )
> > I tried to create a "mod_corba" using Mico and it worked fine, even in
> > non-multithreaded model. ( the only problem: mico uses C++ - but my
> > target was to allow a Java program call Apache APIs - it worked fine as
> > a demo - and it was almost easy to implement it).
>
> Is this a module you can share? I'd at least like to try and get my hands
> dirty with it...
It's just an experiment. I don't like the current unidirectional protocol used
for
Java servlets (AJP1) - and Corba is something I tried. Right now I'm using
JNI to run a virtual machine in the same process. ( again, without multithreading
it will not fly, but apache-apr seems to work).
(of course, I can share it if someone is interested - I'll just have to clean it
up first,
and make sure it still works - I still keep it as a backup solution and I
do small changes as I work to the other project)
Regards,
Costin
Re: Apache 2 mutiple pools feature request
Posted by Brian Behlendorf <br...@hyperreal.org>.
On Wed, 3 Mar 1999, Costin Manolache wrote:
> but it seems you just described Corba ( or at least GIOP).
> Having IIOP as wire protocol and a good API that allows access to the full
> Apache API will solve a lot of problems ( and you get for free many years of
> research and many existing implementations ).
I've not had much experience with Corba, but those who have say it's too
slow and complex for server-side apps concerned about speed. I think it'd
be great to see if that was due to the protocol itself, or simply due to
poor implementations. But it'll be tough to beat a custom protocol.
> I tried to create a "mod_corba" using Mico and it worked fine, even in
> non-multithreaded model. ( the only problem: mico uses C++ - but my
> target was to allow a Java program call Apache APIs - it worked fine as
> a demo - and it was almost easy to implement it).
Is this a module you can share? I'd at least like to try and get my hands
dirty with it...
Brian
Re: Apache 2 mutiple pools feature request
Posted by Costin Manolache <co...@tdiinc.com>.
> Perhaps the most flexible thing to do is provide an API which works for
> either the single-process space and the multi-process space. E.g.
>
> Apache2 multithread daemon -API- module
>
> Apache2 multithread daemon -TCP- daemonprocess -API- module
>
> So let me craft the discussion like this - is there an API we could write,
> for dynamic content engines specifically (forget about cool stuff like
> being able to create your own mod_status output just for a sec), that
> could optionally and efficiently exist on the other side of a TCP
> connection from the main HTTP server. I think we can do that... and if
> you work on the API, I bet others will figure out the wire protocol side
> of things....
Hi,
I'm new to the list and I may have no ideea what you are talking about,
but it seems you just described Corba ( or at least GIOP).
Having IIOP as wire protocol and a good API that allows access to the full
Apache API will solve a lot of problems ( and you get for free many years of
research and many existing implementations ).
I tried to create a "mod_corba" using Mico and it worked fine, even in
non-multithreaded
model. ( the only problem: mico uses C++ - but my target was to allow a Java
program
call Apache APIs - it worked fine as a demo - and it was almost easy to implement
it).
Regards,
Costin
Re: Apache 2 mutiple pools feature request
Posted by Brian Behlendorf <br...@hyperreal.org>.
> > In fact we're really going to want something equivalent to a wire protocol
> > approximating the Apache API anyways, for robustness, when we go
> > multithreaded. Apache is legendary for its stability, which owes no small
> > debt to the multiprocess model, the fact that bad code only kills its own
> > single connection. When we go multithreaded, we run the risk of someone's
> > bad module code (or worse, someone's bad perl code when used with
> > mod_perl, or bad PHP plug-in code...) bringing down the whole server,
> > clearly an untenable solution for ISPs or most anyone else. Having N
> > processes with M threads only reduces the problem by 1/N, and since we're
> > going multithreaded to try and keep N as close as we can to 1....
>
> I think as long as N>1 but still closer to 1 than in the pure
> multi-process model, we still retain the stability while gaining some of
> the benefits of the threaded model, assuming the crashing problem is
> occasional.
I guess I'm just not of the opinion that the crashing problem will be
occasional, *when* third party modules are in use. I am all set to
believe we can make the standard module set robust enough to run dangerous
like this - but I think it'd be really nice to tell module authors they
have a separate process space that's all their own to mess up.
> How would this relay protocol work on WIN32? At least on Win95/98 having
> two processes running trying to speak to each other is nowhere near as
> efficient as loading a DLL into the main process.
Granted. This is worth asking the Java folks, at least asking how it's
impacted them on NT. Win98/95 I could care less about performance, the
main reason to run a web server there is for development/testing. So it
should work OK but latencies can be much higher and still be acceptable.
Perhaps the most flexible thing to do is provide an API which works for
either the single-process space and the multi-process space. E.g.
Apache2 multithread daemon -API- module
Apache2 multithread daemon -TCP- daemonprocess -API- module
That way people can decide whether they trust "module" enough to be in the
same process space, or outside of it.
> I also don't quite understand where the line between needing a backend
> server vs. being a true module is drawn. For example, would mod_status
> now have to be rewritten as a standalone server? What about mod_proxy?
No, I'd leave those in the core.
> mod_dav? mod_cgi? The last two would definitely benefit from the
> different-user ability that this would bring.
mod_cgi already has suexec, and since mod_cgi is just another relay
module in essence, there's not much need to split it out.
mod_dav, I don't know. It can probably use something like suexec to
perform user-specific functions when necessary.
I guess the place I draw the line is programmability. If the module is an
interpreter for a programming language, it probably makes sense to give it
its own process space. So maybe what I'm advocating is just a distinct
API for dynamic content engines - and that certainly doesn't need all the
power of the Apache API.
> The other issue is that this is radically different from how all the other
> servers work. We are currently working on an ISAPI, NSAPI, WSAPI, Apache
> API abstraction layer which will move all server API related code into a
> thin layer outside of PHP and thus making PHP a generic web server module
> that can be used with any server. Adding code to have PHP run as a
> standalone daemon probably wouldn't be too hard, but architecturally it
> would be a bit of a stretch.
Well it sounds like the interface that "thin layer" provides is an
excellent candidate for a wire protocol. :)
More seriously... it sounds like it would be nothing but a benefit to PHP,
because then PHP would be controlling things like shared memory,
its own process management, protecting itself from other server modules,
etc. And then you'd get the per-UID idea for "free" on all servers;
otherwise you'd have to make this same separate-pools suggestion to all
the other server developers too.
> If this IPC protocol thing does become a reality, I would probably just
> provide it as an option for large ISP's who need the security benefits
> it brings. For large dedicated single-user sites, I doubt the IPC
> approach can come anywhere close to the speed the true module architecture
> brings so I think this model is likely to always be the primary approach.
The main performance slowdown I see is the extra I/O involved in copying
the response from the backend daemon, out to the HTTP connection. I don't
have an easy answer for that; ISTR that we don't really want to give
modules direct access to the socket anymore anyways, due to HTTP/1.1
issues.
So let me craft the discussion like this - is there an API we could write,
for dynamic content engines specifically (forget about cool stuff like
being able to create your own mod_status output just for a sec), that
could optionally and efficiently exist on the other side of a TCP
connection from the main HTTP server. I think we can do that... and if
you work on the API, I bet others will figure out the wire protocol side
of things....
Brian
Re: Apache 2 mutiple pools feature request
Posted by Rasmus Lerdorf <ra...@lerdorf.on.ca>.
> Having "multiple pools" of different HTTPD children lying around just
> doesn't sound like good engineering to me. First off, you have httpd
> processes running as a different UID's, and the implications for shared
> memory between these pools seem nontrivial. Next you need to come up with
> an efficient way for processes to pass accepted connections from one to
> another, since they're all listening on the same port. Finally how this
> plugs into a multithreaded 2.0 architecture puzzles me further.
It would be cool if they could do some descriptor passing and listen to
the same port, but even without that it would still be useful. Require
each pool to have either a separate port or a separate IP and treat them
as completely independent Apache instances.
> In fact we're really going to want something equivalent to a wire protocol
> approximating the Apache API anyways, for robustness, when we go
> multithreaded. Apache is legendary for its stability, which owes no small
> debt to the multiprocess model, the fact that bad code only kills its own
> single connection. When we go multithreaded, we run the risk of someone's
> bad module code (or worse, someone's bad perl code when used with
> mod_perl, or bad PHP plug-in code...) bringing down the whole server,
> clearly an untenable solution for ISPs or most anyone else. Having N
> processes with M threads only reduces the problem by 1/N, and since we're
> going multithreaded to try and keep N as close as we can to 1....
I think as long as N>1 but still closer to 1 than in the pure
multi-process model, we still retain the stability while gaining some of
the benefits of the threaded model, assuming the crashing problem is
occasional.
> A more robust model is one where we have a stripped-down, basic
> multithreaded engine that handles a core set of requests (everything we
> can "guarantee" is thread-safe, i.e. what's in the modules/standard/
> base), and then pass off anything more complex to backend daemons.
> Nicely, this "pass off anything more complex" is precisely the same
> problem we will be facing in deciding how to facilitate speed freaks who
> want to put the basic HTTP engine into kernel-loadable modules,
> calling out outside the kernel to userland for complex responses.
>
> Think of it as inetd on steroids. :) I think this is really the better
> way to go, long term. And I don't see why the dynamic content engine
> teams (perl, java, php) couldn't work together on a good relay protocol
> that got them most of what they needed from the Apache API, and for what
> they didn't get, provided a reasonable alternative.
>
> It's useful to start doing this now, so we can understand better what we
> need the Apache2 binary API to need, as distinguished from what can be
> accomplished in the relay protocol. And there's no reason we couldn't
> start playing with the relay protocol before 2.0's out, too.
How would this relay protocol work on WIN32? At least on Win95/98 having
two processes running trying to speak to each other is nowhere near as
efficient as loading a DLL into the main process.
I also don't quite understand where the line between needing a backend
server vs. being a true module is drawn. For example, would mod_status
now have to be rewritten as a standalone server? What about mod_proxy?
mod_dav? mod_cgi? The last two would definitely benefit from the
different-user ability that this would bring.
The other issue is that this is radically different from how all the other
servers work. We are currently working on an ISAPI, NSAPI, WSAPI, Apache
API abstraction layer which will move all server API related code into a
thin layer outside of PHP and thus making PHP a generic web server module
that can be used with any server. Adding code to have PHP run as a
standalone daemon probably wouldn't be too hard, but architecturally it
would be a bit of a stretch.
If this IPC protocol thing does become a reality, I would probably just
provide it as an option for large ISP's who need the security benefits
it brings. For large dedicated single-user sites, I doubt the IPC
approach can come anywhere close to the speed the true module architecture
brings so I think this model is likely to always be the primary approach.
-Rasmus
Re: Apache 2 mutiple pools feature request
Posted by Brian Behlendorf <br...@hyperreal.org>.
On Mon, 1 Mar 1999, Rasmus Lerdorf wrote:
> > For what it's worth, the jserv developers have the same problem. Of
> > course, they have an advantage, in that their model is to run a backend
> > daemon for java requests, and it's not difficult to configure more than
> > one daemon running as different UID's.
>
> But in my case my backend daemon is an Apache process. If I simply split
> out the PHP code and ran that as a separate process all I would have is
> FastCGI which I can do already. If I do that, I lose a lot of the nifty
> features available to me due to the fact that I am part of the httpd
> process itself. Things like the ability to do a sub-request,
> per-directory or per-location configuration directives, access to the
> various ap_table lists, apache_note passing to the logging module and
> other custom modules along with inheriting the log file descriptors and
> being able to write to Apache's log files plus a bunch of other stuff I
> can't think of right now. I suppose an IPC protocol could be built to do
> pass all this around along with all the logic to control the number of
> instances of the daemon, but it seems like a lot of duplicated work when
> this has all been done in the Apache code already.
Well, a lot of that work is going on today in the Java Apache project, and
aside from naming conventions (that could be changed), it doesn't look all
that Java-specific - see the AJP2.0 proposed spec at
http://java.apache.org/jserv/protocol/AJPv21.html
I certainly won't point to it as a paragon for everything needed to
duplicate the Apache API; but then I don't think doing everything the
API can do is necessary, either.
Having "multiple pools" of different HTTPD children lying around just
doesn't sound like good engineering to me. First off, you have httpd
processes running as a different UID's, and the implications for shared
memory between these pools seem nontrivial. Next you need to come up with
an efficient way for processes to pass accepted connections from one to
another, since they're all listening on the same port. Finally how this
plugs into a multithreaded 2.0 architecture puzzles me further.
In fact we're really going to want something equivalent to a wire protocol
approximating the Apache API anyways, for robustness, when we go
multithreaded. Apache is legendary for its stability, which owes no small
debt to the multiprocess model, the fact that bad code only kills its own
single connection. When we go multithreaded, we run the risk of someone's
bad module code (or worse, someone's bad perl code when used with
mod_perl, or bad PHP plug-in code...) bringing down the whole server,
clearly an untenable solution for ISPs or most anyone else. Having N
processes with M threads only reduces the problem by 1/N, and since we're
going multithreaded to try and keep N as close as we can to 1....
A more robust model is one where we have a stripped-down, basic
multithreaded engine that handles a core set of requests (everything we
can "guarantee" is thread-safe, i.e. what's in the modules/standard/
base), and then pass off anything more complex to backend daemons.
Nicely, this "pass off anything more complex" is precisely the same
problem we will be facing in deciding how to facilitate speed freaks who
want to put the basic HTTP engine into kernel-loadable modules,
calling out outside the kernel to userland for complex responses.
Think of it as inetd on steroids. :) I think this is really the better
way to go, long term. And I don't see why the dynamic content engine
teams (perl, java, php) couldn't work together on a good relay protocol
that got them most of what they needed from the Apache API, and for what
they didn't get, provided a reasonable alternative.
It's useful to start doing this now, so we can understand better what we
need the Apache2 binary API to need, as distinguished from what can be
accomplished in the relay protocol. And there's no reason we couldn't
start playing with the relay protocol before 2.0's out, too.
Brian
Re: Apache 2 mutiple pools feature request
Posted by Rasmus Lerdorf <ra...@lerdorf.on.ca>.
> For what it's worth, the jserv developers have the same problem. Of
> course, they have an advantage, in that their model is to run a backend
> daemon for java requests, and it's not difficult to configure more than
> one daemon running as different UID's.
But in my case my backend daemon is an Apache process. If I simply split
out the PHP code and ran that as a separate process all I would have is
FastCGI which I can do already. If I do that, I lose a lot of the nifty
features available to me due to the fact that I am part of the httpd
process itself. Things like the ability to do a sub-request,
per-directory or per-location configuration directives, access to the
various ap_table lists, apache_note passing to the logging module and
other custom modules along with inheriting the log file descriptors and
being able to write to Apache's log files plus a bunch of other stuff I
can't think of right now. I suppose an IPC protocol could be built to do
pass all this around along with all the logic to control the number of
instances of the daemon, but it seems like a lot of duplicated work when
this has all been done in the Apache code already.
> Formalizing this as something that Apache does makes some sense to me;
> e.g., having a standard interface to back-end daemons for launching (as
> root, setting to a particular UID) / restarting / killing, distributing
> requests, etc. Maybe even a standard back-end protocol, like what jserv
> uses today.
I am sure this is useful, but it applies more to mod_jserv's architecture
than to PHP and mod_perl. mod_perl is even less compatible with an
architecture like this since it is tightly integrated with the Apache API
and lets you get right down into the guts of Apache. Doing this over IPC
doesn't seem feasible to me.
-Rasmus
Re: Apache 2 mutiple pools feature request
Posted by Brian Behlendorf <br...@hyperreal.org>.
For what it's worth, the jserv developers have the same problem. Of
course, they have an advantage, in that their model is to run a backend
daemon for java requests, and it's not difficult to configure more than
one daemon running as different UID's.
Formalizing this as something that Apache does makes some sense to me;
e.g., having a standard interface to back-end daemons for launching (as
root, setting to a particular UID) / restarting / killing, distributing
requests, etc. Maybe even a standard back-end protocol, like what jserv
uses today.
I'm sure the Perl folks would prefer to write their daemon in
multithreaded Perl, and of course the jserv developers are already using a
JVM for this, so I don't think we'd need to go so far as create daemon
stub code or anything. This would be a real boon to those people being
asked to run PHP *and* jserv *and* mod_perl on their systems (like me at
taz :) - instead of a mod_perl, mod_php and mod_jserv, we just have
mod_relay which relays a compiled version of the request (or subrequest)
to the backend daemon based on request parameters, and awaits the
response.
Yes, you have the IPC overhead, but on modern architectures that's less
significant. Or so I've been led to believe - the benchmarks out of
mod_jserv are pretty nice.
This is still relevant when we go multithreaded in 2.0, too. It'll be
nice to insulate the raw HTTP engine from dangerous code in dynamic
content engines which may be less robust.
Rasmus, what do you think? I bet you could get started by writing
something using the jserv protocol already :)
Brian
On Mon, 1 Mar 1999, Rasmus Lerdorf wrote:
> I have brought this up before. But since development is fast and furious
> right now, I thought I would restate it.
>
> Many sites are running multiple httpd pools as different user id's these
> days. Many because I advised them to do so because it is the best way to
> run an Apache/PHP or Apache/mod_perl server where you want to make sure
> that different users writing PHP or mod_perl scripts can not step all over
> each other. Running PHP or Perl as a cgi under suExec is of course one
> way, but you lose all the performance the module version. PHP also has a
> safe_mode feature, but it can be restrictive and confusing and I wouldn't
> bet the farm on it. This is one area where Alex Belits' fhttpd server is
> very nice (www.fhttpd.org).
>
> It would be nice if this could be specified from a single httpd.conf file
> as opposed to forcing people to maintain multiple configuration files.
>
> Something like:
>
> User nobody
> MinSpareServers 10
> MaxSpareServers 20
> StartServers 10
> ThreadsPerServer 5
> MaxClients 256
>
> <Pool web1>
> User web1
> MinSpareServers 1
> MaxSpareServers 2
> StartServers 1
> ThreadsPerServer 100
> </Pool>
>
> <Pool web2>
> User web2
> </Pool>
>
> <VirtualHost My.Host>
> ServerName My.Host
> Pool web1
> </VirtualHost>
>
>
> Or I suppose you can just specify different user id's in the VirtualHost
> blocks and figure it out from there, but it might be cleaner to have
> distinct pool definition blocks.
>
> -Rasmus
>