You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Enrico Weigelt <we...@metux.de> on 2002/11/26 08:14:53 UTC
trouble w/ perchild MPM
hi folks
i've got some trouble with the perchild MPM in httpd-2.0.43.
Especially forwarding connections to other childs seems to be
a little bit buggy.
an little bug in receive_from_other_child() was:
> iov[0].iov_base = headers;
> iov[0].iov_len = HUGE_STRING_LEN;
> iov[0].iov_base = request_body;
> iov[0].iov_len = HUGE_STRING_LEN;
perhaps should be:
iov[0].iov_base = headers;
iov[0].iov_len = HUGE_STRING_LEN;
iov[1].iov_base = request_body;
iov[1].iov_len = HUGE_STRING_LEN;
with this correction it mostly works, but sometimes the connection
passing fails, no child seems to receive the connection.
also there are some other strange things:
* on connection forwarding, in the receiving child, perchild_post_read()
is called twice, first w/ the right request, then an empty one.
i've modified the code to drop these requests (return DECLINE;)
* sometimes the socket fd (sock->socketdes) changes on return
from receive_from_other_child() to worker_thread to an
enormously high number (i.e.136489464)
could anyone help ?
~-n
--
Bestes Mittel gegen Mailviren: Outlook löschen.
---------------------------------------------------------------------
Enrico Weigelt == metux ITS
Webhosting ab 5 EUR/Monat. UUCP, rawIP und vieles mehr.
phone: +49 36207 519931 www: http://www.metux.de/
fax: +49 36207 519932 email: contact@metux.de
cellphone: +49 174 7066481 smsgate: sms.weigelt@metux.de
---------------------------------------------------------------------
Diese Mail wurde mit UUCP versandt. http://www.metux.de/uucp/
Re: trouble w/ perchild MPM
Posted by Johannes Erdfelt <jo...@erdfelt.com>.
On Tue, Nov 26, 2002, Enrico Weigelt <we...@metux.de> wrote:
> On Tue, Nov 26, 2002 at 07:06:33AM -0500, Jeff Trawick wrote:
>
> hi folks,
>
> <snip>
> > Please grab perchild.c from current cvs and see how much is still
> > broken. Here is one easy place to grab it. Make sure you get the
> > latest version.
> >
> > http://cvs.apache.org/viewcvs.cgi/httpd-2.0/server/mpm/experimental/perchild/perchild.c
> I've tried it yesterday, but there was no new stuff since 2.0.43,
> which i'm currently using.
> So if you've got something more recent, you're welcomed to post
> it to me :)
2.0.43 seems to be tagged at revision 1.132. The latest version is 1.136
which contains numerous important fixes.
> (i currently like mailing diffs much than using CVS, since ip access
> is quite expensive @ my location, but mail forwarding w/ uucp costs
> almost nothing ... or is there an mail gw for CVS ?)
Should be as easy as this retrieving this URL:
http://cvs.apache.org/viewcvs.cgi/*checkout*/httpd-2.0/server/mpm/experimental/perchild/perchild.c?rev=HEAD&content-type=text/plain
That is the latest version of just that file.
JE
Re: trouble w/ perchild MPM
Posted by Enrico Weigelt <we...@metux.de>.
On Tue, Nov 26, 2002 at 07:06:33AM -0500, Jeff Trawick wrote:
hi folks,
<snip>
> Please grab perchild.c from current cvs and see how much is still
> broken. Here is one easy place to grab it. Make sure you get the
> latest version.
>
> http://cvs.apache.org/viewcvs.cgi/httpd-2.0/server/mpm/experimental/perchild/perchild.c
I've tried it yesterday, but there was no new stuff since 2.0.43,
which i'm currently using.
So if you've got something more recent, you're welcomed to post
it to me :)
(i currently like mailing diffs much than using CVS, since ip access
is quite expensive @ my location, but mail forwarding w/ uucp costs
almost nothing ... or is there an mail gw for CVS ?)
> Any debugging you can provide would be much appreciated. perchild
> is one of the potentially cool features of 2.0, but at the moment it
> seems that core Apache developers don't have time to work on it (hey,
> there's a lot of code here :) ). The handful of people including
> yourself trying to debug/use it could make a dramatic difference in
> its viability, just pushing it forward bit by bit until it works. If
> there were a way to get you folks communicating and trying each others
> patches, it could improve pretty quickly.
well i'll do the best i can :)
~-n
--
Bestes Mittel gegen Mailviren: Outlook löschen.
---------------------------------------------------------------------
Enrico Weigelt == metux ITS
Webhosting ab 5 EUR/Monat. UUCP, rawIP und vieles mehr.
phone: +49 36207 519931 www: http://www.metux.de/
fax: +49 36207 519932 email: contact@metux.de
cellphone: +49 174 7066481 smsgate: sms.weigelt@metux.de
---------------------------------------------------------------------
Diese Mail wurde mit UUCP versandt. http://www.metux.de/uucp/
Re: trouble w/ perchild MPM
Posted by Enrico Weigelt <we...@metux.de>.
On Wed, Nov 27, 2002 at 06:55:13AM +0100, Enrico Weigelt wrote:
hi folks,
something seems to currupt the listener list.
i've running two processes under different uid, both processes
have two listeners. (regenerating pollset before each poll and dumping it)
in one process, the second listener has fd 80 and in the other one
an very huge number, both are invalid.
this causes an fast looping in worker_thread(), which eats up
very much resources.
at this point, the server is not able process requests anylonger.
where can this come from ?
~-n
--
Bestes Mittel gegen Mailviren: Outlook löschen.
---------------------------------------------------------------------
Enrico Weigelt == metux ITS
Webhosting ab 5 EUR/Monat. UUCP, rawIP und vieles mehr.
phone: +49 36207 519931 www: http://www.metux.de/
fax: +49 36207 519932 email: contact@metux.de
cellphone: +49 174 7066481 smsgate: sms.weigelt@metux.de
---------------------------------------------------------------------
Diese Mail wurde mit UUCP versandt. http://www.metux.de/uucp/
Re: trouble w/ perchild MPM
Posted by Enrico Weigelt <we...@metux.de>.
On Wed, Nov 27, 2002 at 01:41:14AM +0000, James Ponder wrote:
<snip>
> I meant 1 or more processes with 1 or more threads each. Has any work
> been done on testing whether it's best to have 100 processes listening or
> 10 processes with 10 threads each all listening, or 10 processes with 10
> threads each with one listener thread per process, etc. etc.? I was
> vague because I don't know best practice.
perhaps we could write an test application to probe it out.
<snip>
> I'd like to ditch the current model of using suexec, to a perchild model.
This is also my plan. I'm using mod_php w/ connection pooling etc. The
server processes currently run as nobody, but now i'm getting some
more customers on my hosts and some wann access their files from
php scripts. suexec is too slow, since it forks and execs, which
is quite expensive, much more expensive than passing fds.
I'm currently thinking about an completely new approach:
The Server Layout is:
+-----------------------+
| Master Process (root) |
+-----------------------+
| | | |
| | | |
primary +----------------+ | | |
listen ==> | Multiplexer | | | |
socket | (wwwrun) | | | \-----------\
+----------------+ | \-----------\ |
| | | \-----------\ | |
| | | | | |
| | | +-------------------+ | | |
| | \--| Server 0 (user A) |-/ | |
| | +-------------------+ | |
| | | |
| | +-------------------+ | |
| \-----| Server 1 (user B) |----/ |
| +-------------------+ |
The lines are marking control connections between the processes.
On startup we first have the master process, which forks the multiplexer(s)
and the servers. The multiplexer server listens on an set of sockets
for accepting incoming connections. Once accepted an connection, the
mux looks up the right server for the request and passes the connection
to it, this server then processes the request as usual.
Each of the processes may work w/ multiple threads, perhaps they
could also be configured individualy for each process.
This concept has one possible leak w/ persistent connections,
when an client sends requests for different vhosts over the same
persistant link. (is this allowed by the http spec?)
Well, I'll call it the mux-mpm ...
btw: how to add an new mpm ?
~-n
--
Bestes Mittel gegen Mailviren: Outlook löschen.
---------------------------------------------------------------------
Enrico Weigelt == metux ITS
Webhosting ab 5 EUR/Monat. UUCP, rawIP und vieles mehr.
phone: +49 36207 519931 www: http://www.metux.de/
fax: +49 36207 519932 email: contact@metux.de
cellphone: +49 174 7066481 smsgate: sms.weigelt@metux.de
---------------------------------------------------------------------
Diese Mail wurde mit UUCP versandt. http://www.metux.de/uucp/
Re: trouble w/ perchild MPM
Posted by James Ponder <ja...@squish.net>.
On Tue, Nov 26, 2002 at 08:18:06PM -0500, Johannes Erdfelt wrote:
> > Perhaps perchild can be improved further to have a set of listener
> > processes/threads, under the generic apache user id or another secured id,
> > which accepts connections but does not process them. These threads then
> > pass the connections on to the correct user-owned child id via the named
> > sockets, allowing these child users to have no listening sockets at all
> > (except the unix domain socket).
>
> It would need to be a process. Mixing uid's among threads is not
> portable.
I meant 1 or more processes with 1 or more threads each. Has any work
been done on testing whether it's best to have 100 processes listening or
10 processes with 10 threads each all listening, or 10 processes with 10
threads each with one listener thread per process, etc. etc.? I was
vague because I don't know best practice.
> Switching to a model like is possible, but I'm willing to be that
> passing fd's is not cheap and would just create overhead.
> Do you want to do this for security reasons?
Yes, I look after an Apache installation with hundreds of web sites, most
are name-based. It's a shared infrastructure with dynamic content allowed,
in a single apache instance. Within these parameters we try and make it
as secure as possible. We can use unix-level system resource monitoring
for quite a lot of things, but having all content delivered by a single
user (the apache generic user) presents us with all sorts of problems, not
least of which is protecting that apache user from running user's code
(e.g. we use suexec, disallow execs in ssi, do not use things like mod_perl).
I'd like to ditch the current model of using suexec, to a perchild model. If
there aren't other problems (perhaps other modules creating file descriptors)
then this would seem a sensible way forward, removing the need for suexec and
opening the possibility that we could offer dynamic content module providers
(mod_perl etc) in a secure way to our users.
Best wishes, James
--
James Ponder; www.squish.net; London, UK
Re: trouble w/ perchild MPM
Posted by Johannes Erdfelt <jo...@erdfelt.com>.
On Wed, Nov 27, 2002, James Ponder <ja...@squish.net> wrote:
> On Tue, Nov 26, 2002 at 07:44:34PM -0500, Johannes Erdfelt wrote:
> > This can include shared (sometimes requiring connections to be passed)
> > and non shared (always answered by the child) sockets.
> >
> > I don't particularly see the non shared case as a concern. The shared
> > case can be a problem.
> >
> > If either are a problem, I suspect that perchild is not the MPM you want
> > to use.
>
> Perhaps perchild can be improved further to have a set of listener
> processes/threads, under the generic apache user id or another secured id,
> which accepts connections but does not process them. These threads then
> pass the connections on to the correct user-owned child id via the named
> sockets, allowing these child users to have no listening sockets at all
> (except the unix domain socket).
It would need to be a process. Mixing uid's among threads is not
portable.
Switching to a model like is possible, but I'm willing to be that
passing fd's is not cheap and would just create overhead.
Do you want to do this for security reasons?
JE
Re: trouble w/ perchild MPM
Posted by James Ponder <ja...@squish.net>.
On Tue, Nov 26, 2002 at 07:44:34PM -0500, Johannes Erdfelt wrote:
> This can include shared (sometimes requiring connections to be passed)
> and non shared (always answered by the child) sockets.
>
> I don't particularly see the non shared case as a concern. The shared
> case can be a problem.
>
> If either are a problem, I suspect that perchild is not the MPM you want
> to use.
Perhaps perchild can be improved further to have a set of listener
processes/threads, under the generic apache user id or another secured id,
which accepts connections but does not process them. These threads then
pass the connections on to the correct user-owned child id via the named
sockets, allowing these child users to have no listening sockets at all
(except the unix domain socket).
Best wishes, James
--
James Ponder; www.squish.net; London, UK
Re: trouble w/ perchild MPM
Posted by Johannes Erdfelt <jo...@erdfelt.com>.
On Wed, Nov 27, 2002, James Ponder <ja...@squish.net> wrote:
> On Tue, Nov 26, 2002 at 07:06:33AM -0500, Jeff Trawick wrote:
> > Any debugging you can provide would be much appreciated. perchild
> > is one of the potentially cool features of 2.0, but at the moment it
>
> I'm trying to understand perchild, so I've had a quick look at the source.
> Could you confirm my understanding?
>
> It appears that perchild starts by spawning processes, each process uses the
> inherited table to find out if it should change owner via setuid() and does so
> accordingly. The children then spawn worker threads, all of those threads
> listen simultaneously, serialising on a cross-process mutex. The first process
> that got this mutex accepts the connection, processes it until the post_read
> hook, passes it to another process if necessary via a named socket, and then
> longjmps out of the request so it can abort the connection processing itself.
That's how it currently is implemented.
> My questions are - why does worker use a listener thread, and perchild doesn't?
I suspect because it was originally thought it may not have been needed.
The existing implementation still has a design flaw where moving to a
listener thread model can help solve.
I'm currently in the process of implementing this.
> And secondly, surely allowing all these differently-owned processes access to
> the listening sockets means that any user with gdb can attach to their Apache
> process and intercept requests? or am I missing the purpose of perchild?
That's a limitation of the existing implementation. Each child listens
on all sockets, even if it can't possibly answer a request for it. This
should be relatively easy to fix.
To answer your question about gdb, that is a configuration issue. Once
the code has been changed to not listen to all sockets, the security
implications are limited to sockets that the child is responsible for.
This can include shared (sometimes requiring connections to be passed)
and non shared (always answered by the child) sockets.
I don't particularly see the non shared case as a concern. The shared
case can be a problem.
If either are a problem, I suspect that perchild is not the MPM you want
to use.
JE
Re: trouble w/ perchild MPM
Posted by James Ponder <ja...@squish.net>.
On Tue, Nov 26, 2002 at 07:06:33AM -0500, Jeff Trawick wrote:
> Any debugging you can provide would be much appreciated. perchild
> is one of the potentially cool features of 2.0, but at the moment it
I'm trying to understand perchild, so I've had a quick look at the source.
Could you confirm my understanding?
It appears that perchild starts by spawning processes, each process uses the
inherited table to find out if it should change owner via setuid() and does so
accordingly. The children then spawn worker threads, all of those threads
listen simultaneously, serialising on a cross-process mutex. The first process
that got this mutex accepts the connection, processes it until the post_read
hook, passes it to another process if necessary via a named socket, and then
longjmps out of the request so it can abort the connection processing itself.
My questions are - why does worker use a listener thread, and perchild doesn't?
And secondly, surely allowing all these differently-owned processes access to
the listening sockets means that any user with gdb can attach to their Apache
process and intercept requests? or am I missing the purpose of perchild?
Best wishes, James
--
James Ponder; www.squish.net; London, UK
Re: trouble w/ perchild MPM
Posted by Jeff Trawick <tr...@attglobal.net>.
Enrico Weigelt <we...@metux.de> writes:
> i've got some trouble with the perchild MPM in httpd-2.0.43.
> Especially forwarding connections to other childs seems to be
> a little bit buggy.
>
> an little bug in receive_from_other_child() was:
>
> > iov[0].iov_base = headers;
> > iov[0].iov_len = HUGE_STRING_LEN;
> > iov[0].iov_base = request_body;
> > iov[0].iov_len = HUGE_STRING_LEN;
>
> perhaps should be:
>
> iov[0].iov_base = headers;
> iov[0].iov_len = HUGE_STRING_LEN;
> iov[1].iov_base = request_body;
> iov[1].iov_len = HUGE_STRING_LEN;
>
> with this correction it mostly works, but sometimes the connection
> passing fails, no child seems to receive the connection.
This has already been fixed.
> also there are some other strange things:
>
> * on connection forwarding, in the receiving child, perchild_post_read()
> is called twice, first w/ the right request, then an empty one.
> i've modified the code to drop these requests (return DECLINE;)
>
> * sometimes the socket fd (sock->socketdes) changes on return
> from receive_from_other_child() to worker_thread to an
> enormously high number (i.e.136489464)
>
> could anyone help ?
Please grab perchild.c from current cvs and see how much is still
broken. Here is one easy place to grab it. Make sure you get the
latest version.
http://cvs.apache.org/viewcvs.cgi/httpd-2.0/server/mpm/experimental/perchild/perchild.c
Any debugging you can provide would be much appreciated. perchild
is one of the potentially cool features of 2.0, but at the moment it
seems that core Apache developers don't have time to work on it (hey,
there's a lot of code here :) ). The handful of people including
yourself trying to debug/use it could make a dramatic difference in
its viability, just pushing it forward bit by bit until it works. If
there were a way to get you folks communicating and trying each others
patches, it could improve pretty quickly.
--
Jeff Trawick | trawick@attglobal.net
Born in Roswell... married an alien...