You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Edgar Frank <ef...@email.de> on 2009/11/24 16:05:56 UTC
[mod_fcgid] Feedback / Suggestions
Hi dev,
I'd like to suggest to following changes / offer feedback for mod_fcgid:
(1)
mod_fcgid should be capable of specifying an external FCGI server.
Assume a scenario where the backend FCGI server (e.g. PHP with
php-fpm) runs for security considerations prespawned on a separate
machine with a firewall only letting FCGI traffic through.
This is possible with mod_proxy_fcgi or mod_fastcgi (with
mod_proxy_fcgi only being in trunk and mod_fastcgi kinda outdated).
(2)
In conjunction with (1), mod_fcgid should be able to select the backend
server based on request data.
Current, this kind of behaviour is possible with mod_proxy and
mod_proxy_fcgi using either interpolated environment variables and/or
mod_rewrite with the [P] suffix in a RewriteRule.
Unfortunately mod_proxy_fcgi currently exists only in trunk, seems to
be kind of abandoned and has some bugs (I filed two bug reports in BZ
under 48272 and 48273).
As mod_fcgid seems to be the current "weapon of choice", it would be
great to have this feature(s) in mod_fcgid (or mod_proxy_fcgi fixed
otherwise).
Admittedly I have a urgent need for this feature in a production
environment. If this feature can't be implemented (respectivly fixed)
in the near future (which can I fully understand as a software
developer), what would be your suggestion to get an Apache running this
way? I'd really hate it to abandon Apache because of this (and am
willing to do something against it as far as I still have time in my
current project)
(3)
mod_fcgid currently buffers the complete input from the client
(occasionaly in a temp-file if the request is large) before it passes it
through to a FCGI backend. Could this be made configurable in a way like
File|Memory|Pipeline? (file - as is | memory - buffer in memory always
| pipeline - directly pass the read data to the backend)
Or otherwise, can someone explain the details to me why it is as it is?
Especially in terms of not pipeling data directly (maybe after a little
buffering to build proper FCGI packets)? The comment in
fcgid_bridge.c:452 (add_request_body) left me clueless. Why would this
keep the server in processing too long? Processing takes its time either
way, I'd assume. Looking forward to enlightment. :)
(4)
Would it make sense to use the FCGI feature to multiplex several
requests over a single connection? Does any backend support this
feature anyway?
When thinking of an external FCGI backend with a socket connection and
very high Requests/s, this could keep open connections and
used/available ports much lower.
Really looking forward to your comments.
Regards,
Edgar Frank
Re: [mod_fcgid] Feedback / Suggestions
Posted by Olaf van der Spek <ol...@gmail.com>.
On Tue, Nov 24, 2009 at 4:50 PM, pqf <pq...@mailtech.cn> wrote:
> In this case, one httpd thread(process) will have to bind to one FastCGI process.
> I don't think connect() to a local pipe/unix domain socket is the bottle-neck, so let it be?
Doesn't it require an extra round trip? It's probably not too bad, but
would be nice to avoid.
Olaf
Re: [mod_fcgid] Feedback / Suggestions
Posted by Jeff Trawick <tr...@gmail.com>.
On Tue, Nov 24, 2009 at 10:50 AM, pqf <pq...@mailtech.cn> wrote:
>>> (3)
>>> mod_fcgid currently buffers the complete input from the client
>>> (occasionaly in a temp-file if the request is large) before it passes it
>>> through to a FCGI backend. Could this be made configurable in a way like
>>> File|Memory|Pipeline? (file - as is | memory - buffer in memory always
>>> | pipeline - directly pass the read data to the backend)
>>
>> This will definitely be resolved ;)
>>
> Pipeline is a little complex. FastCGI process would write during reading. so
> mod_fcgid should check both read and write on the socket, and read/write while
> the socket is readable/writeable.
> But is do-able :) Any one interesting?
I'm interested, but need to find adequate time. (I can certainly find
time to review and help test somebody else's implementation.)
>
>>>
>>> Or otherwise, can someone explain the details to me why it is as it is?
>>> Especially in terms of not pipeling data directly (maybe after a little
>>> buffering to build proper FCGI packets)? The comment in
>>> fcgid_bridge.c:452 (add_request_body) left me clueless. Why would this
>>> keep the server in processing too long? Processing takes its time either
>>> way, I'd assume. Looking forward to enlightment. :)
>>
>> I can only guess that the problem at hand when this was implemented
>> was that some backend application processes were so expensive that
>> that they couldn't be tied up until all data had been read from slow
>> clients.
>>
> Yes, Jeff is right :)
This is a reasonable feature; once streaming to the app is implemented
this alternate mechanism can be enabled with a per-request envvar
(e.g., SetEnv in the directory or location).
Re: [mod_fcgid] Feedback / Suggestions
Posted by pqf <pq...@mailtech.cn>.
--------------------------------------------------
From: "Jeff Trawick" <tr...@gmail.com>
Sent: Tuesday, November 24, 2009 11:24 PM
To: <de...@httpd.apache.org>
Subject: Re: [mod_fcgid] Feedback / Suggestions
> On Tue, Nov 24, 2009 at 10:05 AM, Edgar Frank <ef...@email.de> wrote:
>> Hi dev,
>>
>> I'd like to suggest to following changes / offer feedback for mod_fcgid:
>
> my 2cents below
>
>>
>> (1)
>> mod_fcgid should be capable of specifying an external FCGI server.
>> (2)
>> In conjunction with (1), mod_fcgid should be able to select the backend
>> server based on request data.
>
> I'd much rather see effort put into mod_proxy_fcgi to support this use
> case. I wish somebody, perhaps myself, had time to work on it. It
> doesn't seem that hard a task.
>
> In the interim, is mod_fastcgi really that bad?
>
>>
>> (3)
>> mod_fcgid currently buffers the complete input from the client
>> (occasionaly in a temp-file if the request is large) before it passes it
>> through to a FCGI backend. Could this be made configurable in a way like
>> File|Memory|Pipeline? (file - as is | memory - buffer in memory always
>> | pipeline - directly pass the read data to the backend)
>
> This will definitely be resolved ;)
>
Pipeline is a little complex. FastCGI process would write during reading. so
mod_fcgid should check both read and write on the socket, and read/write while
the socket is readable/writeable.
But is do-able :) Any one interesting?
>>
>> Or otherwise, can someone explain the details to me why it is as it is?
>> Especially in terms of not pipeling data directly (maybe after a little
>> buffering to build proper FCGI packets)? The comment in
>> fcgid_bridge.c:452 (add_request_body) left me clueless. Why would this
>> keep the server in processing too long? Processing takes its time either
>> way, I'd assume. Looking forward to enlightment. :)
>
> I can only guess that the problem at hand when this was implemented
> was that some backend application processes were so expensive that
> that they couldn't be tied up until all data had been read from slow
> clients.
>
Yes, Jeff is right :)
>> (4)
>> Would it make sense to use the FCGI feature to multiplex several
>> requests over a single connection? Does any backend support this
>> feature anyway?
>
> no idea here
>
In this case, one httpd thread(process) will have to bind to one FastCGI process.
I don't think connect() to a local pipe/unix domain socket is the bottle-neck, so let it be?
>> When thinking of an external FCGI backend with a socket connection and
>> very high Requests/s, this could keep open connections and
>> used/available ports much lower.
>
connect() may be bottle-neck if using external FCGI backend. But now mod_fcgid don't support
external FCGI backend yet.
Re: [mod_fcgid] Feedback / Suggestions
Posted by Jeff Trawick <tr...@gmail.com>.
On Tue, Nov 24, 2009 at 11:08 AM, Olaf van der Spek
<ol...@gmail.com> wrote:
> On Tue, Nov 24, 2009 at 5:03 PM, Jeff Trawick <tr...@gmail.com> wrote:
>>> What advantages does fcgid have over proxy_fcgi (except being ready)?
>>
>> integrated, on-demand process management
>
> How valuable is that?
> In most cases a static number of backends seems fine.
compared with mod_proxy_fcgi, the little or no required
configuration/management of application processes with mod_fcgid makes
it easier for newbies and/or casual users; at the same time it is
sufficient for most sites, though perhaps with a little more config
tweaking to allow higher spawn rates or larger pools per application
>
>>> mod_fcgid isn't in 2.2, right?
>>
>> mod_fcgid is actually not bundled with the HTTP server. It is
>> released on its own cycle, and supports httpd 2.0.x, 2.2.x, and trunk
>> (future httpd 2.4.x) with one delivery.
>
> Ah, nice. What's the reason it's not bundled though?
I expect to see relatively high activity (compared with any particular
bundled module) in mod_fcgid over the next 6 months or so. Given
that,
* bundling means considerable extra work to get fixes into both httpd
trunk and the separate mod_fcgid tree that would continue to support
httpd 2.0.x/2.2.x; that gets worse when httpd 2.4 is GA, as a fix
could apply to httpd trunk, httpd 2.4.x branch, and the separate
mod_fcgid tree
* releasing of mod_fcgid fixes would be held up by the httpd release cycle
>
> In this case, I'd love to see support for TCP/IP backends too.
> Shouldn't be too hard to implement.
>
>>> So what's the plan for 2.4? Have both of them? Or is mod_proxy_fcgi
>>> expected to be not ready for 2.4?
>>
>> mod_fcgid will support 2.4. proxy-fcgi folk(s), care to speak up on your baby?
>
> Olaf
>
Re: [mod_fcgid] Feedback / Suggestions
Posted by Olaf van der Spek <ol...@gmail.com>.
On Tue, Nov 24, 2009 at 5:03 PM, Jeff Trawick <tr...@gmail.com> wrote:
>> What advantages does fcgid have over proxy_fcgi (except being ready)?
>
> integrated, on-demand process management
How valuable is that?
In most cases a static number of backends seems fine.
>> mod_fcgid isn't in 2.2, right?
>
> mod_fcgid is actually not bundled with the HTTP server. It is
> released on its own cycle, and supports httpd 2.0.x, 2.2.x, and trunk
> (future httpd 2.4.x) with one delivery.
Ah, nice. What's the reason it's not bundled though?
In this case, I'd love to see support for TCP/IP backends too.
Shouldn't be too hard to implement.
>> So what's the plan for 2.4? Have both of them? Or is mod_proxy_fcgi
>> expected to be not ready for 2.4?
>
> mod_fcgid will support 2.4. proxy-fcgi folk(s), care to speak up on your baby?
Olaf
Re: [mod_fcgid] Feedback / Suggestions
Posted by Jeff Trawick <tr...@gmail.com>.
On Tue, Nov 24, 2009 at 10:58 AM, Olaf van der Spek
<ol...@gmail.com> wrote:
> On Tue, Nov 24, 2009 at 4:47 PM, Jeff Trawick <tr...@gmail.com> wrote:
>>> What was the reason to import mod_fcgi again? Wasn't the ETA of
>>> mod_proxy_fcgi too high?
>>
>> mod_fcgid was imported because it was
>>
>> * widely used
>> * not actively maintained
>> * httpd developers were willing to adopt it
>>
>> I felt that it was a nice addition particularly because it had a
>> different approach to this important problem space compared with
>> mod_proxy_fcgi.
>
> What advantages does fcgid have over proxy_fcgi (except being ready)?
integrated, on-demand process management
>
>>>
>>>> In the interim, is mod_fastcgi really that bad?
>>>
>>> I assume mod_fcgi wasn't developed without proper reason.
>>
>> So do I. (FWIW, I've put a lot of time into mod_fcgid and expect to
>> continue doing so in the future.)
>>
>> The situation at hand seems to be
>>
>> * mod_proxy_fcgi promises to meet those requirements but falls short at present
>> * mod_fcgid doesn't even try
>> * mod_fastcgi implements at least some of those requirements and is mature
>
> mod_fcgid isn't in 2.2, right?
mod_fcgid is actually not bundled with the HTTP server. It is
released on its own cycle, and supports httpd 2.0.x, 2.2.x, and trunk
(future httpd 2.4.x) with one delivery.
> So what's the plan for 2.4? Have both of them? Or is mod_proxy_fcgi
> expected to be not ready for 2.4?
mod_fcgid will support 2.4. proxy-fcgi folk(s), care to speak up on your baby?
Re: [mod_fcgid] Feedback / Suggestions
Posted by Olaf van der Spek <ol...@gmail.com>.
On Tue, Nov 24, 2009 at 4:47 PM, Jeff Trawick <tr...@gmail.com> wrote:
>> What was the reason to import mod_fcgi again? Wasn't the ETA of
>> mod_proxy_fcgi too high?
>
> mod_fcgid was imported because it was
>
> * widely used
> * not actively maintained
> * httpd developers were willing to adopt it
>
> I felt that it was a nice addition particularly because it had a
> different approach to this important problem space compared with
> mod_proxy_fcgi.
What advantages does fcgid have over proxy_fcgi (except being ready)?
>>
>>> In the interim, is mod_fastcgi really that bad?
>>
>> I assume mod_fcgi wasn't developed without proper reason.
>
> So do I. (FWIW, I've put a lot of time into mod_fcgid and expect to
> continue doing so in the future.)
>
> The situation at hand seems to be
>
> * mod_proxy_fcgi promises to meet those requirements but falls short at present
> * mod_fcgid doesn't even try
> * mod_fastcgi implements at least some of those requirements and is mature
mod_fcgid isn't in 2.2, right?
So what's the plan for 2.4? Have both of them? Or is mod_proxy_fcgi
expected to be not ready for 2.4?
Olaf
Re: [mod_fcgid] Feedback / Suggestions
Posted by Jeff Trawick <tr...@gmail.com>.
On Tue, Nov 24, 2009 at 10:34 AM, Olaf van der Spek
<ol...@gmail.com> wrote:
> On Tue, Nov 24, 2009 at 4:24 PM, Jeff Trawick <tr...@gmail.com> wrote:
>>> (1)
>>> mod_fcgid should be capable of specifying an external FCGI server.
>>> (2)
>>> In conjunction with (1), mod_fcgid should be able to select the backend
>>> server based on request data.
>>
>> I'd much rather see effort put into mod_proxy_fcgi to support this use
>> case. I wish somebody, perhaps myself, had time to work on it. It
>> doesn't seem that hard a task.
>
> What was the reason to import mod_fcgi again? Wasn't the ETA of
> mod_proxy_fcgi too high?
mod_fcgid was imported because it was
* widely used
* not actively maintained
* httpd developers were willing to adopt it
I felt that it was a nice addition particularly because it had a
different approach to this important problem space compared with
mod_proxy_fcgi.
>
>> In the interim, is mod_fastcgi really that bad?
>
> I assume mod_fcgi wasn't developed without proper reason.
So do I. (FWIW, I've put a lot of time into mod_fcgid and expect to
continue doing so in the future.)
The situation at hand seems to be
* mod_proxy_fcgi promises to meet those requirements but falls short at present
* mod_fcgid doesn't even try
* mod_fastcgi implements at least some of those requirements and is mature
Re: [mod_fcgid] Feedback / Suggestions
Posted by Olaf van der Spek <ol...@gmail.com>.
On Tue, Nov 24, 2009 at 4:24 PM, Jeff Trawick <tr...@gmail.com> wrote:
>> (1)
>> mod_fcgid should be capable of specifying an external FCGI server.
>> (2)
>> In conjunction with (1), mod_fcgid should be able to select the backend
>> server based on request data.
>
> I'd much rather see effort put into mod_proxy_fcgi to support this use
> case. I wish somebody, perhaps myself, had time to work on it. It
> doesn't seem that hard a task.
What was the reason to import mod_fcgi again? Wasn't the ETA of
mod_proxy_fcgi too high?
> In the interim, is mod_fastcgi really that bad?
I assume mod_fcgi wasn't developed without proper reason.
Olaf
Re: [mod_fcgid] Feedback / Suggestions
Posted by Barry Scott <ba...@onelan.co.uk>.
Eric Covener wrote:
> On 12/4/09, Barry Scott <ba...@onelan.co.uk> wrote:
>
>> Eric Covener wrote:
>>
>>
>>> On Thu, Dec 3, 2009 at 5:57 AM, Barry Scott <ba...@onelan.co.uk>
>>>
>> wrote:
>>
>>>
>>>> Jeff Trawick wrote:
>>>>
>>>>
>>>>
>>>>> On Tue, Nov 24, 2009 at 10:05 AM, Edgar Frank <ef...@email.de>
>>>>>
>> wrote:
>>
>>>>> In the interim, is mod_fastcgi really that bad?
>>>>>
>>>>>
>>>>>
>>>>>
>>>> mod_fastcgi is fine for handling GET/POST requests, but it fails to
>>>> implement
>>>> Authorization or Authenication.
>>>>
>>>> So yes mod_fastcgi is really bad.
>>>>
>>>>
>>>>
>>> mod_fastcgi has supported this for many years:
>>>
>>> http://www.fastcgi.com/drupal/node/25#FastCgiAuthorizer
>>>
>>>
>> http://www.fastcgi.com/drupal/node/25#FastCgiAuthenticator
>>
>>>
>>>
>> It does not work or I'd have used it. And I tried to make it work.
>> There is a lot of missing code, compare to mod_fcgid implementation
>> of the same.
>>
>
> Simple tests work for me.
>
>
Hmm, Then I must have got something wrong when I tried to get this
working, or you have
patches I don't. When I looked at the sources and compared to mod_fcgid
it looked like there
was code missing. It's too long ago now for me to recall the details to
discuss.
Barry
Re: [mod_fcgid] Feedback / Suggestions
Posted by Eric Covener <co...@gmail.com>.
On 12/4/09, Barry Scott <ba...@onelan.co.uk> wrote:
> Eric Covener wrote:
>
> > On Thu, Dec 3, 2009 at 5:57 AM, Barry Scott <ba...@onelan.co.uk>
> wrote:
> >
> >
> > > Jeff Trawick wrote:
> > >
> > >
> > > > On Tue, Nov 24, 2009 at 10:05 AM, Edgar Frank <ef...@email.de>
> wrote:
> > > >
> > > > In the interim, is mod_fastcgi really that bad?
> > > >
> > > >
> > > >
> > > mod_fastcgi is fine for handling GET/POST requests, but it fails to
> > > implement
> > > Authorization or Authenication.
> > >
> > > So yes mod_fastcgi is really bad.
> > >
> > >
> >
> > mod_fastcgi has supported this for many years:
> >
> > http://www.fastcgi.com/drupal/node/25#FastCgiAuthorizer
> >
> http://www.fastcgi.com/drupal/node/25#FastCgiAuthenticator
> >
> >
> >
> It does not work or I'd have used it. And I tried to make it work.
> There is a lot of missing code, compare to mod_fcgid implementation
> of the same.
Simple tests work for me.
--
Eric Covener
covener@gmail.com
Re: [mod_fcgid] Feedback / Suggestions
Posted by Barry Scott <ba...@onelan.co.uk>.
Eric Covener wrote:
> On Thu, Dec 3, 2009 at 5:57 AM, Barry Scott <ba...@onelan.co.uk> wrote:
>
>> Jeff Trawick wrote:
>>
>>> On Tue, Nov 24, 2009 at 10:05 AM, Edgar Frank <ef...@email.de> wrote:
>>>
>>> In the interim, is mod_fastcgi really that bad?
>>>
>>>
>> mod_fastcgi is fine for handling GET/POST requests, but it fails to
>> implement
>> Authorization or Authenication.
>>
>> So yes mod_fastcgi is really bad.
>>
>
> mod_fastcgi has supported this for many years:
>
> http://www.fastcgi.com/drupal/node/25#FastCgiAuthorizer
> http://www.fastcgi.com/drupal/node/25#FastCgiAuthenticator
>
>
It does not work or I'd have used it. And I tried to make it work.
There is a lot of missing code, compare to mod_fcgid implementation
of the same.
Barry
Re: [mod_fcgid] Feedback / Suggestions
Posted by Eric Covener <co...@gmail.com>.
On Thu, Dec 3, 2009 at 5:57 AM, Barry Scott <ba...@onelan.co.uk> wrote:
> Jeff Trawick wrote:
>>
>> On Tue, Nov 24, 2009 at 10:05 AM, Edgar Frank <ef...@email.de> wrote:
>>
>> In the interim, is mod_fastcgi really that bad?
>>
>
> mod_fastcgi is fine for handling GET/POST requests, but it fails to
> implement
> Authorization or Authenication.
>
> So yes mod_fastcgi is really bad.
mod_fastcgi has supported this for many years:
http://www.fastcgi.com/drupal/node/25#FastCgiAuthorizer
http://www.fastcgi.com/drupal/node/25#FastCgiAuthenticator
--
Eric Covener
covener@gmail.com
Re: [mod_fcgid] Feedback / Suggestions
Posted by Barry Scott <ba...@onelan.co.uk>.
Jeff Trawick wrote:
> On Tue, Nov 24, 2009 at 10:05 AM, Edgar Frank <ef...@email.de> wrote:
>
>
> In the interim, is mod_fastcgi really that bad?
>
>
mod_fastcgi is fine for handling GET/POST requests, but it fails to
implement
Authorization or Authenication.
So yes mod_fastcgi is really bad.
mod_fcgid is a very welcome as a supported httpd module.
Barry
Re: [mod_fcgid] Feedback / Suggestions
Posted by Jeff Trawick <tr...@gmail.com>.
On Tue, Nov 24, 2009 at 10:05 AM, Edgar Frank <ef...@email.de> wrote:
> Hi dev,
>
> I'd like to suggest to following changes / offer feedback for mod_fcgid:
my 2cents below
>
> (1)
> mod_fcgid should be capable of specifying an external FCGI server.
> (2)
> In conjunction with (1), mod_fcgid should be able to select the backend
> server based on request data.
I'd much rather see effort put into mod_proxy_fcgi to support this use
case. I wish somebody, perhaps myself, had time to work on it. It
doesn't seem that hard a task.
In the interim, is mod_fastcgi really that bad?
>
> (3)
> mod_fcgid currently buffers the complete input from the client
> (occasionaly in a temp-file if the request is large) before it passes it
> through to a FCGI backend. Could this be made configurable in a way like
> File|Memory|Pipeline? (file - as is | memory - buffer in memory always
> | pipeline - directly pass the read data to the backend)
This will definitely be resolved ;)
>
> Or otherwise, can someone explain the details to me why it is as it is?
> Especially in terms of not pipeling data directly (maybe after a little
> buffering to build proper FCGI packets)? The comment in
> fcgid_bridge.c:452 (add_request_body) left me clueless. Why would this
> keep the server in processing too long? Processing takes its time either
> way, I'd assume. Looking forward to enlightment. :)
I can only guess that the problem at hand when this was implemented
was that some backend application processes were so expensive that
that they couldn't be tied up until all data had been read from slow
clients.
> (4)
> Would it make sense to use the FCGI feature to multiplex several
> requests over a single connection? Does any backend support this
> feature anyway?
no idea here
> When thinking of an external FCGI backend with a socket connection and
> very high Requests/s, this could keep open connections and
> used/available ports much lower.