You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by "Helmut K. C. Tessarek" <te...@evermeet.cx> on 2017/07/06 18:08:14 UTC
mod_proxy_fcgi and flush
One of the comments on the documentation page of mod_proxy_fcgi
(http://httpd.apache.org/docs/2.4/mod/mod_proxy_fcgi.html) mentions an
issue with flush:
There is just no flush support it seems. I attempt to use PHP flush()
and it won't work until you fill up a buffer first, rendering Server
Sent Events impossible with proxy_fcgi. It worked very well with
"-flush" with mod_fastcgi FastCgiExternalServer.
If this is really the case, all web applications that use push
notifications won't be working with mod_proxy_fcgi.
The wiki page on https://wiki.apache.org/httpd/php also suggests to use
mod_proxy_fcgi, but if above is true, this might lead to problems.
What is your take on this?
--
regards Helmut K. C. Tessarek
lookup https://sks-keyservers.net/i for KeyID 0xC11F128D
/*
Thou shalt not follow the NULL pointer for chaos and madness
await thee at its end.
*/
Re: mod_proxy_fcgi and flush
Posted by "Helmut K. C. Tessarek" <te...@evermeet.cx>.
Hi Luca,
Thanks for looking into this.
On 2017-07-08 03:51, Luca Toscano wrote:
> I checked mod_fcgi as Helmut suggested and it seems to me that the
> -flush feature is a simple "flush every data that you receive", so I
> tested the following patch with Jacob's php example code and it seems
> doing what Helmut asked (please correct me if I am wrong).
No, you are correct. But I was merely referring to the comment on the
documentation page.
But yes, this seems to mirror the "-flush" option that is present for
FastCgiExternalServer.
> Caveat: I had to set output_buffering = Off in my php-fpm's php.ini
> config file.
This should be ok, since people who are using output buffering are
usually aware that it can influence the app's behaviour. I think that
was even stated in the PHP manual somewhere.
Although I also remember that there was a section that states that both
ob_flush() and flush() have to be called to make flush work with output
buffering on.
> So if what I wrote vaguely makes sense, we might think about adding a
> new directive that enables explicit flushing to mod_proxy_fcgi, so
> admins can twist its behavior if needed?
As far as I can tell it makes sense, but I haven't gone through the
entire proxy code yet.
Are you thinking of a directive like ProxyFCGIFlush or an option to the
proxy worker:
<Proxy "fcgi://localhost/" flush=yes>
</Proxy>
> Completely agree with Jacob, this use case might not be the best one for
> HTTP :)
As mentioned before, in a perfect world I'd agree. But this option was
available with FastCgiExternalServer, thus it should be available with
proxy_fcgi as well, especially when proxy_fcgi is the preferred way for
Apache >= 2.4.
Cheers,
K. C.
--
regards Helmut K. C. Tessarek
lookup https://sks-keyservers.net/i for KeyID 0xC11F128D
/*
Thou shalt not follow the NULL pointer for chaos and madness
await thee at its end.
*/
Re: mod_proxy_fcgi and flush
Posted by Ruediger Pluem <rp...@apache.org>.
On 07/13/2017 07:36 PM, Luca Toscano wrote:
> I didn't find a real use case for flushpackets=auto in mod_proxy_fcgi, but I have two proposals for fluspackets=on:
>
> http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-force_flush.patch
> http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-force_flush-v2.patch
>
> The second one is a bit more conservative and does not insert the FLUSH bucket if it knows that the content len (clen)
> is bigger than the iobuf_size (since there is a goto that forces another get_data() in this case).
I only looked at it very briefly, but from that I would favor v2 of the patch.
Regards
Rüdiger
Re: mod_proxy_fcgi and flush
Posted by "Helmut K. C. Tessarek" <te...@evermeet.cx>.
Hi Luca,
On 2017-07-13 13:36, Luca Toscano wrote:
> It would be interesting to also know from Helmut a simple (PHP?) use
> case to use as a test, so I'll not base my code only on Jacob's example
> (that was really useful though!!).
I don't have a specific example. I referred to a comment on the
proxy_fcgi documentation web page in my first post in this thread.
It struck me as odd that flushing was possible with an option for
FastCgiExternalServer, but apparently not available in a proxy_fcgi setup.
I recently switched my setup to an event mpm with php-fpm and
proxy_fcgi. I'm also still working out a few kinks with mod_rewrite
which seems to behave differently all of a sudden. When I can't figure
it out by reading the debug logs, I will most likely post another
question on this mailing list. ;-)
IMO the same rules should yield the same results, no matter the backend.
Anyway, I'm getting off-topic here...
--
regards Helmut K. C. Tessarek KeyID 0xF7832007C11F128D
Key fingerprint = 28A3 1666 4FE8 D72C CFD5 8B23 F783 2007 C11F 128D
/*
Thou shalt not follow the NULL pointer for chaos and madness
await thee at its end.
*/
Re: mod_proxy_fcgi and flush
Posted by Jacob Champion <ch...@gmail.com>.
On 07/18/2017 01:38 AM, Luca Toscano wrote:
> I didn't do it on purpose because I didn't want to mess with the
> main apr_poll, but if you find an elegant solution I am all for it.
It'll take some work, but I think it's doable.
> My understanding is that in this case we don't need to listen to
> POLLOUT events since we are focusing on flushing (giving priority to
> it), and then we'll pay attention to POLLOUT when the main apr_poll
> will happen.
There's nothing semantically wrong with doing that, but I think it can
lead to some hard-to-debug performance issues in certain cases.
Especially if outgoing chunks from the application are blocked waiting
on incoming chunks for the client.
There are some more issues that came up while I was playing around with
the architecture:
- Reads from the request body brigade are blocking, which leads to a
common pattern: the backend has written data that is ready to be sent to
the client, but it's not going anywhere because we are waiting for the
client to send something.
- If the backend sends FCGI_END_REQUEST without closing its stdout
stream, we don't seem to correctly end the response with a zero-length
chunk. I have not investigated this fully yet, but maybe it has
something to do with not correctly sending an EOS bucket?
- For scripts that are not running in NPH mode, we're required by the
CGI spec to strip any incoming transfer-codings and present a final
Content-Length to the application. Omitting CONTENT_LENGTH is supposed
to mean that there is no request body. But for incoming chunked
transfers, we just pass the chunks on to the backend without a
CONTENT_LENGTH, which appears to confuse e.g. PHP-FPM. Jim pointed out
that mod_proxy already has the concept of request body spooling, so we
should probably make use of it.
--Jacob
Re: mod_proxy_fcgi and flush
Posted by Luca Toscano <to...@gmail.com>.
Hello Jacob!
2017-07-18 1:25 GMT+02:00 Jacob Champion <ch...@gmail.com>:
> On 07/14/2017 02:29 AM, Luca Toscano wrote:
>
>> http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-
>> force_flush-v3.patch created, I tested it quickly and it seems working
>> fine (still unsure about using r->connection->pool vs conn->pool in
>> palloc). More tests following in the weekend :)
>>
> Some thoughts:
>
Thanks a lot for reviewing :)
> - This will flush each time a POLLIN event is successfully handled, even
> if the event doesn't cause data to be put on the brigade (as is the case
> for STDERR or END_REQUEST records, or for the empty end-of-stream record).
>
This is true, I thought that the "if (rv != APR_SUCCESS)" right before the
new code was enough to remove corner cases but I was wrong. Maybe it is
only a matter of checking that clen is > 0? Will try to check it today.
> - While we're waiting for a POLLIN event in auto-flush mode, we're not
> listening to POLLOUT events. Seems like we could/should merge the apr_poll
> calls into one. I'm playing around with that now.
I didn't do it on purpose because I didn't want to mess with the main
apr_poll, but if you find an elegant solution I am all for it. My
understanding is that in this case we don't need to listen to POLLOUT
events since we are focusing on flushing (giving priority to it), and then
we'll pay attention to POLLOUT when the main apr_poll will happen.
Luca
Re: mod_proxy_fcgi and flush
Posted by Jacob Champion <ch...@gmail.com>.
On 07/14/2017 02:29 AM, Luca Toscano wrote:
> http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-force_flush-v3.patch
> created, I tested it quickly and it seems working fine (still unsure
> about using r->connection->pool vs conn->pool in palloc). More tests
> following in the weekend :)
Some thoughts:
- This will flush each time a POLLIN event is successfully handled, even
if the event doesn't cause data to be put on the brigade (as is the case
for STDERR or END_REQUEST records, or for the empty end-of-stream record).
- While we're waiting for a POLLIN event in auto-flush mode, we're not
listening to POLLOUT events. Seems like we could/should merge the
apr_poll calls into one. I'm playing around with that now.
--Jacob
Re: mod_proxy_fcgi and flush
Posted by Yann Ylavic <yl...@gmail.com>.
On Fri, Jul 14, 2017 at 11:38 AM, Plüm, Rüdiger, Vodafone Group
<ru...@vodafone.com> wrote:>
> Von: Luca Toscano [mailto:toscano.luca@gmail.com] Gesendet:
>>
>> http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-force_flush-v3.patch
LGTM (with r->pool as suggested by Rüdiger below), thanks Luca.
>>
>> created, I tested it quickly and it seems working fine (still
>> unsure about using r->connection->pool vs conn->pool in palloc).
>> More tests following in the weekend :)
>
> r->pool is the correct pool to use.
+1
Re: mod_proxy_fcgi and flush
Posted by Jim Jagielski <ji...@jaguNET.com>.
Agreed. r->pool for the win.
> On Jul 14, 2017, at 5:38 AM, Plüm, Rüdiger, Vodafone Group <ru...@vodafone.com> wrote:
>
>
>
> Von: Luca Toscano [mailto:toscano.luca@gmail.com]
> Gesendet: Freitag, 14. Juli 2017 11:30
> An: Apache HTTP Server Development List <de...@httpd.apache.org>
> Betreff: Re: mod_proxy_fcgi and flush
>
> Hi Yann,
>
> 2017-07-13 22:15 GMT+02:00 Yann Ylavic <yl...@gmail.com>:
> On Thu, Jul 13, 2017 at 7:36 PM, Luca Toscano <to...@gmail.com> wrote:
> >
> > 1) read the FCGI headers to gather how much data the record is carrying
> > (clen)
> > 2) read the data in iobuf_size batches, sending each time the bytes
> > collected to the output filter chain.
> > 3) finally read the padding bytes (if any).
> >
> > I tried to use the same trick as mod_ajp for flushpackets=auto, but apr_poll
> > returned immediately most of the times due to the padding bytes ready to
> > read (took me a while to realize this simple thing).
>
> I forgot some details about proxy_fcgi so am asking before (re)looking at it :p
>
> Can't we poll() after 3) ?
> If so, maybe we could flush only if a small/zero timeout poll() times
> out, and hence still be able to coalesce multiple (closed) records.
>
> http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-force_flush-v3.patch created, I tested it quickly and it seems working fine (still unsure about using r->connection->pool vs conn->pool in palloc). More tests following in the weekend :)
>
> r->pool is the correct pool to use.
>
> Regards
>
> Rüdiger
>
> Thanks for the hint!
>
> Luca
AW: mod_proxy_fcgi and flush
Posted by Plüm,
Rüdiger,
Vodafone Group <ru...@vodafone.com>.
Von: Luca Toscano [mailto:toscano.luca@gmail.com]
Gesendet: Freitag, 14. Juli 2017 11:30
An: Apache HTTP Server Development List <de...@httpd.apache.org>
Betreff: Re: mod_proxy_fcgi and flush
Hi Yann,
2017-07-13 22:15 GMT+02:00 Yann Ylavic <yl...@gmail.com>>:
On Thu, Jul 13, 2017 at 7:36 PM, Luca Toscano <to...@gmail.com>> wrote:
>
> 1) read the FCGI headers to gather how much data the record is carrying
> (clen)
> 2) read the data in iobuf_size batches, sending each time the bytes
> collected to the output filter chain.
> 3) finally read the padding bytes (if any).
>
> I tried to use the same trick as mod_ajp for flushpackets=auto, but apr_poll
> returned immediately most of the times due to the padding bytes ready to
> read (took me a while to realize this simple thing).
I forgot some details about proxy_fcgi so am asking before (re)looking at it :p
Can't we poll() after 3) ?
If so, maybe we could flush only if a small/zero timeout poll() times
out, and hence still be able to coalesce multiple (closed) records.
http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-force_flush-v3.patch created, I tested it quickly and it seems working fine (still unsure about using r->connection->pool vs conn->pool in palloc). More tests following in the weekend :)
r->pool is the correct pool to use.
Regards
Rüdiger
Thanks for the hint!
Luca
Re: mod_proxy_fcgi and flush
Posted by Luca Toscano <to...@gmail.com>.
Hi Yann,
2017-07-13 22:15 GMT+02:00 Yann Ylavic <yl...@gmail.com>:
> On Thu, Jul 13, 2017 at 7:36 PM, Luca Toscano <to...@gmail.com>
> wrote:
> >
> > 1) read the FCGI headers to gather how much data the record is carrying
> > (clen)
> > 2) read the data in iobuf_size batches, sending each time the bytes
> > collected to the output filter chain.
> > 3) finally read the padding bytes (if any).
> >
> > I tried to use the same trick as mod_ajp for flushpackets=auto, but
> apr_poll
> > returned immediately most of the times due to the padding bytes ready to
> > read (took me a while to realize this simple thing).
>
> I forgot some details about proxy_fcgi so am asking before (re)looking at
> it :p
>
> Can't we poll() after 3) ?
> If so, maybe we could flush only if a small/zero timeout poll() times
> out, and hence still be able to coalesce multiple (closed) records.
>
http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-force_flush-v3.patch
created, I tested it quickly and it seems working fine (still unsure about
using r->connection->pool vs conn->pool in palloc). More tests following in
the weekend :)
Thanks for the hint!
Luca
Re: mod_proxy_fcgi and flush
Posted by Yann Ylavic <yl...@gmail.com>.
On Thu, Jul 13, 2017 at 7:36 PM, Luca Toscano <to...@gmail.com> wrote:
>
> 1) read the FCGI headers to gather how much data the record is carrying
> (clen)
> 2) read the data in iobuf_size batches, sending each time the bytes
> collected to the output filter chain.
> 3) finally read the padding bytes (if any).
>
> I tried to use the same trick as mod_ajp for flushpackets=auto, but apr_poll
> returned immediately most of the times due to the padding bytes ready to
> read (took me a while to realize this simple thing).
I forgot some details about proxy_fcgi so am asking before (re)looking at it :p
Can't we poll() after 3) ?
If so, maybe we could flush only if a small/zero timeout poll() times
out, and hence still be able to coalesce multiple (closed) records.
Regards,
Yann.
Re: mod_proxy_fcgi and flush
Posted by Luca Toscano <to...@gmail.com>.
2017-07-10 11:42 GMT+02:00 Luca Toscano <to...@gmail.com>:
> Hi Rüdiger,
>
> 2017-07-10 8:31 GMT+02:00 Plüm, Rüdiger, Vodafone Group <
> ruediger.pluem@vodafone.com>:
>
>>
>>
>>
>>
>> *Von:* Luca Toscano [mailto:toscano.luca@gmail.com]
>> *Gesendet:* Samstag, 8. Juli 2017 09:52
>> *An:* Apache HTTP Server Development List <de...@httpd.apache.org>
>> *Betreff:* Re: mod_proxy_fcgi and flush
>>
>>
>>
>> Hi Jacob, Helmut!
>>
>>
>>
>> 2017-07-06 20:54 GMT+02:00 Jacob Champion <ch...@gmail.com>:
>>
>> On 07/06/2017 11:13 AM, Jim Jagielski wrote:
>>
>> works 4 me...
>>
>>
>> Doesn't for me. E.g. with a script like
>>
>> <?php
>> print("hi!\n")
>> flush();
>> sleep(1);
>> print("hi!\n");
>> ?>
>>
>> it takes 1 second to receive a single chunk with both lines in it.
>>
>> From a quick skim I assume this is because we don't use nonblocking
>> sockets in the proxy implementation. (There's even a note in mod_proxy_fcgi
>> that says, "Yes it sucks to [get the actual data] in a second recv call,
>> this will eventually change when we move to real nonblocking recv calls.")
>>
>>
>>
>> Quick check from my side too, so not sure if it makes sense. IIUC the
>> comment is about getting all the headers from the FCGI backend
>> (get_data_full(..., AP_FCGI_HEADER_LEN)), then get the response body in
>> another one (the [actual data]).
>>
>>
>>
>> I checked mod_fcgi as Helmut suggested and it seems to me that the -flush
>> feature is a simple "flush every data that you receive", so I tested the
>> following patch with Jacob's php example code and it seems doing what
>> Helmut asked (please correct me if I am wrong).
>>
>>
>>
>> Caveat: I had to set output_buffering = Off in my php-fpm's php.ini
>> config file.
>>
>>
>>
>> http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-
>> force_flush.patch
>>
>>
>>
>> Flushing unconditionally is a bad idea performance wise. Please have a
>> look how ajp solved this issue:
>>
>>
>>
>> https://svn.apache.org/r327185
>>
>> https://svn.apache.org/r384580
>>
>> https://svn.apache.org/r390210
>>
>> https://svn.apache.org/viewvc/httpd/httpd/trunk/modules/prox
>> y/mod_proxy_ajp.c?r1=325879&r2=390210
>>
>>
>>
>> Hint: The above diff contains further unrelated changes.
>>
>>
>>
> Definitely, I added that quick and dirty patch to show my idea, really far
> from being ready for a commit :)
> Will review the commits that you posted, thanks!
>
With a bit of delay I have something to share to the list :)
I kept the following test as the "use case":
<?php
print("hi!\n")
flush();
sleep(1);
print("hi!\n");
?>
First of all, I tested it with httpd-trunk and it seems working fine thanks
to several core non-blocking writes, removing the need of explicit FLUSH
buckets. I am very ignorant about this part of httpd so I can't say much
more, but it is really nice to see!
I then checked Rüdiger's commits and this is my understanding about mod_ajp:
- if "flushpackets" is set to "on" by mod_proxy then mod_proxy_ajp will
insert a FLUSH bucket as soon as data is ready to be passed to the output
filter chain.
- if "flushpackets" is set to "auto" then mod_proxy_ajp will be a bit
smarter and will apr_poll for "flushwait" seconds before inserting any
FLUSH bucket, since new data might come from the backend and it would be
more efficient to force a FLUSH with more data.
IIUC mod_proxy_fcgi leverages the fact that the backend sends FCGI records
with a predefined structure (FCGI_Record), containing "FCGI headers"
(content len, padding len, version, type, etc..) that can be used to know
how much data to expect from the backend. As far as I can see (with
php-fpm) our use case is handled with multiple FCGI records, one for each
"flush()" done in the php script. mod_proxy_fcgi does the following for
each of them:
1) read the FCGI headers to gather how much data the record is carrying
(clen)
2) read the data in iobuf_size batches, sending each time the bytes
collected to the output filter chain.
3) finally read the padding bytes (if any).
I tried to use the same trick as mod_ajp for flushpackets=auto, but
apr_poll returned immediately most of the times due to the padding bytes
ready to read (took me a while to realize this simple thing).
I didn't find a real use case for flushpackets=auto in mod_proxy_fcgi, but
I have two proposals for fluspackets=on:
http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-force_flush.patch
http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-force_flush-v2.patch
The second one is a bit more conservative and does not insert the FLUSH
bucket if it knows that the content len (clen) is bigger than the
iobuf_size (since there is a goto that forces another get_data() in this
case).
Let me know if what I wrote make sense, suggestions and comments are
welcome. It would be interesting to also know from Helmut a simple (PHP?)
use case to use as a test, so I'll not base my code only on Jacob's example
(that was really useful though!!).
Thanks!
Luca
Re: mod_proxy_fcgi and flush
Posted by Luca Toscano <to...@gmail.com>.
Hi Rüdiger,
2017-07-10 8:31 GMT+02:00 Plüm, Rüdiger, Vodafone Group <
ruediger.pluem@vodafone.com>:
>
>
>
>
> *Von:* Luca Toscano [mailto:toscano.luca@gmail.com]
> *Gesendet:* Samstag, 8. Juli 2017 09:52
> *An:* Apache HTTP Server Development List <de...@httpd.apache.org>
> *Betreff:* Re: mod_proxy_fcgi and flush
>
>
>
> Hi Jacob, Helmut!
>
>
>
> 2017-07-06 20:54 GMT+02:00 Jacob Champion <ch...@gmail.com>:
>
> On 07/06/2017 11:13 AM, Jim Jagielski wrote:
>
> works 4 me...
>
>
> Doesn't for me. E.g. with a script like
>
> <?php
> print("hi!\n")
> flush();
> sleep(1);
> print("hi!\n");
> ?>
>
> it takes 1 second to receive a single chunk with both lines in it.
>
> From a quick skim I assume this is because we don't use nonblocking
> sockets in the proxy implementation. (There's even a note in mod_proxy_fcgi
> that says, "Yes it sucks to [get the actual data] in a second recv call,
> this will eventually change when we move to real nonblocking recv calls.")
>
>
>
> Quick check from my side too, so not sure if it makes sense. IIUC the
> comment is about getting all the headers from the FCGI backend
> (get_data_full(..., AP_FCGI_HEADER_LEN)), then get the response body in
> another one (the [actual data]).
>
>
>
> I checked mod_fcgi as Helmut suggested and it seems to me that the -flush
> feature is a simple "flush every data that you receive", so I tested the
> following patch with Jacob's php example code and it seems doing what
> Helmut asked (please correct me if I am wrong).
>
>
>
> Caveat: I had to set output_buffering = Off in my php-fpm's php.ini config
> file.
>
>
>
> http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_
> fcgi-force_flush.patch
>
>
>
> Flushing unconditionally is a bad idea performance wise. Please have a
> look how ajp solved this issue:
>
>
>
> https://svn.apache.org/r327185
>
> https://svn.apache.org/r384580
>
> https://svn.apache.org/r390210
>
> https://svn.apache.org/viewvc/httpd/httpd/trunk/modules/
> proxy/mod_proxy_ajp.c?r1=325879&r2=390210
>
>
>
> Hint: The above diff contains further unrelated changes.
>
>
>
Definitely, I added that quick and dirty patch to show my idea, really far
from being ready for a commit :)
Will review the commits that you posted, thanks!
Luca
AW: mod_proxy_fcgi and flush
Posted by Plüm,
Rüdiger,
Vodafone Group <ru...@vodafone.com>.
Von: Luca Toscano [mailto:toscano.luca@gmail.com]
Gesendet: Samstag, 8. Juli 2017 09:52
An: Apache HTTP Server Development List <de...@httpd.apache.org>
Betreff: Re: mod_proxy_fcgi and flush
Hi Jacob, Helmut!
2017-07-06 20:54 GMT+02:00 Jacob Champion <ch...@gmail.com>>:
On 07/06/2017 11:13 AM, Jim Jagielski wrote:
works 4 me...
Doesn't for me. E.g. with a script like
<?php
print("hi!\n")
flush();
sleep(1);
print("hi!\n");
?>
it takes 1 second to receive a single chunk with both lines in it.
From a quick skim I assume this is because we don't use nonblocking sockets in the proxy implementation. (There's even a note in mod_proxy_fcgi that says, "Yes it sucks to [get the actual data] in a second recv call, this will eventually change when we move to real nonblocking recv calls.")
Quick check from my side too, so not sure if it makes sense. IIUC the comment is about getting all the headers from the FCGI backend (get_data_full(..., AP_FCGI_HEADER_LEN)), then get the response body in another one (the [actual data]).
I checked mod_fcgi as Helmut suggested and it seems to me that the -flush feature is a simple "flush every data that you receive", so I tested the following patch with Jacob's php example code and it seems doing what Helmut asked (please correct me if I am wrong).
Caveat: I had to set output_buffering = Off in my php-fpm's php.ini config file.
http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-force_flush.patch
Flushing unconditionally is a bad idea performance wise. Please have a look how ajp solved this issue:
https://svn.apache.org/r327185
https://svn.apache.org/r384580
https://svn.apache.org/r390210
https://svn.apache.org/viewvc/httpd/httpd/trunk/modules/proxy/mod_proxy_ajp.c?r1=325879&r2=390210
Hint: The above diff contains further unrelated changes.
Regards
Rüdiger
Re: mod_proxy_fcgi and flush
Posted by Luca Toscano <to...@gmail.com>.
Hi Jacob, Helmut!
2017-07-06 20:54 GMT+02:00 Jacob Champion <ch...@gmail.com>:
> On 07/06/2017 11:13 AM, Jim Jagielski wrote:
>
>> works 4 me...
>>
>
> Doesn't for me. E.g. with a script like
>
> <?php
> print("hi!\n")
> flush();
> sleep(1);
> print("hi!\n");
> ?>
>
> it takes 1 second to receive a single chunk with both lines in it.
>
> From a quick skim I assume this is because we don't use nonblocking
> sockets in the proxy implementation. (There's even a note in mod_proxy_fcgi
> that says, "Yes it sucks to [get the actual data] in a second recv call,
> this will eventually change when we move to real nonblocking recv calls.")
Quick check from my side too, so not sure if it makes sense. IIUC the
comment is about getting all the headers from the FCGI backend
(get_data_full(..., AP_FCGI_HEADER_LEN)), then get the response body in
another one (the [actual data]).
I checked mod_fcgi as Helmut suggested and it seems to me that the -flush
feature is a simple "flush every data that you receive", so I tested the
following patch with Jacob's php example code and it seems doing what
Helmut asked (please correct me if I am wrong).
Caveat: I had to set output_buffering = Off in my php-fpm's php.ini config
file.
http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-force_flush.patch
I am still not sure if the get_data function (responsible to read the body
after the headers in dispatch()) works in this way only my local test or
not, but it seems simply calling apr_socket_recv with a buffer len and
return whatever data it comes from the backend. IIUC apr_socket_recv should
return data even if less than the buffer specified..
So if what I wrote vaguely makes sense, we might think about adding a new
directive that enables explicit flushing to mod_proxy_fcgi, so admins can
twist its behavior if needed?
> On 07/06/2017 11:08 AM, Helmut K. C. Tessarek wrote:
> > What is your take on this?
>
> If I'm honest, my brutally blunt take on it is "stop using HTTP to try to
> emulate push notifications within a single response; pretty much everything
> in the ecosystem is actively working against you at this point; responses
> are designed to be cacheable and deliverable as a unit, not as multiple
> pieces; and we've had *real* solutions like WebSocket for five years now
> rabble rabble rabble." But I don't actually think that's going to be the
> accepted answer.
>
> It probably makes sense to work on a nonblocking architecture for proxied
> responses in general.
>
> -Jacob
>
Completely agree with Jacob, this use case might not be the best one for
HTTP :)
Luca
Re: mod_proxy_fcgi and flush
Posted by "Helmut K. C. Tessarek" <te...@evermeet.cx>.
On 2017-07-07 17:50, Jacob Champion wrote:
> I'm probably not the person you're looking for; I don't know the
> mod_proxy code very well. You may find that my root cause analysis is
> completely incorrect. But, if you'd like to bounce ideas off me, go
> for it. (As Luca already knows, I can be a little sporadic when it
> comes to this sort of thing. Fair warning! :) )
I'm gonna have a look at the mod_proxy and mod_proxy_fcgi code. Who
wrote the code in the first place? Maybe I can bug that person with my
questions. ;-)
Let's see, maybe I'm totally out of my depth anyway. Sometimes it's very
hard to chime in, when you haven't been there from the beginning.
> #httpd-dev on Freenode is also a place -- very quiet, usually, but I
> try to be there when I can.
Yea, not a fan of irc. I always find it very frustrating. Asking
something, waiting forever, not getting any answers. I'm more into IM.
I've noticed that people on IM are more responsive, since they really
seem to be online when the status says they are online.
Anyway, I don't think that real time chat is necessary, at least not
right away.
--
regards Helmut K. C. Tessarek
lookup https://sks-keyservers.net/i for KeyID 0xC11F128D
/*
Thou shalt not follow the NULL pointer for chaos and madness
await thee at its end.
*/
Re: mod_proxy_fcgi and flush
Posted by Jacob Champion <ch...@gmail.com>.
On 07/06/2017 08:25 PM, Helmut K. C. Tessarek wrote:
> On 2017-07-06 14:54, Jacob Champion wrote:
>> It probably makes sense to work on a nonblocking architecture for
>> proxied responses in general.
>
> I'm not familiar with that particular code, but would be interested in
> looking into it. Does anybody volunteer as a mentor?
I'm probably not the person you're looking for; I don't know the
mod_proxy code very well. You may find that my root cause analysis is
completely incorrect. But, if you'd like to bounce ideas off me, go for
it. (As Luca already knows, I can be a little sporadic when it comes to
this sort of thing. Fair warning! :) )
#httpd-dev on Freenode is also a place -- very quiet, usually, but I try
to be there when I can.
--Jacob
Re: mod_proxy_fcgi and flush
Posted by "Helmut K. C. Tessarek" <te...@evermeet.cx>.
On 2017-07-06 14:54, Jacob Champion wrote:
> If I'm honest, my brutally blunt take on it is "stop using HTTP to try
> to emulate push notifications within a single response; pretty much
> everything in the ecosystem is actively working against you at this
> point; responses are designed to be cacheable and deliverable as a unit,
> not as multiple pieces; and we've had *real* solutions like WebSocket
> for five years now rabble rabble rabble." But I don't actually think
> that's going to be the accepted answer.
In a perfect world, I'm right there with you, but we've seen (as long as
technology has existed) that people twist the way how technology is
applied and used. It's hard to convince those people otherwise,
especially when most of the time it has been possible to get it to work
- somehow.
> It probably makes sense to work on a nonblocking architecture for
> proxied responses in general.
I'm not familiar with that particular code, but would be interested in
looking into it. Does anybody volunteer as a mentor?
Cheers,
K. C.
--
regards Helmut K. C. Tessarek
lookup https://sks-keyservers.net/i for KeyID 0xC11F128D
/*
Thou shalt not follow the NULL pointer for chaos and madness
await thee at its end.
*/
Re: mod_proxy_fcgi and flush
Posted by Jacob Champion <ch...@gmail.com>.
On 07/06/2017 11:13 AM, Jim Jagielski wrote:
> works 4 me...
Doesn't for me. E.g. with a script like
<?php
print("hi!\n")
flush();
sleep(1);
print("hi!\n");
?>
it takes 1 second to receive a single chunk with both lines in it.
From a quick skim I assume this is because we don't use nonblocking
sockets in the proxy implementation. (There's even a note in
mod_proxy_fcgi that says, "Yes it sucks to [get the actual data] in a
second recv call, this will eventually change when we move to real
nonblocking recv calls.")
On 07/06/2017 11:08 AM, Helmut K. C. Tessarek wrote:
> What is your take on this?
If I'm honest, my brutally blunt take on it is "stop using HTTP to try
to emulate push notifications within a single response; pretty much
everything in the ecosystem is actively working against you at this
point; responses are designed to be cacheable and deliverable as a unit,
not as multiple pieces; and we've had *real* solutions like WebSocket
for five years now rabble rabble rabble." But I don't actually think
that's going to be the accepted answer.
It probably makes sense to work on a nonblocking architecture for
proxied responses in general.
-Jacob
Re: mod_proxy_fcgi and flush
Posted by Jim Jagielski <ji...@jaguNET.com>.
works 4 me... :/
> On Jul 6, 2017, at 2:08 PM, Helmut K. C. Tessarek <te...@evermeet.cx> wrote:
>
> One of the comments on the documentation page of mod_proxy_fcgi
> (http://httpd.apache.org/docs/2.4/mod/mod_proxy_fcgi.html) mentions an
> issue with flush:
>
> There is just no flush support it seems. I attempt to use PHP flush()
> and it won't work until you fill up a buffer first, rendering Server
> Sent Events impossible with proxy_fcgi. It worked very well with
> "-flush" with mod_fastcgi FastCgiExternalServer.
>
> If this is really the case, all web applications that use push
> notifications won't be working with mod_proxy_fcgi.
>
> The wiki page on https://wiki.apache.org/httpd/php also suggests to use
> mod_proxy_fcgi, but if above is true, this might lead to problems.
>
> What is your take on this?
>
> --
> regards Helmut K. C. Tessarek
> lookup https://sks-keyservers.net/i for KeyID 0xC11F128D
>
> /*
> Thou shalt not follow the NULL pointer for chaos and madness
> await thee at its end.
> */
Re: mod_proxy_fcgi and flush
Posted by Jacob Champion <ch...@gmail.com>.
On Jul 8, 2017 6:14 PM, "Nick Kew" <ni...@apache.org> wrote:
> > > It probably makes sense to work on a nonblocking architecture for
> > > proxied responses in general.
Is that really the issue in the first place? We have a concept of
flushing, and can implement more finely-tuned throughput than merely
blocking vs non-blocking. The point at issue is for PHP to communicate
a flush with proxy_fcgi, and for proxy_fcgi then to honour it.
It seems one or both of those things isn't happening.
Not sure yet, but based on Luca's note, I think there's a good chance that
my skim of the code gave me the wrong idea. If FPM sends a full message in
a flush (and I can't see why it wouldn't) this might be fixable with a
single FPM option.
--Jacob
Re: mod_proxy_fcgi and flush
Posted by Nick Kew <ni...@apache.org>.
On Thu, 2017-07-06 at 14:08 -0400, Helmut K. C. Tessarek wrote:
> One of the comments on the documentation page of mod_proxy_fcgi
> (http://httpd.apache.org/docs/2.4/mod/mod_proxy_fcgi.html) mentions an
> issue with flush:
>
> There is just no flush support it seems. I attempt to use PHP flush()
> and it won't work until you fill up a buffer first, rendering Server
> Sent Events impossible with proxy_fcgi.
Do we have any idea what (if anything) proxy_fcgi receives from a
PHP flush()? Your fix could be as simple as propagating an event.
> > In a perfect world, I'm right there with you, but we've seen (as long as
> > technology has existed) that people twist the way how technology is
> > applied and used. It's hard to convince those people otherwise,
> > especially when most of the time it has been possible to get it to work
That takes me back ...
https://www.theregister.co.uk/2007/08/24/everything_over_http/
> > > It probably makes sense to work on a nonblocking architecture for
> > > proxied responses in general.
Is that really the issue in the first place? We have a concept of
flushing, and can implement more finely-tuned throughput than merely
blocking vs non-blocking. The point at issue is for PHP to communicate
a flush with proxy_fcgi, and for proxy_fcgi then to honour it.
It seems one or both of those things isn't happening.
> > I'm not familiar with that particular code, but would be interested in
> > looking into it. Does anybody volunteer as a mentor?
Best mentor is this list. Or #httpd-dev on Freenode (IRC),
if someone's paying attention there.
> lookup https://sks-keyservers.net/i for KeyID 0xC11F128D
Please update that: it's too easy to spoof.
See https://evil32.com/ , or my blog article at
https://bahumbug.wordpress.com/2017/04/27/pretty-good-phishing/
--
Nick Kew