You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Ruediger Pluem <rp...@apache.org> on 2006/01/02 22:18:19 UTC
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
On 12/20/2005 10:14 PM, Ruediger Pluem wrote:
[..cut..]
> But you pointed me to an interesting thing: If the main response is
> T-E chunked and the backend error happened during the subrequest, the
> chunked filter may sometimes add the last-chunk marker (if the brigade
> containing the error bucket does *not* contain and eos bucket) and
> sometimes not (if the brigade containing the error bucket does contain
> an eos bucket).
> In the case that the broken backend happend on the main request the brigade
> always contains both buckets as they both get added to the brigade. But in
> the subrequest case I guess the eos bucket (of the subrequest) gets removed
> and the main request adds its own one later and maybe to a different brigade
> then the error bucket.
Meanwhile I took some time to investigate this further. I checked with the
subrequests caused by mod_include's #include virtual tag. In this case the
error bucket and the eos bucket seems to get passed to the chunk filter on
different brigades. This means the chunk filter does sent the last chunk marker
in this case. OTOH the error bucket causes the keepalive connection on the
main request to be closed and the partially generated page of mod_include gets
cached.
Before I take any further actions I would like to discuss the desired behaviour
in the subrequest case:
1. Proposal
If a subrequest has a broken backend also set r->no_cache for the main request
and ensure that the chunk filter does not sent the last chunk marker in this case.
2. Proposal
If a subrequest has a broken backend do not sent the error bucket. Only set r->no_cache
to ensure that this subrequest response does not get cached.
Further proposals are welcome.
Furthermore I am wondering if the current behaviour of mod_include is correct.
Shouldn't we prevent caching if any of the mod_include operations got wrong?
The broken backend for the #include virtual tag is only one example for this.
And if we decide that we should not cache in this situation, how do we let external
caches know? Should we also do not sent the last chunk marker in this case?
> I am even getting unsure if the brigade always contains error and eos bucket
> in the main request case, as there might happen an brigade split on the way
> to the chunk filter. Does anybody know if this can happen?
>
Anybody found an answer to this question? If this is not sure it may be a good
idea to memorize the fact that the error bucket was seen in the context of
the chunk filter.
Regards
Rüdiger
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
On 1/3/06, Ruediger Pluem <rp...@apache.org> wrote:
> >>2. Proposal
> >>If a subrequest has a broken backend do not sent the error bucket. Only
> >>set r->no_cache to ensure that this subrequest response does not get
> >>cached.
> >
> >
> > I think we still need to ensure that an error bucket is sent too, right?
> > Otherwise, the connection will be reused - what am I missing? -- justin
>
> No, you are not missing anything. The question to me was: Do we need to close
> a keepalive on the main request just because a subrequest failed in the middle
> of the response?
> Or to be more precise: Should the behaviour to cut off the keepalive be the default
> behaviour in such cases with the chance for subrequest creators to remove the error
> bucket and to make the response cacheable again or should it be the other way round
> that the subrequest creator is reponsible for preventing caching and closing the
> keepalive by sending the error bucket by himself if he thinks that this is needed?
> While writing this I personally come to the conclusion that the 1. proposal
> (sending the error bucket) is saver as a default behaviour.
Oh, I didn't realize you intended it as an either/or scenario. Then,
yes, I agree that #1 is correct. =) -- justin
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by Ruediger Pluem <rp...@apache.org>.
On 01/03/2006 03:52 AM, Justin Erenkrantz wrote:
> On Mon, Jan 02, 2006 at 10:18:19PM +0100, Ruediger Pluem wrote:
>
[..cut..]
>
>>2. Proposal
>>If a subrequest has a broken backend do not sent the error bucket. Only
>>set r->no_cache to ensure that this subrequest response does not get
>>cached.
>
>
> I think we still need to ensure that an error bucket is sent too, right?
> Otherwise, the connection will be reused - what am I missing? -- justin
No, you are not missing anything. The question to me was: Do we need to close
a keepalive on the main request just because a subrequest failed in the middle
of the response?
Or to be more precise: Should the behaviour to cut off the keepalive be the default
behaviour in such cases with the chance for subrequest creators to remove the error
bucket and to make the response cacheable again or should it be the other way round
that the subrequest creator is reponsible for preventing caching and closing the
keepalive by sending the error bucket by himself if he thinks that this is needed?
While writing this I personally come to the conclusion that the 1. proposal
(sending the error bucket) is saver as a default behaviour.
Regards
Rüdiger
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
On Mon, Jan 02, 2006 at 10:18:19PM +0100, Ruediger Pluem wrote:
> Before I take any further actions I would like to discuss the desired behaviour
> in the subrequest case:
>
> 1. Proposal
> If a subrequest has a broken backend also set r->no_cache for the main
> request and ensure that the chunk filter does not sent the last chunk
> marker in this case.
+1.
> 2. Proposal
> If a subrequest has a broken backend do not sent the error bucket. Only
> set r->no_cache to ensure that this subrequest response does not get
> cached.
I think we still need to ensure that an error bucket is sent too, right?
Otherwise, the connection will be reused - what am I missing? -- justin
Re: mod_proxy vs. serverpush
Posted by Ruediger Pluem <rp...@apache.org>.
On 01/05/2006 11:27 PM, Graham Leggett wrote:
> Ruediger Pluem wrote:
[..cut..]
> Looking deeper into this, if the above was never true, then the loop
> would spin resulting in 100% processor usage for this process/thread
> while the download was running.
>
> Are you 100% sure this is never called?
Meanwhile I found the reason for this behaviour. We are never doing non blocking reads:
#0 socket_bucket_read (a=0x8196748, str=0x40ad052c, len=0x40ad0530, block=APR_BLOCK_READ) at
buckets/apr_buckets_socket.c:22
#1 0x40052463 in apr_brigade_split_line (bbOut=0x819c3a8, bbIn=0x819c288, block=APR_BLOCK_READ, maxbytes=8192) at
buckets/apr_brigade.c:292
#2 0x080721d7 in ap_core_input_filter (f=0x8194cb0, b=0x819c3a8, mode=AP_MODE_GETLINE, block=APR_BLOCK_READ, readbytes=0)
at core_filters.c:155
#3 0x0807c119 in ap_get_brigade (next=0x8194cb0, bb=0x819c3a8, mode=AP_MODE_GETLINE, block=APR_BLOCK_READ,
readbytes=583581946604225836)
at util_filter.c:489
#4 0x404dbb47 in logio_in_filter (f=0x8194c88, bb=0x819c3a8, mode=AP_MODE_GETLINE, block=APR_BLOCK_READ, readbytes=0)
at mod_logio.c:115
#5 0x0807c119 in ap_get_brigade (next=0x8194c88, bb=0x819c3a8, mode=AP_MODE_GETLINE, block=APR_BLOCK_READ,
readbytes=583581774805533996)
at util_filter.c:489
#6 0x0807f27c in ap_http_filter (f=0x819c330, b=0x8195058, mode=AP_MODE_READBYTES, block=APR_NONBLOCK_READ, readbytes=8192)
at http_filters.c:292
#7 0x0807c119 in ap_get_brigade (next=0x819c330, bb=0x8195058, mode=AP_MODE_READBYTES, block=APR_NONBLOCK_READ,
readbytes=583712238732117292) at util_filter.c:489
#8 0x405bae91 in ap_proxy_http_process_response (p=0x8193f38, r=0x8197f80, backend=0x818d908, origin=0x8194790,
conf=0x818c9b0,
server_portstr=0x40ad282c "") at mod_proxy_http.c:1463
#9 0x405bb9d2 in proxy_http_handler (r=0x8197f80, worker=0x818b808, conf=0x818c9b0, url=0x8194780 "/test/long.jsp",
proxyname=0x0,
proxyport=0) at mod_proxy_http.c:1732
#10 0x405a4063 in proxy_run_scheme_handler (r=0x8197f80, worker=0x818b808, conf=0x818c9b0,
url=0x8199730 "http://127.0.0.1:8080/test/long.jsp", proxyhost=0x0, proxyport=0) at mod_proxy.c:1941
ap_http_filter changes the mode from APR_NONBLOCK_READ to APR_BLOCK_READ.
So I think we must check if we can adjust ap_http_filter. I guess this is not an easy task.
Maybe this changes once Brian makes further progress on its async-read branch.
Regards
Rüdiger
Re: mod_proxy vs. serverpush
Posted by Graham Leggett <mi...@sharp.fm>.
Ruediger Pluem wrote:
>>> Anyway, it does not work as expected, as it seems that the condition
>>> (APR_STATUS_IS_EAGAIN(rv)
>>> || (rv == APR_SUCCESS &&
>>> APR_BRIGADE_EMPTY(bb))) {
>>> never gets true.
Looking deeper into this, if the above was never true, then the loop
would spin resulting in 100% processor usage for this process/thread
while the download was running.
Are you 100% sure this is never called?
Regards,
Graham
--
Re: mod_proxy vs. serverpush
Posted by Ruediger Pluem <rp...@apache.org>.
On 01/05/2006 10:25 PM, Graham Leggett wrote:
> Ruediger Pluem wrote:
>
[..cut..]
>> Anyway, it does not work as expected, as it seems that the condition
>> (APR_STATUS_IS_EAGAIN(rv)
>> || (rv == APR_SUCCESS &&
>> APR_BRIGADE_EMPTY(bb))) {
>> never gets true.
>
>
> I think this if statement covers the case where a non blocking read is
> attempted, and zero bytes are returned, in which case another non
> blocking read might also return zero bytes, causing the loop to spin at
> 100% processor usage.
>
> The problem lies in the code further down:
>
> /* try send what we read */
> if (ap_pass_brigade(r->output_filters, bb) !=
> APR_SUCCESS
> || c->aborted) {
> /* Ack! Phbtt! Die! User aborted! */
> backend->close = 1; /* this causes socket close
> below *
> /
> finish = TRUE;
> }
No, I do not think so from the original idea. From my point of view the typical situation
on the backend that wants to flush some data is:
1. It sends some data.
2. It does no send any more data for a while.
If the condition would work as I expect, then all data from the backend is read (maybe in
several loop iterations and thus passed down the filter chain already by the code you mention above).
Then the backend would stop sending data for some time causing the condition to become true.
This would cause the code to add a flush bucket and pass the brigade down the filter chain.
>
> Without explicitly adding flush buckets to the output filter stack, the
> output filter stack seems to buffer before sending (rational behaviour).
>
> To change this, we would need to add an output flush bucket after each
> read.
I do not think that this a good idea. It leads to too much traffic overhead. Basicly I like
the idea: Read from the backend as long as data is present (non-blocking). If no more data
is present flush filter chain and switch to blocking mode to wait for further data.
Regards
Rüdiger
Re: mod_proxy vs. serverpush
Posted by Graham Leggett <mi...@sharp.fm>.
Ruediger Pluem wrote:
> The logic in mod_proxy_http.c of 2.2.x already tries to address this issue by
> flushing the data if no more data is available from the backend right now:
>
> apr_read_type_e mode = APR_NONBLOCK_READ;
> int finish = FALSE;
>
> do {
> apr_off_t readbytes;
> apr_status_t rv;
>
> rv = ap_get_brigade(rp->input_filters, bb,
> AP_MODE_READBYTES, mode,
> conf->io_buffer_size);
>
> /* ap_get_brigade will return success with an empty brigade
> * for a non-blocking read which would block: */
> if (APR_STATUS_IS_EAGAIN(rv)
> || (rv == APR_SUCCESS && APR_BRIGADE_EMPTY(bb))) {
> /* flush to the client and switch to blocking mode */
> e = apr_bucket_flush_create(c->bucket_alloc);
> APR_BRIGADE_INSERT_TAIL(bb, e);
> if (ap_pass_brigade(r->output_filters, bb)
> || c->aborted) {
> backend->close = 1;
> break;
> }
> apr_brigade_cleanup(bb);
> mode = APR_BLOCK_READ;
> continue;
> }
> else if (rv == APR_EOF) {
> break;
> }
> else if (rv != APR_SUCCESS) {
> ap_log_cerror(APLOG_MARK, APLOG_ERR, rv, c,
> "proxy: error reading response");
> break;
> }
> /* next time try a non-blocking read */
> mode = APR_NONBLOCK_READ;
>
>
> Anyway, it does not work as expected, as it seems that the condition
> (APR_STATUS_IS_EAGAIN(rv)
> || (rv == APR_SUCCESS && APR_BRIGADE_EMPTY(bb))) {
> never gets true.
I think this if statement covers the case where a non blocking read is
attempted, and zero bytes are returned, in which case another non
blocking read might also return zero bytes, causing the loop to spin at
100% processor usage.
The problem lies in the code further down:
/* try send what we read */
if (ap_pass_brigade(r->output_filters, bb) !=
APR_SUCCESS
|| c->aborted) {
/* Ack! Phbtt! Die! User aborted! */
backend->close = 1; /* this causes socket
close below *
/
finish = TRUE;
}
Without explicitly adding flush buckets to the output filter stack, the
output filter stack seems to buffer before sending (rational behaviour).
To change this, we would need to add an output flush bucket after each read.
Is this a rational thing to do in the general case? Or should the
addition of the flush be configurable?
Regards,
Graham
--
Re: mod_proxy vs. serverpush
Posted by Ruediger Pluem <rp...@apache.org>.
On 01/05/2006 09:04 PM, Graham Leggett wrote:
> Matthias Behrens wrote:
[..cut..]
>
> This is an interesting problem, but definitely worth looking into fixing.
The logic in mod_proxy_http.c of 2.2.x already tries to address this issue by
flushing the data if no more data is available from the backend right now:
apr_read_type_e mode = APR_NONBLOCK_READ;
int finish = FALSE;
do {
apr_off_t readbytes;
apr_status_t rv;
rv = ap_get_brigade(rp->input_filters, bb,
AP_MODE_READBYTES, mode,
conf->io_buffer_size);
/* ap_get_brigade will return success with an empty brigade
* for a non-blocking read which would block: */
if (APR_STATUS_IS_EAGAIN(rv)
|| (rv == APR_SUCCESS && APR_BRIGADE_EMPTY(bb))) {
/* flush to the client and switch to blocking mode */
e = apr_bucket_flush_create(c->bucket_alloc);
APR_BRIGADE_INSERT_TAIL(bb, e);
if (ap_pass_brigade(r->output_filters, bb)
|| c->aborted) {
backend->close = 1;
break;
}
apr_brigade_cleanup(bb);
mode = APR_BLOCK_READ;
continue;
}
else if (rv == APR_EOF) {
break;
}
else if (rv != APR_SUCCESS) {
ap_log_cerror(APLOG_MARK, APLOG_ERR, rv, c,
"proxy: error reading response");
break;
}
/* next time try a non-blocking read */
mode = APR_NONBLOCK_READ;
Anyway, it does not work as expected, as it seems that the condition
(APR_STATUS_IS_EAGAIN(rv)
|| (rv == APR_SUCCESS && APR_BRIGADE_EMPTY(bb))) {
never gets true. I did not have the time to dig in deeper, but maybe you want
to do some search.
BTW: mod_proxy_ajp currently addresses this issue with a bandaid until
the AJP protocol has some sort of flushing command. It is also based on
a check if more data is available from the backend right now.
[..cut..]
Regards
Rüdiger
Re: mod_proxy vs. serverpush
Posted by Graham Leggett <mi...@sharp.fm>.
Matthias Behrens wrote:
> i am running a cgi-serverfarm behind an apache webserver with mod_rewrite / mod_proxy
>
> one cgi is supposed to send a progress bar using serverpush
> everytime the programm completes 5% of its work, it sends further html-code which lets the bar grow
>
> u can all test this by going to
>
> http://www.gulp.de/kb/tools/trend.htm
>
> just type some it-skills in the form and press send. you have to write some unusual combinations or choose older data to prevent the programms cachefunction sending an immediate reply.
>
> as you can see the bar is completed only in very big steps. but actually every imagepart of the bar is sended seperately. some internal cache in the mod_proxy module prevents the results from beeing sended until (i guess) 8192 bytes are reached. i allready added 1024 spaces to every package to make it go a little more smooth but this is unacceptable for people with low bandwith.
This is an interesting problem, but definitely worth looking into fixing.
The 8k buffer is correct, proxy tries to read up to 8k at a time, and
then after receiving a full 8k, it sends that 8k down the filter stack,
over the network to the browser.
I think the root of the problem is that there is no way (or maybe there
is and I don't know how to do it yet) to say to the input filter stack
"give me what you got up to a maximum of 8k". If 10 bytes had arrived,
then 10 bytes would be returned, and the next read would block waiting
for the next piece to arrive.
The downside of this approach is that if a backend server wrote one byte
at a time to the filter stack, then the proxy would write out chunks
containing one byte per chunk, resulting in a large multiplication of
bandwidth. Perhaps making this configurable would be the answer.
Regards,
Graham
--
mod_proxy vs. serverpush
Posted by Matthias Behrens <ma...@gulp.de>.
hi everybody
i am running a cgi-serverfarm behind an apache webserver with mod_rewrite / mod_proxy
one cgi is supposed to send a progress bar using serverpush
everytime the programm completes 5% of its work, it sends further html-code which lets the bar grow
u can all test this by going to
http://www.gulp.de/kb/tools/trend.htm
just type some it-skills in the form and press send. you have to write some unusual combinations or choose older data to prevent the programms cachefunction sending an immediate reply.
as you can see the bar is completed only in very big steps. but actually every imagepart of the bar is sended seperately. some internal cache in the mod_proxy module prevents the results from beeing sended until (i guess) 8192 bytes are reached. i allready added 1024 spaces to every package to make it go a little more smooth but this is unacceptable for people with low bandwith.
can u help me with this cosmetical issue?
thx
matthias
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by Nick Kew <ni...@webthing.com>.
On Thursday 05 January 2006 11:58, Joe Orton wrote:
> On Thu, Jan 05, 2006 at 12:48:25PM +0100, Ruediger Pluem wrote:
> > But I remember myself that there had been casting issues in the past that
> > created compiler warnings especially with gcc 4. The patch below compiles
> > fine with my gcc 3.2.2 with -Wall. So if someone could give a comment
> > if
> >
> > f->ctx = (void *)(1)
> >
> > is fine it would be great.
>
> André's trick of using invented unique pointers by doing:
>
> static char sentinel; (in global scope)
>
> f->ctx = &sentinel;
>
> is neater and avoids the casting mess. (pick a suitable variable name
> though :)
Why go to the trouble of inventing a pointer? You already have unique
pointers in global scope. An obvious example:
module some_module;
f->ctx = &some_module;
--
Nick Kew
Re: mod_dbd and Windows
Posted by Steffen <in...@apachelounge.com>.
mod_dbd together with the apr_dbd framework, is listed in the "new features"
document for 2.2, which rather implies that it should work and is to some
degree supported.
As far as my and other win users experience, I hade to conclude that
mod_dbd/mod_authn_dbd is experimental.
Steffen
----- Original Message -----
From: "Carsten Wiedmann" <ca...@gmx.de>
To: <de...@httpd.apache.org>
Sent: Wednesday, January 11, 2006 11:42 PM
Subject: Re: mod_dbd and Windows
> Nick Kew schrieb:
>
>> On Friday 06 January 2006 19:20, Steffen wrote:
>> > Several ppl tried it, but it crashes Apache or it looses connections,
>> > see: http://www.apachelounge.com/forum/viewtopic.php?t=52
>> >
>> > To get it compiled, as i told you before, mysql.h has to be changed,
>> > see: http://www.apachelounge.com/forum/viewtopic.php?p=280#280
>>
>> Looking at your URL, you've changed __WIN__ to __WIN32__ in mysql.h.
>> That seems to be something internal to MySQL itself, outside the
>> reach of the apr_dbd_mysql driver.
>
> I've also tried to use mod_dbd and MySQL on Windows with no luck. I don't
> must change "mysql.h" to compile the modules. Instead I must add this line
> in "apr_dbd_mysql.c":
> | #include <mysql/my_global.h>
> before:
> | #include <mysql/mysql.h>
>
> All build fine (Apache 2.2.0 / MySQL 5.0.18). But every time I access a
> secured page the Apache child process give up with:
> | [notice] Parent: child process exited with status 3221225477 --
> Restarting.
>
> After the automatic restart of the child process and a refresh in the
> browser I can see the page. Next (an other) page the same thing. I don't
> know where is the problem. "mod_dbd" or "apr_dbd*"?
>
> BTW:
> Have you ever try to use mod_dbd with sqlite2 or sqlite3 on Windows? When
> i try to use this, I get this message at server startup (and during
> accessing a page):
> | [crit] (20014)Internal error: DBD: failed to initialise
>
> But I don't realy know if my configuration in "httpd.conf" is correct for
> SQLite (there are no examples...).
>
> Regards,
> Carsten Wiedmann
>
>
>
Re: mod_dbd and Windows
Posted by Carsten Wiedmann <ca...@gmx.de>.
Nick Kew schrieb:
> On Friday 06 January 2006 19:20, Steffen wrote:
> > Several ppl tried it, but it crashes Apache or it looses connections,
> > see: http://www.apachelounge.com/forum/viewtopic.php?t=52
> >
> > To get it compiled, as i told you before, mysql.h has to be changed,
> > see: http://www.apachelounge.com/forum/viewtopic.php?p=280#280
>
> Looking at your URL, you've changed __WIN__ to __WIN32__ in mysql.h.
> That seems to be something internal to MySQL itself, outside the
> reach of the apr_dbd_mysql driver.
I've also tried to use mod_dbd and MySQL on Windows with no luck. I don't
must change "mysql.h" to compile the modules. Instead I must add this line
in "apr_dbd_mysql.c":
| #include <mysql/my_global.h>
before:
| #include <mysql/mysql.h>
All build fine (Apache 2.2.0 / MySQL 5.0.18). But every time I access a
secured page the Apache child process give up with:
| [notice] Parent: child process exited with status 3221225477 --
Restarting.
After the automatic restart of the child process and a refresh in the
browser I can see the page. Next (an other) page the same thing. I don't
know where is the problem. "mod_dbd" or "apr_dbd*"?
BTW:
Have you ever try to use mod_dbd with sqlite2 or sqlite3 on Windows? When i
try to use this, I get this message at server startup (and during accessing
a page):
| [crit] (20014)Internal error: DBD: failed to initialise
But I don't realy know if my configuration in "httpd.conf" is correct for
SQLite (there are no examples...).
Regards,
Carsten Wiedmann
Re: mod_dbd and Windows
Posted by Nick Kew <ni...@webthing.com>.
On Friday 06 January 2006 19:20, Steffen wrote:
> Hi,
>
> Maybe there is some help for the Windows community,
> we are still struggling to get mod_dbd working with Windows.
You seem to be equating dbd with mysql here?
> Several ppl tried it, but it crashes Apache or it looses connections,
> see: http://www.apachelounge.com/forum/viewtopic.php?t=52
>
> To get it compiled, as i told you before, mysql.h has to be changed,
> see: http://www.apachelounge.com/forum/viewtopic.php?p=280#280
I thought you said it worked with MySQL 4.1 but caused trouble with 5.0?
Looking at your URL, you've changed __WIN__ to __WIN32__ in mysql.h.
That seems to be something internal to MySQL itself, outside the
reach of the apr_dbd_mysql driver.
Anyway, as far as I'm concerned, the MySQL driver was only ever a quick
clone of the PostgreSQL driver (which I use myself). If you or anyone else
would like to take over maintenance of it (under the GPL), that's fine by me.
Or of course you could fork a Windows version of it.
Alternatively, it's now under subversion, and (round tuits permitting) should
also come under bug tracking in the not-too-distant future. If you think
you have something to contribute, let me know.
--
Nick Kew
Re: mod_dbd and Windows
Posted by Nick Kew <ni...@webthing.com>.
On Friday 06 January 2006 19:20, Steffen wrote:
> Hi,
>
> Maybe there is some help for the Windows community,
> we are still struggling to get mod_dbd working with Windows.
FWIW, I've just (coincidentally) got a genuine bug report from
another user, and fixed the bug. It's in dbd_mysql_open, so
if that's what gives you trouble, try again!
http://apache.webthing.com/svn/apache/apr/
--
Nick Kew
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by Ruediger Pluem <rp...@apache.org>.
On 01/06/2006 08:03 PM, Jim Jagielski wrote:
>
> On Jan 6, 2006, at 1:47 PM, Jim Jagielski wrote:
>
[..cut..]
>>
>
> I should clarify that: when the comment says "or" yet the
> code does an "and" then it causes undue confusion, even
> if the 2 do sync up.
I think with the adjustments you made to the comments it is now much
clearer what is done and this point is closed. Thanks for doing this.
Regards
Rüdiger
mod_dbd and Windows
Posted by Steffen <in...@apachelounge.com>.
Hi,
Maybe there is some help for the Windows community,
we are still struggling to get mod_dbd working with Windows.
Several ppl tried it, but it crashes Apache or it looses connections,
see: http://www.apachelounge.com/forum/viewtopic.php?t=52
To get it compiled, as i told you before, mysql.h has to be changed,
see: http://www.apachelounge.com/forum/viewtopic.php?p=280#280
Thanks,
Steffen
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by Jim Jagielski <ji...@jaguNET.com>.
On Jan 6, 2006, at 1:47 PM, Jim Jagielski wrote:
> Still not sure why you are using a specific error detection
> filter rather than the generic one in -trunk
>
> On Jan 5, 2006, at 2:59 PM, Ruediger Pluem wrote:
>
>> @@ -146,13 +162,20 @@
>> * 2) the trailer
>> * 3) the end-of-chunked body CRLF
>> *
>> - * If there is no EOS bucket, then do nothing.
>> + * If there is no EOS bucket, or if we had seen an error
>> bucket with
>> + * status HTTP_BAD_GATEWAY then do nothing. We have
>> memorized an
>> + * error bucket that we had seen in the filter context.
>> + * The error bucket with status HTTP_BAD_GATEWAY
>> indicates that the
>> + * connection to the backend (mod_proxy) broke in the
>> middle of the
>> + * response. In order to signal the client that something
>> went wrong
>> + * we do not create the last-chunk marker and set c-
>> >keepalive to
>> + * AP_CONN_CLOSE in the core output filter.
>> *
>> * XXX: it would be nice to combine this with the end-of-
>> chunk
>> * marker above, but this is a bit more straight-forward for
>> * now.
>> */
>> - if (eos != NULL) {
>> + if (eos && !f->ctx) {
>
> Code logic doesn't match comment.
>
I should clarify that: when the comment says "or" yet the
code does an "and" then it causes undue confusion, even
if the 2 do sync up.
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by Ruediger Pluem <rp...@apache.org>.
On 01/06/2006 07:47 PM, Jim Jagielski wrote:
> Still not sure why you are using a specific error detection
> filter rather than the generic one in -trunk
>
Arghh. Sorry I must have missed to include your patch that changed this.
Thanks for catching this. I will provide a new patch.
[..cut..]
>> + /*
>> + * Ensure that we sent an EOS bucket thru the filter chain, if
>> we already
>> + * have sent some data. Maybe ap_proxy_backend_broke was called
>> and added
>> + * one to the brigade already. So we should not do this in this
>> case.
>> + */
>> + if (data_sent && !r->eos_sent && APR_BRIGADE_EMPTY
>> (output_brigade)) {
>> + e = apr_bucket_eos_create(r->connection->bucket_alloc);
>> + APR_BRIGADE_INSERT_TAIL(output_brigade, e);
>> + }
>> +
>>
>
> Also, if data_sent is true, then ap_proxy_backend_broke() already
> sent the EOS, so why are we checking if it's true again? I
> think the logic is wrong...
No, there is also the case that the client aborted the connection.
In this case
status == APR_SUCCESS
So the outer if condition around ap_proxy_backend_broke is not true and it is
not called. Anyway since we had sent some data we should sent an EOS if we
have not done already to make all filters on the chain aware of it.
Especially mod_disk_cache needs to know in order to remove the temporary
file created for the cache entry.
Regards
Rüdiger
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by Jim Jagielski <ji...@jaguNET.com>.
Still not sure why you are using a specific error detection
filter rather than the generic one in -trunk
On Jan 5, 2006, at 2:59 PM, Ruediger Pluem wrote:
> @@ -146,13 +162,20 @@
> * 2) the trailer
> * 3) the end-of-chunked body CRLF
> *
> - * If there is no EOS bucket, then do nothing.
> + * If there is no EOS bucket, or if we had seen an error
> bucket with
> + * status HTTP_BAD_GATEWAY then do nothing. We have
> memorized an
> + * error bucket that we had seen in the filter context.
> + * The error bucket with status HTTP_BAD_GATEWAY indicates
> that the
> + * connection to the backend (mod_proxy) broke in the
> middle of the
> + * response. In order to signal the client that something
> went wrong
> + * we do not create the last-chunk marker and set c-
> >keepalive to
> + * AP_CONN_CLOSE in the core output filter.
> *
> * XXX: it would be nice to combine this with the end-of-
> chunk
> * marker above, but this is a bit more straight-forward for
> * now.
> */
> - if (eos != NULL) {
> + if (eos && !f->ctx) {
Code logic doesn't match comment.
> + /*
> + * Ensure that we sent an EOS bucket thru the filter chain, if
> we already
> + * have sent some data. Maybe ap_proxy_backend_broke was
> called and added
> + * one to the brigade already. So we should not do this in
> this case.
> + */
> + if (data_sent && !r->eos_sent && APR_BRIGADE_EMPTY
> (output_brigade)) {
> + e = apr_bucket_eos_create(r->connection->bucket_alloc);
> + APR_BRIGADE_INSERT_TAIL(output_brigade, e);
> + }
> +
>
Also, if data_sent is true, then ap_proxy_backend_broke() already
sent the EOS, so why are we checking if it's true again? I
think the logic is wrong...
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
William A. Rowe, Jr. wrote:
> Ruediger Pluem wrote:
>
>>
>> Thanks to Jim for reviewing the patch. He detected one missed patch
>> and made some comments in the code clearer. The new patch list now:
>
>
> Quick consideration;
>
> Rather than look for HTTP_BAD_GATEWAY error bucket, we can actually
> generalize the problem. ANY metadata bucket that isn't recognized and
> handled by an intermediate filter probably indiciates a problem; and
> therefore the result is a non-cacheable, broken response.
Actually two cases. In the error bucket case, it's non-cacheable, and broken.
In the unrecognized bucket type case, it's non-cacheable (a 'complex' response),
but it is likely serveable to the front end client. In both cases, if mod_cache
doesn't grok what it sees, then something 'interesting' is going on and we would
not want to deposit into the cache.
This would guard against new designs causing new troublesome interactions with
mod_cache. As some new bucket type is 'taught' to mod_cache, it can grow to
handle otherwise harmless requests. But I suspect that the module should err
on the side of paranoia.
Bill
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by Graham Leggett <mi...@sharp.fm>.
William A. Rowe, Jr. wrote:
> Rather than look for HTTP_BAD_GATEWAY error bucket, we can actually
> generalize the problem. ANY metadata bucket that isn't recognized and
> handled by an intermediate filter probably indiciates a problem; and
> therefore the result is a non-cacheable, broken response.
+1.
Regards,
Graham
--
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Ruediger Pluem wrote:
>
> Thanks to Jim for reviewing the patch. He detected one missed patch
> and made some comments in the code clearer. The new patch list now:
Quick consideration;
Rather than look for HTTP_BAD_GATEWAY error bucket, we can actually
generalize the problem. ANY metadata bucket that isn't recognized and
handled by an intermediate filter probably indiciates a problem; and
therefore the result is a non-cacheable, broken response.
Thoughts?
Bill
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by Ruediger Pluem <rp...@apache.org>.
On 01/05/2006 08:59 PM, Ruediger Pluem wrote:
>
> On 01/05/2006 01:51 PM, Ruediger Pluem wrote:
>
> [..cut..]
>
> I finally merged all the commits done to the trunk on this issue
Thanks to Jim for reviewing the patch. He detected one missed patch
and made some comments in the code clearer. The new patch list now:
r354628
r354636
r357461
r357519
r358022 <- new
r365374
r366181
r366554 <- new
r366558 <- new
I created a new merged patch that works with 2.2.x. So the same procedure
again please :).
I would like Brian (Akins) to give it a try and of course if someone else has a
look on it, it wouldn't hurt.
Regards
Rüdiger
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by Ruediger Pluem <rp...@apache.org>.
On 01/05/2006 01:51 PM, Ruediger Pluem wrote:
[..cut..]
I finally merged all the commits done to the trunk on this issue
r354628
r354636
r357461
r357519
r365374
r366181
into one patch that works with 2.2.x. From my current point of view all
aspects of this issue should be considered by this patch.
I would like Brian (Akins) to give it a try and of course if someone else has a
look on it, it wouldn't hurt.
Regards
Rüdiger
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by Ruediger Pluem <rp...@apache.org>.
On 01/05/2006 12:58 PM, Joe Orton wrote:
> On Thu, Jan 05, 2006 at 12:48:25PM +0100, Ruediger Pluem wrote:
[..cut..]
>
> André's trick of using invented unique pointers by doing:
>
> static char sentinel; (in global scope)
>
> f->ctx = &sentinel;
>
> is neater and avoids the casting mess. (pick a suitable variable name
> though :)
Thanks Joe for *pointing* me to this. :-)
This is really nice trick. I will use it.
Regards
Rüdiger
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by Joe Orton <jo...@redhat.com>.
On Thu, Jan 05, 2006 at 12:48:25PM +0100, Ruediger Pluem wrote:
> But I remember myself that there had been casting issues in the past that
> created compiler warnings especially with gcc 4. The patch below compiles
> fine with my gcc 3.2.2 with -Wall. So if someone could give a comment
> if
>
> f->ctx = (void *)(1)
>
> is fine it would be great.
André's trick of using invented unique pointers by doing:
static char sentinel; (in global scope)
f->ctx = &sentinel;
is neater and avoids the casting mess. (pick a suitable variable name
though :)
joe
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by Ruediger Pluem <rp...@apache.org>.
On 01/03/2006 06:15 PM, Jim Jagielski wrote:
>
> On Jan 2, 2006, at 4:18 PM, Ruediger Pluem wrote:
>
>>
>> 1. Proposal
>> If a subrequest has a broken backend also set r->no_cache for the
>> main request
>> and ensure that the chunk filter does not sent the last chunk marker
>> in this case.
>>
>> 2. Proposal
>> If a subrequest has a broken backend do not sent the error bucket.
>> Only set r->no_cache
>> to ensure that this subrequest response does not get cached.
>>
>
I am currently working on proposal #1. In order to ensure that the chunk
filter does not sent a last chunk marker in this case I try to memorize
a seen error bucket in the filter context.
But I remember myself that there had been casting issues in the past that
created compiler warnings especially with gcc 4. The patch below compiles
fine with my gcc 3.2.2 with -Wall. So if someone could give a comment
if
f->ctx = (void *)(1)
is fine it would be great.
Regards
Rüdiger
Index: chunk_filter.c
===================================================================
--- chunk_filter.c (Revision 366160)
+++ chunk_filter.c (Arbeitskopie)
@@ -47,7 +47,6 @@
apr_bucket_brigade *more;
apr_bucket *e;
apr_status_t rv;
- int bad_gateway_seen = 0;
for (more = NULL; b; b = more, more = NULL) {
apr_off_t bytes = 0;
@@ -71,8 +70,11 @@
if (AP_BUCKET_IS_ERROR(e)
&& (((ap_bucket_error *)(e->data))->status
== HTTP_BAD_GATEWAY)) {
- /* We had a broken backend. Memorize this. */
- bad_gateway_seen = 1;
+ /*
+ * We had a broken backend. Memorize this in the filter
+ * context.
+ */
+ f->ctx = (void *)(1);
continue;
}
if (APR_BUCKET_IS_FLUSH(e)) {
@@ -155,7 +157,8 @@
* 3) the end-of-chunked body CRLF
*
* If there is no EOS bucket, or if we had seen an error bucket with
- * status HTTP_BAD_GATEWAY then do nothing.
+ * status HTTP_BAD_GATEWAY then do nothing. We have memorized an
+ * error bucket that we had seen in the filter context.
* The error bucket with status HTTP_BAD_GATEWAY indicates that the
* connection to the backend (mod_proxy) broke in the middle of the
* response. In order to signal the client that something went wrong
@@ -166,7 +169,7 @@
* marker above, but this is a bit more straight-forward for
* now.
*/
- if (eos && !bad_gateway_seen) {
+ if (eos && !f->ctx) {
/* XXX: (2) trailers ... does not yet exist */
e = apr_bucket_immortal_create(ASCII_ZERO ASCII_CRLF
/* <trailers> */
Re: AW: AW: AW: 2.2 mod_http_proxy and "partial" pages
Posted by Jim Jagielski <ji...@apache.org>.
On Jan 2, 2006, at 4:18 PM, Ruediger Pluem wrote:
>
> 1. Proposal
> If a subrequest has a broken backend also set r->no_cache for the
> main request
> and ensure that the chunk filter does not sent the last chunk
> marker in this case.
>
> 2. Proposal
> If a subrequest has a broken backend do not sent the error bucket.
> Only set r->no_cache
> to ensure that this subrequest response does not get cached.
>
Proposal #1