You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Stas Bekman <st...@stason.org> on 2004/03/02 05:07:06 UTC
EOS is not being sent for HEAD requests
in 2.0 - Connection output filters don't recieve the EOS bucket when the
request method is HEAD. This breaks the idiom of flushing any stored data when
the eos bucket is seen. And some filters could be broken because of that.
A typical response filter sees this for the following 3 brigades for a HEAD
request which otherwise would have sent a response body:
>>> connection output filter
o bucket 1: HEAP
[HTTP/1.1 200 OK
Date: Tue, 02 Mar 2004 04:04:30 GMT
Server: Apache/2.0.49-dev (Unix) mod_perl/1.99_13-dev Perl/v5.8.3
mod_ssl/2.0.49-dev OpenSSL/0.9.7b DAV/2
Connection: close
Content-Type: text/plain
]
>>> connection output filter
o bucket 1: FLUSH
[]
>>> connection output filter
o bucket 1: FLUSH
[]
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: EOS is not being sent for HEAD requests
Posted by Bojan Smojver <bo...@rexursive.com>.
On Wed, 2004-03-03 at 10:27, Cliff Woolley wrote:
> That would be a problem because then you'd have a FLUSH bucket *after*
> your EOS bucket in brigade bb (or at least that's what I assume from the
> context given here without actually pulling up the whole module), which is
> illegal.
I think the code I was trying out was actually slightly different. I was
inserting FLUSH before the EOS with APR_BUCKET_INSERT_BEFORE. Anyway, I
might try that with Apache 2.1 one of these days. No promises, however,
some other stuff is keeping me pretty busy these days.
> PS: a shorthand for APR_BUCKET_REMOVE()+apr_bucket_destroy() is
> apr_bucket_delete().
Thanks.
--
Bojan
Re: EOS is not being sent for HEAD requests
Posted by Cliff Woolley <jw...@virginia.edu>.
On Wed, 3 Mar 2004, Bojan Smojver wrote:
> If you have mod_logio configured, this might be its doing. It replaces
> the EOS with FLUSH. This is in order to make sure all output is counted
> and logged properly for each request.
That's bad. :)
If you have to do that, then something else is broken.
> /* End of data, make sure we flush */
> if (APR_BUCKET_IS_EOS(b)) {
> APR_BRIGADE_INSERT_TAIL(bb,
> apr_bucket_flush_create(f->c->bucket_alloc));
> APR_BUCKET_REMOVE(b);
> apr_bucket_destroy(b);
> }
>
> return ap_pass_brigade(f->next, bb);
> }
> -------------------------------------------------------
>
> I remember trying without APR_BUCKET_REMOVE/apr_bucket_destroy, but that
> kept giving me problems. This was long time ago and I don't remember
> exactly what the nature of those problems.
That would be a problem because then you'd have a FLUSH bucket *after*
your EOS bucket in brigade bb (or at least that's what I assume from the
context given here without actually pulling up the whole module), which is
illegal.
PS: a shorthand for APR_BUCKET_REMOVE()+apr_bucket_destroy() is
apr_bucket_delete().
--Cliff
Re: EOS is not being sent for HEAD requests
Posted by Bojan Smojver <bo...@rexursive.com>.
On Wed, 2004-03-03 at 07:42, Stas Bekman wrote:
> ... but he just gave a good example
> where a filter is not well behaved (by a non-existing well-behaved-filters
> Apache spec).
I'm not sure exactly why I can't have a FLUSH bucket followed by an EOS
bucket, but I'm pretty sure I wanted to keep the EOS, it just that the
filter wouldn't work properly that way :-(. Anyhow, if Apache 2.1 is
different in that regard, maybe I should download the CVS stuff and give
it another go.
Also, if you remember, there was a thread a while back about multiple
FILE buckets in the same brigade. That was causing dramas unless after
each FILE bucket there was a FLUSH one. So, now in my own module, which
creates a brigade with multiple FILE buckets, I have insert FLUSH after
each one. I bet this isn't all that good for performance.
--
Bojan
Re: EOS is not being sent for HEAD requests
Posted by Stas Bekman <st...@stason.org>.
Bojan Smojver wrote:
> If you have mod_logio configured, this might be its doing. It replaces
> the EOS with FLUSH. This is in order to make sure all output is counted
> and logged properly for each request.
>
> Here is the code from mod_logio that does that:
>
> -------------------------------------------------------
> static apr_status_t logio_out_filter(ap_filter_t *f,
> apr_bucket_brigade *bb) {
> apr_bucket *b = APR_BRIGADE_LAST(bb);
>
> /* End of data, make sure we flush */
> if (APR_BUCKET_IS_EOS(b)) {
> APR_BRIGADE_INSERT_TAIL(bb,
> apr_bucket_flush_create(f->c->bucket_alloc));
> APR_BUCKET_REMOVE(b);
> apr_bucket_destroy(b);
> }
>
> return ap_pass_brigade(f->next, bb);
> }
> -------------------------------------------------------
>
> I remember trying without APR_BUCKET_REMOVE/apr_bucket_destroy, but that
> kept giving me problems. This was long time ago and I don't remember
> exactly what the nature of those problems.
>
> Anyway, may be the culprit...
>
Thanks Bojan, but no I don't use it.
It just shows how fragile the reliance on buckets to deliver information is.
Any filter may decide to do whatever it wants and if you get this kind of
problem how do you figure out where the problem is? Plug yourself into the
filter chain and watch which filter eats the bucket? Supposedly I can do that
on my machine. What happens to a user who can't do that? And even after
finding the faulty filter it'll take time to fix the problem. Meanwhile my
module will be unusable as long as users don't update the faulty filter. If I
didn't have to rely on other filters behaving well and my code is proper I
won't need to worry about things and spend time creating new things rather
than trying to find the faulty filter.
That's why I'd prefer for Apache to have an API for these events, in addition
to the meta buckets...
p.s. I have nothing against Bojan's mod_logio, but he just gave a good example
where a filter is not well behaved (by a non-existing well-behaved-filters
Apache spec).
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: EOS is not being sent for HEAD requests
Posted by Bojan Smojver <bo...@rexursive.com>.
If you have mod_logio configured, this might be its doing. It replaces
the EOS with FLUSH. This is in order to make sure all output is counted
and logged properly for each request.
Here is the code from mod_logio that does that:
-------------------------------------------------------
static apr_status_t logio_out_filter(ap_filter_t *f,
apr_bucket_brigade *bb) {
apr_bucket *b = APR_BRIGADE_LAST(bb);
/* End of data, make sure we flush */
if (APR_BUCKET_IS_EOS(b)) {
APR_BRIGADE_INSERT_TAIL(bb,
apr_bucket_flush_create(f->c->bucket_alloc));
APR_BUCKET_REMOVE(b);
apr_bucket_destroy(b);
}
return ap_pass_brigade(f->next, bb);
}
-------------------------------------------------------
I remember trying without APR_BUCKET_REMOVE/apr_bucket_destroy, but that
kept giving me problems. This was long time ago and I don't remember
exactly what the nature of those problems.
Anyway, may be the culprit...
--
Bojan
Re: EOS is not being sent for HEAD requests
Posted by Stas Bekman <st...@stason.org>.
Justin Erenkrantz wrote:
> --On Monday, March 1, 2004 11:20 PM -0800 Stas Bekman <st...@stason.org>
> wrote:
>
>> I've stepped through with gdb, ap_finalize_request_protocol is called and
>> EOS is sent, but it gets lost and doesn't reach the connection output
>> filter.
>
>
> That'd mean that a filter isn't passing it on as it should be. So, what
> filter loses it? Is it one of ours? Or, is it a custom one?
>
> As always, more details would be appreciated. -- justin
I'm just installing a passive snooper filter (one snooping on incoming
connection and one on the outgoing) that passes the data through as is and
dumps bbs contents and bucket types to error log. So it must be some of the
core filters I believe. Here are the traces:
HEAD "/index.html"
----------------------------
<<< connection input filter
o bucket 1: HEAP
[HEAD /index.html HTTP/1.0
]
<<< connection input filter
o bucket 1: HEAP
[Host: localhost.localdomain:8529
]
<<< connection input filter
o bucket 1: HEAP
[User-Agent: libwww-perl/5.76
]
<<< connection input filter
o bucket 1: HEAP
[
]
>>> connection output filter
o bucket 1: HEAP
[HTTP/1.1 200 OK
Date: Tue, 02 Mar 2004 18:22:46 GMT
Server: Apache/2.0.49-dev (Unix) mod_perl/1.99_13-dev Perl/v5.8.3
mod_ssl/2.0.49-dev OpenSSL/0.9.7b DAV/2
Last-Modified: Thu, 07 Aug 2003 19:13:15 GMT
ETag: "23cb9-2d0-830a88c0"
Accept-Ranges: bytes
Content-Length: 720
Connection: close
Content-Type: text/html
]
>>> connection output filter
o bucket 1: FLUSH
[]
>>> connection output filter
o bucket 1: FLUSH
[]
----------------------------
No EOS as you can see
Now issue GET "/index.html"
----------------------------
<<< connection input filter
o bucket 1: HEAP
[GET /index.html HTTP/1.0
]
<<< connection input filter
o bucket 1: HEAP
[Host: localhost.localdomain:8529
]
<<< connection input filter
o bucket 1: HEAP
[User-Agent: libwww-perl/5.76
]
<<< connection input filter
o bucket 1: HEAP
[
]
>>> connection output filter
o bucket 1: HEAP
[HTTP/1.1 200 OK
Date: Tue, 02 Mar 2004 18:22:45 GMT
Server: Apache/2.0.49-dev (Unix) mod_perl/1.99_13-dev Perl/v5.8.3
mod_ssl/2.0.49-dev OpenSSL/0.9.7b DAV/2
Last-Modified: Thu, 07 Aug 2003 19:13:15 GMT
ETag: "23cb9-2d0-830a88c0"
Accept-Ranges: bytes
Content-Length: 720
Connection: close
Content-Type: text/html
]
>>> connection output filter
o bucket 1: MMAP
[<!-- WARNING: this file is generated, do not edit
01: /home/stas/apache.org/modperl-2.0/Apache-Test/lib/Apache/TestConfig.pm:739
02: /home/stas/apache.org/modperl-2.0/Apache-Test/lib/Apache/TestConfig.pm:756
03: /home/stas/apache.org/modperl-2.0/Apache-Test/lib/Apache/TestConfig.pm:1030
04: /home/stas/apache.org/modperl-2.0/Apache-Test/lib/Apache/TestConfig.pm:1233
05: /home/stas/apache.org/modperl-2.0/Apache-Test/lib/Apache/TestRun.pm:408
06: /home/stas/apache.org/modperl-2.0/Apache-Test/lib/Apache/TestRunPerl.pm:41
07: /home/stas/apache.org/modperl-2.0/Apache-Test/lib/Apache/TestRun.pm:589
08: /home/stas/apache.org/modperl-2.0/Apache-Test/lib/Apache/TestRun.pm:589
09: t/TEST:21
-->
welcome to localhost:8529
]
o bucket 2: EOS
[]
>>> connection output filter
o bucket 1: FLUSH
[]
>>> connection output filter
o bucket 1: FLUSH
[]
----------------------------
EOS is there.
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: EOS is not being sent for HEAD requests
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
--On Monday, March 1, 2004 11:20 PM -0800 Stas Bekman <st...@stason.org> wrote:
> I've stepped through with gdb, ap_finalize_request_protocol is called and
> EOS is sent, but it gets lost and doesn't reach the connection output filter.
That'd mean that a filter isn't passing it on as it should be. So, what
filter loses it? Is it one of ours? Or, is it a custom one?
As always, more details would be appreciated. -- justin
Re: EOS is not being sent for HEAD requests
Posted by Stas Bekman <st...@stason.org>.
Justin Erenkrantz wrote:
> --On Monday, March 1, 2004 9:55 PM -0800 Stas Bekman <st...@stason.org>
> wrote:
>
>> I see that with a non-broken handler which sends the response body
>> unconditionally. The output I've posted in my original email is all
>> that's
>> seen by an output connection filter.
>
>
> Why is ap_finalize_request_protocol not being called?
>
> That is the code that should be ensuring that EOS is passed down even if
> the handler is 'broken.' -- justin
I've stepped through with gdb, ap_finalize_request_protocol is called and EOS
is sent, but it gets lost and doesn't reach the connection output filter.
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: EOS is not being sent for HEAD requests
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
--On Monday, March 1, 2004 9:55 PM -0800 Stas Bekman <st...@stason.org> wrote:
> I see that with a non-broken handler which sends the response body
> unconditionally. The output I've posted in my original email is all that's
> seen by an output connection filter.
Why is ap_finalize_request_protocol not being called?
That is the code that should be ensuring that EOS is passed down even if the
handler is 'broken.' -- justin
Re: EOS is not being sent for HEAD requests
Posted by Stas Bekman <st...@stason.org>.
André Malo wrote:
> * Stas Bekman <st...@stason.org> wrote:
>
>
>>in 2.0 - Connection output filters don't recieve the EOS bucket when the
>>request method is HEAD. This breaks the idiom of flushing any stored data
>>when the eos bucket is seen. And some filters could be broken because of
>>that.
>
>
> I believe that is because of broken handlers, which don't send anything if
> they detect HEAD. In 2.0 a handler should not care about GET or HEAD, but send
> all the time its data, so that all the processing (C-L filter etc) works
> properly.
I see that with a non-broken handler which sends the response body
unconditionally. The output I've posted in my original email is all that's
seen by an output connection filter.
Besides, shouldn't the output connection filter see the response data
nevertheless? otherwise how can you calculate a correct C-L when the filter
weren't given a change to change the data, potentially changing its length.
I'm talking about dynamic handlers, not static data here.
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Re: EOS is not being sent for HEAD requests
Posted by André Malo <nd...@perlig.de>.
* Stas Bekman <st...@stason.org> wrote:
> in 2.0 - Connection output filters don't recieve the EOS bucket when the
> request method is HEAD. This breaks the idiom of flushing any stored data
> when the eos bucket is seen. And some filters could be broken because of
> that.
I believe that is because of broken handlers, which don't send anything if
they detect HEAD. In 2.0 a handler should not care about GET or HEAD, but send
all the time its data, so that all the processing (C-L filter etc) works
properly.
nd