You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modproxy-dev@apache.org by Ian Holsman <ia...@cnet.com> on 2001/07/17 03:56:33 UTC
Re: Nasty chunking bug (in MSIE?) when using ap_rwrite()/ap_rflush()
TOKILEY@aol.com wrote:
>In a message dated 01-07-16 18:16:37 EDT, Ian Holsman wrote...
>
>>On 25 Jun 2001 12:13:51 -0400, Bill Stoddard wrote:
>> > I have a module that calls ap_rwrite() followed by ap_rflush(). Content
>>length is not
>> > provided so Apache 2.0 chunks the response.
>> >
>> > Here is what happens...
>> > I call ap_rwrite() to write a x75 len byte stream. All the correct
>>headers are built, and
>> > the content is buffered by the OLD_WRITE filter. Then I call ap_rflush()
>>which causes the
>> > headers to be sent on the wire along with the first chunk (x75 length).
>>
>My
>
>>handler is done
>> > at this point and returns control to Apache. The next thing Apache
>>
>sends
>
>>on the wire is
>> > the 0 byte chunk header to indicate that the response is done.
>> >
>> > The problem: IE chokes when it receives just the 0 byte chunk header in
>>
>a
>
>>packet.
>> >
>> > Thoughts?
>> >
>> > Bill
>>
>>
>> hi Bill,
>> I was wondering if you got anywhere with this
>> I'm seeing this with mod_proxy at the moment.
>>
>> my inital thought was to put some logic in the chunking filter
>> just not to send the 0 byte chunk out.. can you see anything barfing
>> on this?
>>
>> what do you think?
>>
>> --
>> Ian Holsman IanH@cnet.com
>> Performance Measurement & Analysis
>> CNET Networks - (415) 364-8608
>>
>
>You have to send the 0 byte chunk... but that's not all
>there is to it. The 'Entity headers' and then the final
>'blank line' (CR/LF) is what actually ENDS the
>chunked transfer.
>
>You have to 'hang tough' on the client side on a 'Keep-Alive'
>and only use the final blank CR/LF as an EOD signal. You
>can't bail when the '0' chunk length shows up or you could
>end up leaving the Entity headers + final blank CR/LF in
>the input buffer and think these chars are the start of
>the next response when you get back to reading your
>input buffers on the NIC on next Keep-Alive.
>
>Whoever/whatever is 'adding' the '0' byte must also be
>adding A) The CR/LF that follows the 0 byte hex string
>B) All of the Entity Headers + CR/LF ending each one
>C) The final 'blank' CR/LF that ends the chunking.
>
>If any of the above is missing maybe that is why
>MSIE is 'choking'.
>
MSIE is choking due to a '0' byte happening in the middle of the
response being sent back.
I think this is due to the proxy code sending a EOS bucket after the
subrequest. (it doesn't happen with non-proxied
requests), hopefully it will be easy to find.
>
>
>I would find it hard to belive that a modern browser would
>screw up a chunking byte length appearing all by itself
>at the top of a new packet ( Even though that's highly
>unusual... usually it's right at the end of the data in
>the same final data packet ).
>
>Kevin Kiley
>
..Ian
Re: Nasty chunking bug (in MSIE?) when using ap_rwrite()/ap_rflush()
Posted by Chuck Murcko <ch...@topsail.org>.
On Thursday, July 19, 2001, at 06:01 PM, Ian Holsman wrote:
> seeing how people are tagging/building the next .21 beta
> do we want this patch applied?
> as AS IS using IE and HTTP/1.1 is kinda hosed.
>
>
>
One more try...my mail is bad today.
+1 as long as we fix the real problem for release.
If no one objects, Ian can commit and I will tag & build a source
release.
Chuck Murcko
Topsail Group
http://www.topsail.org/
Re: Nasty chunking bug (in MSIE?) when using ap_rwrite()/ap_rflush()
Posted by Ian Holsman <ia...@cnet.com>.
seeing how people are tagging/building the next .21 beta
do we want this patch applied?
as AS IS using IE and HTTP/1.1 is kinda hosed.
On 17 Jul 2001 11:10:10 -0700, Ian Holsman wrote:
> Victor J. Orlikowski wrote:
>
> > > this seems to fix the problem...
> > > can someone more familiar with brigades/buckets review it
> > >
> > > Index: proxy_http.c
> > > ===================================================================
> > > RCS file: /home/cvspublic/httpd-proxy/module-2.0/proxy_http.c,v
> > > retrieving revision 1.79
> > > diff -u -r1.79 proxy_http.c
> > > --- proxy_http.c 2001/07/16 17:54:38 1.79
> > > +++ proxy_http.c 2001/07/17 16:07:32
> > > @@ -782,18 +782,20 @@
> > >
> > > /* read the body, pass it to the output filters */
> > > while (ap_get_brigade(rp->input_filters, bb, AP_MODE_BLOCKING,
> > > &readbyte
> > > s) == APR_SUCCESS) {
> > > - if (APR_BUCKET_IS_EOS(APR_BRIGADE_LAST(bb))) {
> > > + apr_bucket *b = APR_BRIGADE_LAST(bb);
> > > + if (APR_BUCKET_IS_EOS(b)) {
> > > + APR_BUCKET_REMOVE(b);
> > > e = apr_bucket_flush_create();
> > > APR_BRIGADE_INSERT_TAIL(bb, e);
> > > ap_pass_brigade(r->output_filters, bb);
> >
> >Hum....
> >I may be mistaken, but the end-of-stream bucket still needs to be
> >there. This patch makes certain that there will *never* be an EOS
> >bucket.
> >
> I'm still seeing a EOS bucket being processed
> [Tue Jul 17 09:07:20 2001] [error] [client 127.0.0.1] EOS Bucket in
> Chunk_filter /test/main.shtml (created at
> c:\src\apache\httpd-2.0\server\protocol.c 736)
>
> >
> >
> >Feel free to correct me on this.
> >
> >Victor
> >
>
>
--
Ian Holsman IanH@cnet.com
Performance Measurement & Analysis
CNET Networks - (415) 364-8608
Re: Nasty chunking bug (in MSIE?) when using ap_rwrite()/ap_rflush()
Posted by Ian Holsman <ia...@cnet.com>.
Victor J. Orlikowski wrote:
> > this seems to fix the problem...
> > can someone more familiar with brigades/buckets review it
> >
> > Index: proxy_http.c
> > ===================================================================
> > RCS file: /home/cvspublic/httpd-proxy/module-2.0/proxy_http.c,v
> > retrieving revision 1.79
> > diff -u -r1.79 proxy_http.c
> > --- proxy_http.c 2001/07/16 17:54:38 1.79
> > +++ proxy_http.c 2001/07/17 16:07:32
> > @@ -782,18 +782,20 @@
> >
> > /* read the body, pass it to the output filters */
> > while (ap_get_brigade(rp->input_filters, bb, AP_MODE_BLOCKING,
> > &readbyte
> > s) == APR_SUCCESS) {
> > - if (APR_BUCKET_IS_EOS(APR_BRIGADE_LAST(bb))) {
> > + apr_bucket *b = APR_BRIGADE_LAST(bb);
> > + if (APR_BUCKET_IS_EOS(b)) {
> > + APR_BUCKET_REMOVE(b);
> > e = apr_bucket_flush_create();
> > APR_BRIGADE_INSERT_TAIL(bb, e);
> > ap_pass_brigade(r->output_filters, bb);
>
>Hum....
>I may be mistaken, but the end-of-stream bucket still needs to be
>there. This patch makes certain that there will *never* be an EOS
>bucket.
>
I'm still seeing a EOS bucket being processed
[Tue Jul 17 09:07:20 2001] [error] [client 127.0.0.1] EOS Bucket in
Chunk_filter /test/main.shtml (created at
c:\src\apache\httpd-2.0\server\protocol.c 736)
>
>
>Feel free to correct me on this.
>
>Victor
>
Re: Nasty chunking bug (in MSIE?) when using ap_rwrite()/ap_rflush()
Posted by Chuck Murcko <ch...@topsail.org>.
On Tuesday, July 17, 2001, at 12:54 PM, Chuck Murcko wrote:
> Well, if the dechunk filter is not removing intermediate eos inserted
> by the http_in or core_in filters, as it should, then this allows the
> whole set of chunks through, and we should close the connection when
> finished, and we make the extra call into the dechunk filter to reset
> its state machine.
>
> While this will work, I think the fix should go into the dechunk filter
> in http_protocol.c. Then the eos check here (and more importantly, in
> any other filter using the dechunk filter ahead of it) works as it
> should.
>
> The dechunk filter should remove all intermediate eos and append one
> final one when chunked content ends.
>
I believe the proxy may still have to check for APR_EOF buckets, as well.
> Chuck
>
> On Tuesday, July 17, 2001, at 12:15 PM, Victor J. Orlikowski wrote:
> Hum....
> I may be mistaken, but the end-of-stream bucket still needs to be
> there. This patch makes certain that there will *never* be an EOS
> bucket.
>
> Feel free to correct me on this.
>
>
>>> this seems to fix the problem...
>>> can someone more familiar with brigades/buckets review it
>>>
>>> Index: proxy_http.c
>>> ===================================================================
>>> RCS file: /home/cvspublic/httpd-proxy/module-2.0/proxy_http.c,v
>>> retrieving revision 1.79
>>> diff -u -r1.79 proxy_http.c
>>> --- proxy_http.c 2001/07/16 17:54:38 1.79
>>> +++ proxy_http.c 2001/07/17 16:07:32
>>> @@ -782,18 +782,20 @@
>>>
>>> /* read the body, pass it to the output filters */
>>> while (ap_get_brigade(rp->input_filters, bb, AP_MODE_BLOCKING,
>>> &readbyte
>>> s) == APR_SUCCESS) {
>>> - if (APR_BUCKET_IS_EOS(APR_BRIGADE_LAST(bb))) {
>>> + apr_bucket *b = APR_BRIGADE_LAST(bb);
>>> + if (APR_BUCKET_IS_EOS(b)) {
>>> + APR_BUCKET_REMOVE(b);
>>> e = apr_bucket_flush_create();
>>> APR_BRIGADE_INSERT_TAIL(bb, e);
>>> ap_pass_brigade(r->output_filters, bb);
>>
Chuck Murcko
Topsail Group
http://www.topsail.org/
>
Re: Nasty chunking bug (in MSIE?) when using ap_rwrite()/ap_rflush()
Posted by Chuck Murcko <ch...@topsail.org>.
Well, if the dechunk filter is not removing intermediate eos inserted by
the http_in or core_in filters, as it should, then this allows the whole
set of chunks through, and we should close the connection when finished,
and we make the extra call into the dechunk filter to reset its state
machine.
While this will work, I think the fix should go into the dechunk filter
in http_protocol.c. Then the eos check here (an more importantly, in any
other filter using the dechunk filter ahead of it) works as it should.
The dechunk filter should remove all internediate eos and append one
final one when chunked content ends.
Chuck
On Tuesday, July 17, 2001, at 12:15 PM, Victor J. Orlikowski wrote:
>> this seems to fix the problem...
>> can someone more familiar with brigades/buckets review it
>>
>> Index: proxy_http.c
>> ===================================================================
>> RCS file: /home/cvspublic/httpd-proxy/module-2.0/proxy_http.c,v
>> retrieving revision 1.79
>> diff -u -r1.79 proxy_http.c
>> --- proxy_http.c 2001/07/16 17:54:38 1.79
>> +++ proxy_http.c 2001/07/17 16:07:32
>> @@ -782,18 +782,20 @@
>>
>> /* read the body, pass it to the output filters */
>> while (ap_get_brigade(rp->input_filters, bb, AP_MODE_BLOCKING,
>> &readbyte
>> s) == APR_SUCCESS) {
>> - if (APR_BUCKET_IS_EOS(APR_BRIGADE_LAST(bb))) {
>> + apr_bucket *b = APR_BRIGADE_LAST(bb);
>> + if (APR_BUCKET_IS_EOS(b)) {
>> + APR_BUCKET_REMOVE(b);
>> e = apr_bucket_flush_create();
>> APR_BRIGADE_INSERT_TAIL(bb, e);
>> ap_pass_brigade(r->output_filters, bb);
>
> Hum....
> I may be mistaken, but the end-of-stream bucket still needs to be
> there. This patch makes certain that there will *never* be an EOS
> bucket.
>
> Feel free to correct me on this.
>
> Victor
> --
> Victor J. Orlikowski | The Wall is Down, But the Threat Remains!
> ==================================================================
> v.j.orlikowski@gte.net | orlikowski@apache.org | vjo@us.ibm.com
>
Chuck Murcko
Topsail Group
http://www.topsail.org/
Re: Nasty chunking bug (in MSIE?) when using ap_rwrite()/ap_rflush()
Posted by "Victor J. Orlikowski" <v....@gte.net>.
> this seems to fix the problem...
> can someone more familiar with brigades/buckets review it
>
> Index: proxy_http.c
> ===================================================================
> RCS file: /home/cvspublic/httpd-proxy/module-2.0/proxy_http.c,v
> retrieving revision 1.79
> diff -u -r1.79 proxy_http.c
> --- proxy_http.c 2001/07/16 17:54:38 1.79
> +++ proxy_http.c 2001/07/17 16:07:32
> @@ -782,18 +782,20 @@
>
> /* read the body, pass it to the output filters */
> while (ap_get_brigade(rp->input_filters, bb, AP_MODE_BLOCKING,
> &readbyte
> s) == APR_SUCCESS) {
> - if (APR_BUCKET_IS_EOS(APR_BRIGADE_LAST(bb))) {
> + apr_bucket *b = APR_BRIGADE_LAST(bb);
> + if (APR_BUCKET_IS_EOS(b)) {
> + APR_BUCKET_REMOVE(b);
> e = apr_bucket_flush_create();
> APR_BRIGADE_INSERT_TAIL(bb, e);
> ap_pass_brigade(r->output_filters, bb);
Hum....
I may be mistaken, but the end-of-stream bucket still needs to be
there. This patch makes certain that there will *never* be an EOS
bucket.
Feel free to correct me on this.
Victor
--
Victor J. Orlikowski | The Wall is Down, But the Threat Remains!
==================================================================
v.j.orlikowski@gte.net | orlikowski@apache.org | vjo@us.ibm.com
Re: Nasty chunking bug (in MSIE?) when using ap_rwrite()/ap_rflush()
Posted by Ian Holsman <ia...@cnet.com>.
Ian Holsman wrote:
> TOKILEY@aol.com wrote:
>
>> In a message dated 01-07-16 18:16:37 EDT, Ian Holsman wrote...
>>
>>> On 25 Jun 2001 12:13:51 -0400, Bill Stoddard wrote:
>>> > I have a module that calls ap_rwrite() followed by ap_rflush().
>>> Content length is not
>>> > provided so Apache 2.0 chunks the response.
>>> > > Here is what happens...
>>> > I call ap_rwrite() to write a x75 len byte stream. All the
>>> correct headers are built, and
>>> > the content is buffered by the OLD_WRITE filter. Then I call
>>> ap_rflush() which causes the
>>> > headers to be sent on the wire along with the first chunk (x75
>>> length).
>>
>> My
>>
>>> handler is done
>>> > at this point and returns control to Apache. The next thing Apache
>>
>> sends
>>
>>> on the wire is
>>> > the 0 byte chunk header to indicate that the response is done.
>>> > > The problem: IE chokes when it receives just the 0 byte chunk
>>> header in
>>
>> a
>>
>>> packet.
>>> > > Thoughts?
>>> > > Bill
>>>
>>>
>>> hi Bill, I was wondering if you got anywhere with this
>>> I'm seeing this with mod_proxy at the moment.
>>>
>>> my inital thought was to put some logic in the chunking filter
>>> just not to send the 0 byte chunk out.. can you see anything barfing
>>> on this?
>>>
>>> what do you think?
>>>
>>> --
>>> Ian Holsman IanH@cnet.com
>>> Performance Measurement & Analysis
>>> CNET Networks - (415) 364-8608
>>>
>>
>> You have to send the 0 byte chunk... but that's not all there is to
>> it. The 'Entity headers' and then the final
>> 'blank line' (CR/LF) is what actually ENDS the chunked transfer.
>>
>> You have to 'hang tough' on the client side on a 'Keep-Alive'
>> and only use the final blank CR/LF as an EOD signal. You
>> can't bail when the '0' chunk length shows up or you could
>> end up leaving the Entity headers + final blank CR/LF in
>> the input buffer and think these chars are the start of the next
>> response when you get back to reading your
>> input buffers on the NIC on next Keep-Alive.
>>
>> Whoever/whatever is 'adding' the '0' byte must also be
>> adding A) The CR/LF that follows the 0 byte hex string
>> B) All of the Entity Headers + CR/LF ending each one
>> C) The final 'blank' CR/LF that ends the chunking.
>>
>> If any of the above is missing maybe that is why
>> MSIE is 'choking'.
>>
>
> MSIE is choking due to a '0' byte happening in the middle of the
> response being sent back.
>
> I think this is due to the proxy code sending a EOS bucket after the
> subrequest. (it doesn't happen with non-proxied
> requests), hopefully it will be easy to find.
>
>>
>>
>> I would find it hard to belive that a modern browser would
>> screw up a chunking byte length appearing all by itself
>> at the top of a new packet ( Even though that's highly
>> unusual... usually it's right at the end of the data in
>> the same final data packet ).
>>
>> Kevin Kiley
>>
>
> ..Ian
this seems to fix the problem...
can someone more familiar with brigades/buckets review it
Index: proxy_http.c
===================================================================
RCS file: /home/cvspublic/httpd-proxy/module-2.0/proxy_http.c,v
retrieving revision 1.79
diff -u -r1.79 proxy_http.c
--- proxy_http.c 2001/07/16 17:54:38 1.79
+++ proxy_http.c 2001/07/17 16:07:32
@@ -782,18 +782,20 @@
/* read the body, pass it to the output filters */
while (ap_get_brigade(rp->input_filters, bb, AP_MODE_BLOCKING,
&readbyte
s) == APR_SUCCESS) {
- if (APR_BUCKET_IS_EOS(APR_BRIGADE_LAST(bb))) {
+ apr_bucket *b = APR_BRIGADE_LAST(bb);
+ if (APR_BUCKET_IS_EOS(b)) {
+ APR_BUCKET_REMOVE(b);
e = apr_bucket_flush_create();
APR_BRIGADE_INSERT_TAIL(bb, e);
ap_pass_brigade(r->output_filters, bb);
Re: Nasty chunking bug (in MSIE?) when using ap_rwrite()/ap_rflush()
Posted by Ian Holsman <ia...@cnet.com>.
Ian Holsman wrote:
> TOKILEY@aol.com wrote:
>
>> In a message dated 01-07-16 18:16:37 EDT, Ian Holsman wrote...
>>
>>> On 25 Jun 2001 12:13:51 -0400, Bill Stoddard wrote:
>>> > I have a module that calls ap_rwrite() followed by ap_rflush().
>>> Content length is not
>>> > provided so Apache 2.0 chunks the response.
>>> > > Here is what happens...
>>> > I call ap_rwrite() to write a x75 len byte stream. All the
>>> correct headers are built, and
>>> > the content is buffered by the OLD_WRITE filter. Then I call
>>> ap_rflush() which causes the
>>> > headers to be sent on the wire along with the first chunk (x75
>>> length).
>>
>> My
>>
>>> handler is done
>>> > at this point and returns control to Apache. The next thing Apache
>>
>> sends
>>
>>> on the wire is
>>> > the 0 byte chunk header to indicate that the response is done.
>>> > > The problem: IE chokes when it receives just the 0 byte chunk
>>> header in
>>
>> a
>>
>>> packet.
>>> > > Thoughts?
>>> > > Bill
>>>
>>>
>>> hi Bill, I was wondering if you got anywhere with this
>>> I'm seeing this with mod_proxy at the moment.
>>>
>>> my inital thought was to put some logic in the chunking filter
>>> just not to send the 0 byte chunk out.. can you see anything barfing
>>> on this?
>>>
>>> what do you think?
>>>
>>> --
>>> Ian Holsman IanH@cnet.com
>>> Performance Measurement & Analysis
>>> CNET Networks - (415) 364-8608
>>>
>>
>> You have to send the 0 byte chunk... but that's not all there is to
>> it. The 'Entity headers' and then the final
>> 'blank line' (CR/LF) is what actually ENDS the chunked transfer.
>>
>> You have to 'hang tough' on the client side on a 'Keep-Alive'
>> and only use the final blank CR/LF as an EOD signal. You
>> can't bail when the '0' chunk length shows up or you could
>> end up leaving the Entity headers + final blank CR/LF in
>> the input buffer and think these chars are the start of the next
>> response when you get back to reading your
>> input buffers on the NIC on next Keep-Alive.
>>
>> Whoever/whatever is 'adding' the '0' byte must also be
>> adding A) The CR/LF that follows the 0 byte hex string
>> B) All of the Entity Headers + CR/LF ending each one
>> C) The final 'blank' CR/LF that ends the chunking.
>>
>> If any of the above is missing maybe that is why
>> MSIE is 'choking'.
>>
>
> MSIE is choking due to a '0' byte happening in the middle of the
> response being sent back.
>
> I think this is due to the proxy code sending a EOS bucket after the
> subrequest. (it doesn't happen with non-proxied
> requests), hopefully it will be easy to find.
>
>>
>>
>> I would find it hard to belive that a modern browser would
>> screw up a chunking byte length appearing all by itself
>> at the top of a new packet ( Even though that's highly
>> unusual... usually it's right at the end of the data in
>> the same final data packet ).
>>
>> Kevin Kiley
>>
>
> ..Ian
this seems to fix the problem...
can someone more familiar with brigades/buckets review it
Index: proxy_http.c
===================================================================
RCS file: /home/cvspublic/httpd-proxy/module-2.0/proxy_http.c,v
retrieving revision 1.79
diff -u -r1.79 proxy_http.c
--- proxy_http.c 2001/07/16 17:54:38 1.79
+++ proxy_http.c 2001/07/17 16:07:32
@@ -782,18 +782,20 @@
/* read the body, pass it to the output filters */
while (ap_get_brigade(rp->input_filters, bb, AP_MODE_BLOCKING,
&readbyte
s) == APR_SUCCESS) {
- if (APR_BUCKET_IS_EOS(APR_BRIGADE_LAST(bb))) {
+ apr_bucket *b = APR_BRIGADE_LAST(bb);
+ if (APR_BUCKET_IS_EOS(b)) {
+ APR_BUCKET_REMOVE(b);
e = apr_bucket_flush_create();
APR_BRIGADE_INSERT_TAIL(bb, e);
ap_pass_brigade(r->output_filters, bb);
Re: Nasty chunking bug (in MSIE?) when using ap_rwrite()/ap_rflush()
Posted by Chuck Murcko <ch...@topsail.org>.
Also, I don't think we update Content-Length: in the chunked trailers
yet.
Chuck
On Monday, July 16, 2001, at 10:58 PM, Chuck Murcko wrote:
>
> On Monday, July 16, 2001, at 09:56 PM, Ian Holsman wrote:
>
>>
>> MSIE is choking due to a '0' byte happening in the middle of the
>> response being sent back.
>>
>
> Mmm, I think the "0" byte is coming from the dechunk filter. AFAICS the
> proxy correctly looks for eos only, and the dechunk filter is supposed
> to strip eos from its input and append its own at the end of the
> chunked data so we have one that's meaningful after dechunk, trailers,
> etc.
>
Chuck Murcko
Topsail Group
http://www.topsail.org/
>
Re: Nasty chunking bug (in MSIE?) when using ap_rwrite()/ap_rflush()
Posted by Chuck Murcko <ch...@topsail.org>.
On Monday, July 16, 2001, at 09:56 PM, Ian Holsman wrote:
>
> MSIE is choking due to a '0' byte happening in the middle of the
> response being sent back.
>
Mmm, I think the "0" byte is coming from the dechunk filter. AFAICS the
proxy correctly looks for eos only, and the dechunk filter is supposed
to strip eos from its input and append its own at the end of the chunked
data so we have one that's meaningful after dechunk, trailers, etc.
Chuck Murcko
Topsail Group
http://www.topsail.org/