You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Steve Bardsley <SB...@rlwinc.com> on 2007/10/11 20:48:32 UTC
[users@httpd] resubmit: mod_deflate with chunked gzipped input
[Here's a cleaned up resubmittal hoping it's more digestible.]
Is mod_deflate.c unreliable for inflating gzipped and chunked input?
Issue Summary:
Apache 2.2.6 on Fedora Core 7 and Windows XP
I'm sending chunked, gzipped POSTs to apache and it works, but
occasionally loses the request body. After many hours of debug,
mod_deflate is suspect. Debug included using mod_ext_filter and
instrumenting mod_deflate with extra log messages.
After much searching of bugzilla and the mailing lists I found this
thread:
http://thread.gmane.org/gmane.comp.apache.user/61481/focus=61516
The last message in the thread hints at a problem:
"...should be fine for normal pages, but may be less than
optimal for things like streaming contents."
Debug Details:
- I'm using a simple PHP script to catch POSTed urlencoded form data.
- The client gzips the POST body and streams it to the server
(chunked).
- A single form variable contains a large set of data.
- Most of the time the form variable and its associated data is
received, but sometimes it is not.
- There is no failure indication in the log; using LogLevel debug.
- I instrumented mod_deflate.c with LOG messages.
- There is a "FIXME" comment in the code; line 953
http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/filters/mod_defla
te.c?view=annotate
- It *seems* I see the problem whenever this section of code is hit.
Steve Bardsley
--------------------------
RLW Inc
_________________________________________________________________________
Disclaimer: This message is intended only for the use of the noted recipient(s) and may contain information that is privileged, proprietary and confidential.
If you received this message in error, you must not, directly or indirectly, distribute, use, disclose, print, or copy any part of this message. Please notify the sender if you have received this message by mistake and erase this email and its contents from your system. Note that any views or opinions expressed in this message are solely those of the author and do not necessarily represent those of the company. Finally, note that email transmission cannot be guaranteed to be secure or error-free. Information could be intercepted, corrupted, lost, destroyed, received late or incomplete, or infected with a virus. Therefore, the sender does not accept liability for any errors in the content of this message which arise as a result of email transmission.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] resubmit: mod_deflate with chunked gzipped input
Posted by Steve Bardsley <SB...@rlwinc.com>.
> From: Nick Kew on Saturday, October 13, 2007 5:02 AM
>
> On 13 Oct 2007, at 05:43, Steve Bardsley wrote:
>
> > Guys, please ignore the previous patch. There's still a problem.
It's
> > possibly due to cleansing of debug messages. I'll be sure we get the
> > next one right. Sorry for the wasted bandwidth.
>
> Can you supply a testcase that'll reliably reproduce the problem?
>
> It's great that you're doing the work of fixing [whatever it is],
> but your patch stands a much higher chance of being reviewed and
> incorporated if we can see exactly what it fixes.
>
I'd like to provide a test case, but the failing request is sourced from
our embedded device, currently under development. We are working the
problem ourselves because it's not easy to simulate the request outside
the device. I understand the need to reproduce on your end. When/if we
can do that we will. Thanks.
Steve Bardsley
_________________________________________________________________________
Disclaimer: This message is intended only for the use of the noted recipient(s) and may contain information that is privileged, proprietary and confidential.
If you received this message in error, you must not, directly or indirectly, distribute, use, disclose, print, or copy any part of this message. Please notify the sender if you have received this message by mistake and erase this email and its contents from your system. Note that any views or opinions expressed in this message are solely those of the author and do not necessarily represent those of the company. Finally, note that email transmission cannot be guaranteed to be secure or error-free. Information could be intercepted, corrupted, lost, destroyed, received late or incomplete, or infected with a virus. Therefore, the sender does not accept liability for any errors in the content of this message which arise as a result of email transmission.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
[users@httpd] Apache Install problems
Posted by rjdani <rj...@centurytel.net>.
I'm trying to work my way through the "Beginning PHP5, Apache, MySql Web
Development" book i.e. a newbie. I'm having trouble installing Apache on my
HP Windows XP it has service pack 2 and is current with updates when I
unzipped the httpd-2.2.6-win32-src-r2.zip file I do not see the MSI file
mentioned in Appendix I of the book. Can you help me.
RJD
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] resubmit: mod_deflate with chunked gzipped input
Posted by Nick Kew <ni...@webthing.com>.
On 13 Oct 2007, at 05:43, Steve Bardsley wrote:
> Guys, please ignore the previous patch. There's still a problem. It's
> possibly due to cleansing of debug messages. I'll be sure we get the
> next one right. Sorry for the wasted bandwidth.
Can you supply a testcase that'll reliably reproduce the problem?
It's great that you're doing the work of fixing [whatever it is],
but your patch stands a much higher chance of being reviewed and
incorporated if we can see exactly what it fixes.
--
Nick Kew
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] resubmit: mod_deflate with chunked gzipped input
Posted by Steve Bardsley <SB...@rlwinc.com>.
> -----Original Message-----
> From: Steve Bardsley [mailto:SBardsley@rlwinc.com]
> Sent: Friday, October 12, 2007 6:23 PM
> > -----Original Message-----
> > From: Nick Kew [mailto:nick@webthing.com]
> > Sent: Friday, October 12, 2007 5:54 AM
> >
> > On Thu, 11 Oct 2007, William A. Rowe, Jr. wrote:
> > > Nick Kew wrote:
> > > > On Thu, 11 Oct 2007, Steve Bardsley wrote:
> > > >
> > > >> [Here's a cleaned up resubmittal hoping it's more digestible.]
> > > >>
> > > >> Is mod_deflate.c unreliable for inflating gzipped and chunked
> input?
> > > >
> > > > Chunked is irrelevant: the dechunking filter runs in font of
> > mod_deflate.
> > > > It's possible there's a bug, but we don't know it.
> > >
> > > UHM... Nick, you just fixed this bug yourself 2 weeks ago :)
> >
> > Erm, that was a bug in the inflate output filter. OP here spoke
> > of inflating incoming POSTed data. Different filter. Also unlikely
> > the same bug would manifest in a pull filter.
> >
>
> I've attached a patch for mod_deflate.c (version 581198 in SVN). Let
me
> know if you want a different format, but
> patch -p0 <mod_deflate_581198.patch
> should do the trick.
>
> We strongly suggest there is a problem handling the gzip footer while
> inflating input data. The original code does not seem to acquire the
> footer properly in all cases. The changes ensure the footer is
properly
> acquired.
>
> This patch has run for over 24 hours without error. The previous
> version would only run from 2-5 hours before errors occurred under
> identical conditions (loading, data, etc). Again, the errors *always*
> occurred coincident with the "FIXME" section of code (line 953).
>
Guys, please ignore the previous patch. There's still a problem. It's
possibly due to cleansing of debug messages. I'll be sure we get the
next one right. Sorry for the wasted bandwidth.
Regards,
Steve
_________________________________________________________________________
Disclaimer: This message is intended only for the use of the noted recipient(s) and may contain information that is privileged, proprietary and confidential.
If you received this message in error, you must not, directly or indirectly, distribute, use, disclose, print, or copy any part of this message. Please notify the sender if you have received this message by mistake and erase this email and its contents from your system. Note that any views or opinions expressed in this message are solely those of the author and do not necessarily represent those of the company. Finally, note that email transmission cannot be guaranteed to be secure or error-free. Information could be intercepted, corrupted, lost, destroyed, received late or incomplete, or infected with a virus. Therefore, the sender does not accept liability for any errors in the content of this message which arise as a result of email transmission.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
[users@httpd] RE: resubmit: mod_deflate with chunked gzipped input
Posted by Steve Bardsley <SB...@rlwinc.com>.
> -----Original Message-----
> From: Nick Kew [mailto:nick@webthing.com]
> Sent: Friday, October 12, 2007 5:54 AM
>
> On Thu, 11 Oct 2007, William A. Rowe, Jr. wrote:
> > Nick Kew wrote:
> > > On Thu, 11 Oct 2007, Steve Bardsley wrote:
> > >
> > >> [Here's a cleaned up resubmittal hoping it's more digestible.]
> > >>
> > >> Is mod_deflate.c unreliable for inflating gzipped and chunked
input?
> > >
> > > Chunked is irrelevant: the dechunking filter runs in font of
> mod_deflate.
> > > It's possible there's a bug, but we don't know it.
> >
> > UHM... Nick, you just fixed this bug yourself 2 weeks ago :)
>
> Erm, that was a bug in the inflate output filter. OP here spoke
> of inflating incoming POSTed data. Different filter. Also unlikely
> the same bug would manifest in a pull filter.
>
I've attached a patch for mod_deflate.c (version 581198 in SVN). Let me
know if you want a different format, but
patch -p0 <mod_deflate_581198.patch
should do the trick.
We strongly suggest there is a problem handling the gzip footer while
inflating input data. The original code does not seem to acquire the
footer properly in all cases. The changes ensure the footer is properly
acquired.
This patch has run for over 24 hours without error. The previous
version would only run from 2-5 hours before errors occurred under
identical conditions (loading, data, etc). Again, the errors *always*
occurred coincident with the "FIXME" section of code (line 953).
Regards,
Steve Bardsley
_________________________________________________________________________
Disclaimer: This message is intended only for the use of the noted recipient(s) and may contain information that is privileged, proprietary and confidential.
If you received this message in error, you must not, directly or indirectly, distribute, use, disclose, print, or copy any part of this message. Please notify the sender if you have received this message by mistake and erase this email and its contents from your system. Note that any views or opinions expressed in this message are solely those of the author and do not necessarily represent those of the company. Finally, note that email transmission cannot be guaranteed to be secure or error-free. Information could be intercepted, corrupted, lost, destroyed, received late or incomplete, or infected with a virus. Therefore, the sender does not accept liability for any errors in the content of this message which arise as a result of email transmission.
Re: [users@httpd] resubmit: mod_deflate with chunked gzipped input
Posted by Nick Kew <ni...@webthing.com>.
On Thu, 11 Oct 2007, William A. Rowe, Jr. wrote:
> Nick Kew wrote:
> > On Thu, 11 Oct 2007, Steve Bardsley wrote:
> >
> >> [Here's a cleaned up resubmittal hoping it's more digestible.]
> >>
> >> Is mod_deflate.c unreliable for inflating gzipped and chunked input?
> >
> > Chunked is irrelevant: the dechunking filter runs in font of mod_deflate.
> > It's possible there's a bug, but we don't know it.
>
> UHM... Nick, you just fixed this bug yourself 2 weeks ago :)
Erm, that was a bug in the inflate output filter. OP here spoke
of inflating incoming POSTed data. Different filter. Also unlikely
the same bug would manifest in a pull filter.
--
Nick Kew
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] resubmit: mod_deflate with chunked gzipped input
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
Nick Kew wrote:
> On Thu, 11 Oct 2007, Steve Bardsley wrote:
>
>> [Here's a cleaned up resubmittal hoping it's more digestible.]
>>
>> Is mod_deflate.c unreliable for inflating gzipped and chunked input?
>
> Chunked is irrelevant: the dechunking filter runs in font of mod_deflate.
> It's possible there's a bug, but we don't know it.
UHM... Nick, you just fixed this bug yourself 2 weeks ago :)
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] resubmit: mod_deflate with chunked gzipped input
Posted by Nick Kew <ni...@webthing.com>.
On Thu, 11 Oct 2007, Steve Bardsley wrote:
> [Here's a cleaned up resubmittal hoping it's more digestible.]
>
> Is mod_deflate.c unreliable for inflating gzipped and chunked input?
Chunked is irrelevant: the dechunking filter runs in font of mod_deflate.
It's possible there's a bug, but we don't know it.
> Debug Details:
> [chop]
Insert a mod_diagnostics filter before and after the inflate filter.
That should tell you if the byte counts in and out make sense,
and might yield other clues.
--
Nick Kew
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org