You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Justin Erenkrantz <je...@apache.org> on 2002/06/06 18:20:29 UTC
[PATCH] Fix neon inflate bug
With this patch to neon and HEAD of httpd-2.0 and mod_deflate turned
on, I can check out my own website under SVN control with gzip
compression.
So, I think most of the bugs are ironed out now. =)
* src/ne_compress.c (do_inflate): If we have nothing left to inflate,
do not try to re-execute the loop and inflate again. We would get a
Z_BUF_ERROR in this case.
Please let me know if you have any questions. -- justin
Index: src/ne_compress.c
===================================================================
RCS file: /home/cvs/neon/src/ne_compress.c,v
retrieving revision 1.16
diff -u -r1.16 ne_compress.c
--- src/ne_compress.c 2002/06/03 10:37:02 1.16
+++ src/ne_compress.c 2002/06/06 18:04:01
@@ -200,7 +200,8 @@
/* pass on the inflated data */
ctx->reader(ctx->userdata, ctx->outbuf, ctx->zstr.total_out);
- } while (ret == Z_OK && ctx->zstr.avail_out == 0);
+ } while (ret == Z_OK && ctx->zstr.avail_out == 0 &&
+ ctx->zstr.avail_in != 0);
if (ret == Z_STREAM_END) {
NE_DEBUG(NE_DBG_HTTP, "compress: end of data stream, remaining %d.\n",
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH] Fix neon inflate bug
Posted by Ben Collins <bc...@debian.org>.
On Thu, Jun 06, 2002 at 11:53:09AM -0700, Justin Erenkrantz wrote:
> On Thu, Jun 06, 2002 at 02:30:02PM -0400, Ben Collins wrote:
> > Probably something internal to apache/dav/deflate, or maybe even
> > something weird in mod_dav_svn.
> >
> > Of course, something could have been fixed in httpd-2.0 head since last
> > week that I don't know about.
>
> Yeah, I went through earlier this week and squashed some buglets in
> mod_deflate.
>
> If you can, please try with HEAD of httpd-2.0. -- justin
Oh, you bet. Deflate is pertinent to my evil plan!
Thanks.
--
Debian - http://www.debian.org/
Linux 1394 - http://linux1394.sourceforge.net/
Subversion - http://subversion.tigris.org/
Deqo - http://www.deqo.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH] Fix neon inflate bug
Posted by Justin Erenkrantz <je...@apache.org>.
On Thu, Jun 06, 2002 at 02:30:02PM -0400, Ben Collins wrote:
> Probably something internal to apache/dav/deflate, or maybe even
> something weird in mod_dav_svn.
>
> Of course, something could have been fixed in httpd-2.0 head since last
> week that I don't know about.
Yeah, I went through earlier this week and squashed some buglets in
mod_deflate.
If you can, please try with HEAD of httpd-2.0. -- justin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH] Fix neon inflate bug
Posted by Ben Collins <bc...@debian.org>.
On Thu, Jun 06, 2002 at 11:20:29AM -0700, Justin Erenkrantz wrote:
> With this patch to neon and HEAD of httpd-2.0 and mod_deflate turned
> on, I can check out my own website under SVN control with gzip
> compression.
>
> So, I think most of the bugs are ironed out now. =)
>
> * src/ne_compress.c (do_inflate): If we have nothing left to inflate,
> do not try to re-execute the loop and inflate again. We would get a
> Z_BUF_ERROR in this case.
>
> Please let me know if you have any questions. -- justin
Do the test on a > 1meg file from the repo. I tried this last week, and
we still get the bug where the compressed data is corrupted. This
doesn't involve the SVN client, since we get the problem even with
something like:
wget --header "Accept-Encoding: gzip" http://foo/svn/BigFile
zcat BigFile > blah
(error)
Probably something internal to apache/dav/deflate, or maybe even
something weird in mod_dav_svn.
Of course, something could have been fixed in httpd-2.0 head since last
week that I don't know about.
Ben
--
Debian - http://www.debian.org/
Linux 1394 - http://linux1394.sourceforge.net/
Subversion - http://subversion.tigris.org/
Deqo - http://www.deqo.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [neon] [PATCH] Fix neon inflate bug
Posted by Joe Orton <jo...@manyfish.co.uk>.
On Sat, Jun 08, 2002 at 10:06:17AM -0700, Justin Erenkrantz wrote:
> On Sat, Jun 08, 2002 at 01:48:05PM +0100, Joe Orton wrote:
> > On Thu, Jun 06, 2002 at 11:20:29AM -0700, Justin Erenkrantz wrote:
> > > With this patch to neon and HEAD of httpd-2.0 and mod_deflate turned
> > > on, I can check out my own website under SVN control with gzip
> > > compression.
> > >
> > > So, I think most of the bugs are ironed out now. =)
> > >
> > > * src/ne_compress.c (do_inflate): If we have nothing left to inflate,
> > > do not try to re-execute the loop and inflate again. We would get a
> > > Z_BUF_ERROR in this case.
> >
> > Hmmm, I'm not sure it's quite this simple... this only works because
> > avail_in is guaranteed to be > 0 at the end of the stream because of the
> > unconsumed gzip trailer.
>
> Neon would try to inflate when avail_in was 0. If there is nothing
> left to inflate, why call it again? How is that valid? We're not
> yet guaranteed to be at the end of the stream because we're just
> inflating chunks.
Okay, that does make more sense (my interpretation of the inflate()
documentation in zlib.h was that inflate() should be called whilst it
continues to provide inflated output).
Thanks for tracking this down!
Regards,
joe
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [neon] [PATCH] Fix neon inflate bug
Posted by Justin Erenkrantz <je...@apache.org>.
On Sat, Jun 08, 2002 at 01:48:05PM +0100, Joe Orton wrote:
> On Thu, Jun 06, 2002 at 11:20:29AM -0700, Justin Erenkrantz wrote:
> > With this patch to neon and HEAD of httpd-2.0 and mod_deflate turned
> > on, I can check out my own website under SVN control with gzip
> > compression.
> >
> > So, I think most of the bugs are ironed out now. =)
> >
> > * src/ne_compress.c (do_inflate): If we have nothing left to inflate,
> > do not try to re-execute the loop and inflate again. We would get a
> > Z_BUF_ERROR in this case.
>
> Hmmm, I'm not sure it's quite this simple... this only works because
> avail_in is guaranteed to be > 0 at the end of the stream because of the
> unconsumed gzip trailer.
Neon would try to inflate when avail_in was 0. If there is nothing
left to inflate, why call it again? How is that valid? We're not
yet guaranteed to be at the end of the stream because we're just
inflating chunks.
> When does this Z_BUF_ERROR happen - can you send me a document to
> reproduce it? zlib.h says that inflate() does either or both of consume
> input/provide output, not that it needs to consume input every time it
> is called; in my tests inflate() gets called when avail_in is zero
> without any problems.
I was checking out this file from my SVN repository:
http://www.erenkrantz.com/ArtsCore/VanGogh-NightCafe.jpg
Well, whenever avail_in was zero and neon called inflate() with
zlib-1.1.4, it caused Z_BUF_ERROR. And, as stated in the zlib
FAQ (question #6):
6. deflate() or inflate() returns Z_BUF_ERROR
Before making the call, make sure that avail_in and avail_out are
not zero. When setting the parameter flush equal to Z_FINISH,
also make sure that avail_out is big enough to allow processing
all pending input.
I know Ben Collins should be trying mod_deflate/neon with SVN, so he
may also be able to reproduce this case. -- justin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [neon] [PATCH] Fix neon inflate bug
Posted by Joe Orton <jo...@manyfish.co.uk>.
On Thu, Jun 06, 2002 at 11:20:29AM -0700, Justin Erenkrantz wrote:
> With this patch to neon and HEAD of httpd-2.0 and mod_deflate turned
> on, I can check out my own website under SVN control with gzip
> compression.
>
> So, I think most of the bugs are ironed out now. =)
>
> * src/ne_compress.c (do_inflate): If we have nothing left to inflate,
> do not try to re-execute the loop and inflate again. We would get a
> Z_BUF_ERROR in this case.
Hmmm, I'm not sure it's quite this simple... this only works because
avail_in is guaranteed to be > 0 at the end of the stream because of the
unconsumed gzip trailer.
When does this Z_BUF_ERROR happen - can you send me a document to
reproduce it? zlib.h says that inflate() does either or both of consume
input/provide output, not that it needs to consume input every time it
is called; in my tests inflate() gets called when avail_in is zero
without any problems.
Regards,
joe
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org