corrupt gzip output when compression=on in server.xml

[Christopher Taylor <cc...@gmail.com>]

Dear all,

to save bandwidth, we're running tomcat (7.0.14) with compression
enabled in server.xml. While it works great, we encountered a response
that consistently returned a corrupted gzip stream in the response
(inside the zip file attached to this message to make sure that it's
not transparently decompressed by accident). While browsers (tested on
FF and Chrome) had no problem displaying the content, our Java client
that uses java.util.GZIPInputStream (on oracle jre1.6.0_26-b03) errors
out with an IOException complaining about a corrupt gzip trailer:

java.io.IOException: Corrupt GZIP trailer
                at
java.util.zip.GZIPInputStream.readTrailer(GZIPInputStream.java:203)
                at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:94)
                at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
                at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
                at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
                at java.io.InputStreamReader.read(InputStreamReader.java:167)
                at java.io.BufferedReader.fill(BufferedReader.java:136)
                at java.io.BufferedReader.read(BufferedReader.java:157)
                [...]

running gunzip on the response data also yields errors:

$ gunzip -tv < broken_out.gzip
gzip: broken_out.gzip: invalid compressed data--crc error
gzip: broken_out.gzip: invalid compressed data--length error

and uncompressing the data with gzip yields an extra '>' char at the
end of the data.

What is interesting about the broken response is that the compressed
data contains an uncompressed fragment at the end, followed by some
more binary data. I looked around and found that
org.apache.coyote.http11.filters.FlushableGZIPOutputStream seems to
force flushing the stream by switching the compression method to
NO_COMPRESSION and back again to DEFAULT_COMPRESSION -- I wonder if
this is the cause?

Google yielded two bugs that seem to be related:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4813885 deals with
flushing Gzip streams and
https://issues.apache.org/bugzilla/show_bug.cgi?id=48738 seems to
track the introduction of FlushableGZIPOutputStream into tomcat.

I'd be grateful for any pointers,
  --Chris

Re: corrupt gzip output when compression=on in server.xml

[Christopher Schultz <ch...@christopherschultz.net>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Chris,

On 6/30/2011 11:05 AM, Christopher Taylor wrote:
> While it works great, we encountered a response
> that consistently returned a corrupted gzip stream in the response
> (inside the zip file attached to this message to make sure that it's
> not transparently decompressed by accident).

Could you pare this down to a super-simple test case? If so, log a new
issue in Bugzilla and attach that test case as a complete WAR file so we
can reproduce it.

> What is interesting about the broken response is that the compressed
> data contains an uncompressed fragment at the end, followed by some
> more binary data. I looked around and found that
> org.apache.coyote.http11.filters.FlushableGZIPOutputStream seems to
> force flushing the stream by switching the compression method to
> NO_COMPRESSION and back again to DEFAULT_COMPRESSION -- I wonder if
> this is the cause?

That certainly sounds plausible.

Have you tried other TC versions where this same file works?

> Google yielded two bugs that seem to be related:
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4813885 deals with
> flushing Gzip streams and
> https://issues.apache.org/bugzilla/show_bug.cgi?id=48738 seems to
> track the introduction of FlushableGZIPOutputStream into tomcat.

I wonder if you are hitting some super-special edge case, like you are
flushing when the buffer has nothing in it (due to the buffer having
just been flushed naturally) or something like that.

I have to imagine that flushing a GZIPOutputStream is tricky business
because you want gzip to complete it's current block (so it can actually
flush) but also be prepared to continue the stream afterward.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4M4FEACgkQ9CaO5/Lv0PDpqACfesDaPACMyl5nD8bJBd2RZHJA
dG4AnAhyct79YRvFXaSOAzl9+aZGRvGQ
=MrFd
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org