You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by bu...@apache.org on 2009/01/15 14:19:33 UTC
DO NOT REPLY [Bug 46538] New: ETag must differ between compressed
and uncompressed resource versions
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
Summary: ETag must differ between compressed and uncompressed
resource versions
Product: Tomcat 6
Version: 6.0.18
Platform: PC
OS/Version: Windows XP
Status: NEW
Severity: normal
Priority: P2
Component: Catalina
AssignedTo: dev@tomcat.apache.org
ReportedBy: oliver.schoett@capgemini-sdm.com
The Apache folks are about to fix the problem that ETags are the same for
compressed and uncompressed versions of a resource:
https://issues.apache.org/bugzilla/show_bug.cgi?id=39727
Tomcat 6.0.18 suffers from the same problem.
The effect is that if a caching proxy holds a gzipped version of a resource and
is asked by a client for an unzipped version, it requests one from the server
with the ETag of the cached version. The server sees that the ETag of the
version it would send out is the same as that of the version the cache already
holds and tells the cache that its version is OK (response status code 304).
In the case of a Squid cache, this results in a gzipped version to be sent to
the client, and this breaks in IE6 and IE7 when they are configured to use the
HTTP 1.0 protocol.
Squid has been provided with a work-around option for this problem:
http://www.squid-cache.org/Versions/v2/2.6/cfgman/broken_vary_encoding.html
but we should not rely on caches world-wide to provide a work-around for a
Tomcat bug.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
--- Comment #1 from Oliver Schoett <ol...@capgemini-sdm.com> 2009-01-29 03:42:56 PST ---
Created an attachment (id=23190)
--> (https://issues.apache.org/bugzilla/attachment.cgi?id=23190)
Patch to correct ETags and Vary headers for compression
Here is a patch that corrects the ETag and Vary behaviour:
- ETags differ for gzipped and ungzipped output
- Vary: Accept-Encoding ist sent whenever a gzipped version is available
The latter change makes it possible for users of differently capable browsers
to receive gzipped and ungzipped reponses through the same proxy cache.
Previously, an ungzipped cached version would be delivered also to
compression-capable browsers, because the cache could not know there was a
gzipped version available.
This patch will be put in production on an e-commerce website shortly.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
--- Comment #2 from Oliver Schoett <ol...@capgemini-sdm.com> 2009-01-29 03:56:52 PST ---
Here is the new behaviour when a resource is fetched without and with
compression enabled:
curl -v -HPragma: http://tomcat6.example.com/web3/js/hbx.js
> GET /web3/js/hbx.js HTTP/1.1
> User-Agent: curl/7.16.3 (i686-pc-cygwin) libcurl/7.16.3 OpenSSL/0.9.8i zlib/1.2.3 libssh2/0.15-CVS
> Host: tomcat6.example.com
> Accept: */*
< Server: Apache-Coyote/1.1
< Expires: Thu, 29 Jan 2009 13:27:07 GMT
< ETag: W/"15453-1233218608000"
< Last-Modified: Thu, 29 Jan 2009 08:43:28 GMT
< Vary: Accept-Encoding
< Content-Type: text/javascript
< Content-Length: 15453
< Date: Thu, 29 Jan 2009 11:27:07 GMT
curl -v -HPragma: --compressed http://tomcat6.example.com/web3/js/hbx.js | wc
-c
> GET /web3/js/hbx.js HTTP/1.1
> User-Agent: curl/7.16.3 (i686-pc-cygwin) libcurl/7.16.3 OpenSSL/0.9.8i zlib/1.2.3 libssh2/0.15-CVS
> Host: tomcat6.example.com
> Accept: */*
> Accept-Encoding: deflate, gzip
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Expires: Thu, 29 Jan 2009 13:26:48 GMT
< ETag: W/"15453-1233218608000-gz"
< Last-Modified: Thu, 29 Jan 2009 08:43:28 GMT
< Vary: Accept-Encoding
< Content-Type: text/javascript
< Transfer-Encoding: chunked
< Content-Encoding: gzip
< Date: Thu, 29 Jan 2009 11:26:48 GMT
Note that the first response also has a Vary header, and the second response
has a different ETag.
When this patch is employed, the work-around configured by default in Squid
caches (broken_vary_encoding) is no longer necessary. If you want to avoid
this work-around for your server, you might configure a server string that does
NOT start with "Apache".
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
--- Comment #5 from Remy Maucherat <re...@apache.org> 2009-04-16 13:57:58 PST ---
I disagree with this. Regardless on what happens with the transport, the entity
does not change once it is decoded.
-1 for this "fix".
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
--- Comment #17 from Mark Thomas <ma...@apache.org> 2009-09-10 22:43:14 BST ---
Since ETag handling is wholly within the DefaultServlet, just add an option to
that servlet. You can use the DefaultServlet's readOnly option as a template.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
--- Comment #13 from Mark Thomas <ma...@apache.org> 2009-04-29 06:01:47 PST ---
IE7 and IE8 - no joy with T-E
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
--- Comment #8 from Mark Thomas <ma...@apache.org> 2009-04-17 01:44:38 PST ---
I've reverted the fix from trunk and withdrawn the backport proposal as whilst
it fixed this issue, it introduced others.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
--- Comment #6 from Mark Thomas <ma...@apache.org> 2009-04-16 14:16:56 PST ---
The I suggest you read section 14.19 of RFC 2616 that makes it quite clear
ETags are per variant not per resource.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
--- Comment #10 from Remy Maucherat <re...@apache.org> 2009-04-17 08:17:37 PST ---
Yes, that's my point, the only solution I see in Tomcat 6 about this is an
option to remove the etag if compression is active for the request.
And about your spec quoting, it is great to adhere to specs and stuff, but it
might be that clients apparently only really support content-encoding, which is
not supposed to be used for on-the-fly compression (but is, since I am very not
sure about support for transfer-encoding which is the proper way to do that;
originally, I had planned all sorts of filters which would be added according
to the T-E header, but in the end, the only thing which was workable then was a
hardcoded gzip output filter which used the content-encoding header). You have
to do things which work ...
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
--- Comment #11 from Mark Thomas <ma...@apache.org> 2009-04-29 03:00:51 PST ---
The current state of T-E support in the browsers is:
- Opera advertises T-E support, works with T-E
- Moziila doesn't advertise T-E support, works with T-E
- IE doesn't advertise T-E support, doesn't work with T-E
My reading of the C-E discussion above is that any solution is a hack that will
have an issue somewhere. T-E is the right solution. Moving from the current
status quo is as likely or more likely to cause issues compared to the current
behaviour which while wrong, is at least understood. We could provide a handful
of options to allow users to configure the various hacks but this would add a
lot of code (and possibly complexity) to the critical path.
I would like to use T-E by default and fallback to C-E if T-E is not supported.
However, the patchy browser support means that another set of options would be
required to give folks a reasonable chance of configuring a 'good' behaviour
for most clients.
My inclination is to mark this issue as WONTFIX with the longer term plan being
implementing T-E and switching to T-E once the browser support is reasonable.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
Oliver Schoett <ol...@capgemini-sdm.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Attachment #23190|0 |1
is obsolete| |
--- Comment #16 from Oliver Schoett <ol...@capgemini-sdm.com> 2009-09-10 07:17:08 PDT ---
Created an attachment (id=24245)
Disable sending and interpreting ETags (needs to be made into an option)
Not fixing this bug makes it impossible to enable gzip compression on public
web sites, because IE6 users behind Squid 2.6 and 2.7 proxies will receive
broken content: IE6 by default does not allow compression behind a proxy, but
Squid 2.6+ will deliver gzipped content that it already has in the cache, and
which is not accepted by IE.
Squid has implemented the option broken_vary_encoding to work around this,
which by default is enabled for servers whose header begins with "Apache".
However, this option is buggy
(http://www.squid-cache.org/bugs/show_bug.cgi?id=2574), and Tomcat should not
require work-arounds by others for its broken behavior.
Thus, an option is needed to disable ETags to make public sites work reliably.
What needs to be done is contained in the patch, which disables sending and
interpreting ETags. This patch (against 6.0.18) has been used successfully in
production since February on a German e-commerce site (90 Mill. PV/month).
There is no performance impact, because 304 responses are still generated
according to the "If-Modified-Since" logic. Unfortunately, I do not know Tomcat
well enough to make this a configurable option.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
--- Comment #7 from Remy Maucherat <re...@apache.org> 2009-04-16 14:35:15 PST ---
Well, that does not sound very smart (and I had read that on the httpd bug,
sigh ...). But overall, I do think the patch is bad (see status file).
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
--- Comment #9 from Oliver Schoett <ol...@capgemini-sdm.com> 2009-04-17 07:59:02 PST ---
Yes, sorry, the patch is indeed not sufficient. It fixes the "sending" side of
the problem in that it sends out ETags that conform to the spec. However, I
now understand that we also need to fix the "receiving" side of the problem,
that is, deal with the ETags wet get back in If-Match, If-None-Match and
If-Range header fields and make appropriate responses.
This is much harder to fix, as currently the decision "can I compress or not"
is made at a completely different point from the base ETag calculation, and yet
we must know the set of ETags of the variants we are capable of producing to
handle the If-... Requests involving ETags.
Since my client needed a fix urgently (IE users behind proxy caches did receive
unusable JavaScript files), I have removed ETag generation and handling
completely from his production Tomcats (can supply patch if there is interest).
This way, the logic for "304 Not Modified" responses relies entirely on
If-Modified-Since, which works well enough if you keep the date stamps between
identical copies of a resource on different servers of a farm in sync.
--
BTW, the part of RFC2616 that makes it most clear that ETags are per variant is
in Section 13.6; for example:
If an entity tag was assigned to a cached representation,
the forwarded request SHOULD be conditional and include
the entity tags in an If-None-Match header field from all
its cache entries for the resource. This conveys to the
server the set of entities currently held by the cache,
so that if any one of these entities matches the requested
entity, the server can use the ETag header field in its
304 (Not Modified) response to tell the cache which entry
is appropriate.
This algorithm makes no sense unless ETags are per variant. Unfortunately, the
section that defines ETags (14.19) says nothing about this.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
--- Comment #14 from Remy Maucherat <re...@apache.org> 2009-04-29 10:16:59 PST ---
Maybe something could be done when the client advertises the T-E, and drop to
C-E if it does not ?
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
--- Comment #12 from Remy Maucherat <re...@apache.org> 2009-04-29 05:39:05 PST ---
I think I used mostly IE when I tried it back then. Did you test with IE 7 and
8 ?
I agree with this kind of browser support, it is still not doable to use T-E :(
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
Mark Thomas <ma...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |WONTFIX
--- Comment #15 from Mark Thomas <ma...@apache.org> 2009-09-09 18:00:36 BST ---
As suggested in comment 11 I am going to resolve this as WONTFIX.
My reasons are:
- any change to use T-E is as likely or more likely to cause breakage
- patchy browser support means another handful of options would be required to
give sys admins a reasonable chance of configuring a working configuration
compatible with users and their combination of proxies and/or caches
- I believe the complexity this would add to the critical path isn't worth the
benefit
In my view the tipping point will be when IE supports T-E whether or not it
advertises support for it. At that point I would be all for switching to the
spec compliant way of doing compression.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
--- Comment #3 from Oliver Schoett <ol...@capgemini-sdm.com> 2009-01-29 09:05:04 PST ---
Warning: the patch I submitted does not work well in connection with the Akamai
CDN.
First, the Akamai edge servers transparently decompress content without
changing the ETag (so that compressed and uncompressed versions are sent with
the same Etag).
Second, the Akamai servers treat responses with Vary: Accept-Encoding but
without Content-Encoding header as uncacheable (ESConfigGuide-Customer, p. 54,
Note: TTL and the Vary Header). My patch triggers this in the case of
uncompressed responses (due to missing client capability) that the server would
be willing to compress.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 46538] ETag must differ between compressed and
uncompressed resource versions
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46538
--- Comment #4 from Mark Thomas <ma...@apache.org> 2009-04-16 12:34:10 PST ---
Thanks for the patch. I have applied a modified version of it to trunk that
also extended it to the NIO and APR connectors.
The extended patch has been proposed for 6.0.x
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org