You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Dean Gaudet <dg...@arctic.org> on 1998/07/14 21:17:35 UTC
request that we support TE
While it's a good thing to support, it's not a walk in the park. This
comes down to needing a more generic form of caching, which is on the
plate for 2.0. Someone could do a mod_te for 1.3 with only moderate
effort in my opinion, it wouldn't be perfect, but it could do the job.
Dean
---------- Forwarded message ----------
Date: Tue, 14 Jul 1998 07:51:43 -0700
From: Jim Gettys <jg...@pa.dec.com>
To: Dean Gaudet <dg...@arctic.org>
Subject: TE in Apache....
As you know, transfer coding in RFC2068 is somewhat broken; the draft
standard revision includes a new TE header field to fix the problem
(involving caching in proxies, where a compressed document might
get served to an unsuspecting old client).
Our Sigcomm paper shows it has significant good performance properties;
Netscape 5, for example, has implemented and tested it (Jigsaw has
an implementation to test against, as does Henrik Frystyk's libwww library).
Is anyone working on this for Apache? If there were one thing
I could pick to implement in Apache this instant, I'd seriously think about
this one....
- Jim Gettys
---------- Forwarded message ----------
Date: Tue, 14 Jul 1998 12:03:09 -0700
From: Jim Gettys <jg...@pa.dec.com>
To: Dean Gaudet <dg...@arctic.org>
Subject: Re: TE in Apache....
See: http://www.w3.org/Protocols/HTTP/Forum/Reports/ for the implementation
reports... The claim is that NS 5 have implemented and tested the TE
header...
I haven't looked very carefully at the Mozilla code so I don't know first
hand; I've been going by the implementation report that Gagan Sakensa
submitted.
As to implementation, I'd probably do the opposite: convert all reasonable
documents to .gz the first time I saw them, and then uncompress on
the fly (compression is more expensive than decompression). Of course,
this is presuming most stuff goes over the wire compressed, which it won't
initially; the point is that "optimal" depends on whether you presume
most compressable content goes across the wire compressed or not in
the first place... :-).
- Jim
Re: request that we support TE
Posted by Dean Gaudet <dg...@arctic.org>.
There's even more reasons than that one... having to do with server-side
negotiation and crap like that.
Dean
On Tue, 14 Jul 1998, Alexei Kosut wrote:
> On Tue, 14 Jul 1998, Dean Gaudet wrote:
>
> > Yes it's friendlier. They're related, but different.
> >
> > mod_negotiation does annoying things like responding "Content-Encoding:
> > gzip" when the request didn't specify "Accept-Encoding: gzip". It should
> > respond "Content-Type: application/gzip", or octet-stream or something.
> >
> > I forget exactly why it's annoying at the moment. There's more details in
> > the mozilla-general archives.
>
> 1. Browser knows nothing about content-encodings. It requests foo.html.gz
> 2. Apache returns "Content-Type: text/html / Content-Encoding: gzip"
> 3. Browser things "oh! an HTML file. I know how to display this."
> 4. User gets a screen full of garbage.
>
> That's a bad thing.
>
> -- Alexei Kosut <ak...@stanford.edu> <http://www.stanford.edu/~akosut/>
> Stanford University, Class of 2001 * Apache <http://www.apache.org> *
>
>
>
Re: request that we support TE
Posted by Dirk-Willem van Gulik <di...@jrc.it>.
On Tue, 14 Jul 1998, Alexei Kosut wrote:
> On Tue, 14 Jul 1998, Dean Gaudet wrote:
>
> > Yes it's friendlier. They're related, but different.
> >
> > mod_negotiation does annoying things like responding "Content-Encoding:
> > gzip" when the request didn't specify "Accept-Encoding: gzip". It should
> > respond "Content-Type: application/gzip", or octet-stream or something.
> >
> > I forget exactly why it's annoying at the moment. There's more details in
> > the mozilla-general archives.
>
> 1. Browser knows nothing about content-encodings. It requests foo.html.gz
But it has not claimed to know about it; i.e. there was no Accept-Encoding
in the request. Though we are now seeing some request come in from some
custom proxies and some mac browser which does specify an encoding. So at
some point this will become an issue.
> 2. Apache returns "Content-Type: text/html / Content-Encoding: gzip"
> 3. Browser things "oh! an HTML file. I know how to display this."
> 4. User gets a screen full of garbage.
>
> That's a bad thing.
Yup. As is Accept-Type: Barf/Borf; */* The world of negotion is evil.
Dw.
> -- Alexei Kosut <ak...@stanford.edu> <http://www.stanford.edu/~akosut/>
> Stanford University, Class of 2001 * Apache <http://www.apache.org> *
>
>
>
Re: request that we support TE
Posted by Alexei Kosut <ak...@leland.Stanford.EDU>.
On Tue, 14 Jul 1998, Dean Gaudet wrote:
> Yes it's friendlier. They're related, but different.
>
> mod_negotiation does annoying things like responding "Content-Encoding:
> gzip" when the request didn't specify "Accept-Encoding: gzip". It should
> respond "Content-Type: application/gzip", or octet-stream or something.
>
> I forget exactly why it's annoying at the moment. There's more details in
> the mozilla-general archives.
1. Browser knows nothing about content-encodings. It requests foo.html.gz
2. Apache returns "Content-Type: text/html / Content-Encoding: gzip"
3. Browser things "oh! an HTML file. I know how to display this."
4. User gets a screen full of garbage.
That's a bad thing.
-- Alexei Kosut <ak...@stanford.edu> <http://www.stanford.edu/~akosut/>
Stanford University, Class of 2001 * Apache <http://www.apache.org> *
Re: request that we support TE
Posted by Dean Gaudet <dg...@arctic.org>.
On Tue, 14 Jul 1998, Alexei Kosut wrote:
> One thing, though, that I think would be easy to do would be to take the
> mod_mime and mod_negotiation code that exists already to deal with
> Content-Encoding and Accept-Encoding for gzip/compress files, and
> translate that into TE/Transfer-Encoding for 1.1 requests with a TE
> header. That way we could serve already-compressed files and pretend they
> were being compressed on the fly; T-E is friendlier to caches than C-E,
> no?
Yes it's friendlier. They're related, but different.
mod_negotiation does annoying things like responding "Content-Encoding:
gzip" when the request didn't specify "Accept-Encoding: gzip". It should
respond "Content-Type: application/gzip", or octet-stream or something.
I forget exactly why it's annoying at the moment. There's more details in
the mozilla-general archives.
Dean
Re: request that we support TE
Posted by Alexei Kosut <ak...@leland.Stanford.EDU>.
On Tue, 14 Jul 1998, Dean Gaudet wrote:
> > > Layered i/o helps if you want to gzip mod_include, for example.
> >
> > Yes. Or static files, too. And then they get cached (thinking about the
> > "magic cache" from the SF 2.0 discussions).
>
> Right, in 2.0 it's easy. In 1.3 it isn't. (Which I believe was what I
> said in the first post ;)
Yes, you did. It just took me three emails to figure out what you were
talking about, which is not, though very strongly related to,
implementation of the TE header, which is what the original email talked
about.
*shrug*
One thing, though, that I think would be easy to do would be to take the
mod_mime and mod_negotiation code that exists already to deal with
Content-Encoding and Accept-Encoding for gzip/compress files, and
translate that into TE/Transfer-Encoding for 1.1 requests with a TE
header. That way we could serve already-compressed files and pretend they
were being compressed on the fly; T-E is friendlier to caches than C-E,
no?
-- Alexei Kosut <ak...@stanford.edu> <http://www.stanford.edu/~akosut/>
Stanford University, Class of 2001 * Apache <http://www.apache.org> *
Re: request that we support TE
Posted by Dean Gaudet <dg...@arctic.org>.
On Tue, 14 Jul 1998, Alexei Kosut wrote:
> Hmm? I don't see why one would need a cache; the point of gzip/deflate is
> to save bandwidth, not to process the request faster (although not having
> to spend the CPU compressing a file for each request is a nice thing),
> right?
Right, that's the point.
Also the point of caching is that if it's not "point-and-click", if the
admin has to do anything more than set up a directory for httpd to
scribble on and tell it how much to scribble, the admin won't do it... the
web designers won't make the effort to compress anything, yadda. You have
to do it for them.
> > Layered i/o helps if you want to gzip mod_include, for example.
>
> Yes. Or static files, too. And then they get cached (thinking about the
> "magic cache" from the SF 2.0 discussions).
Right, in 2.0 it's easy. In 1.3 it isn't. (Which I believe was what I
said in the first post ;)
Dean
Re: request that we support TE
Posted by Alexei Kosut <ak...@leland.Stanford.EDU>.
On Tue, 14 Jul 1998, Dean Gaudet wrote:
> > > Yes, I'm saying that transfer-coding deflate (and gzip, but it isn't as
> > > efficient as deflate due to the unneeded file overhead) is a good thing...
> > > To do so in a routine way that is cache safe, you need to deal with TE.
> > > As I said, RFC2068 is buggy in this area, and TE is the fix.
> >
> > Ah. Okay, I get it now.
> >
> > Yes, I agree it would be good to support deflate. But I agree with Dean
> > that it's non-trivial doing it with the way Apache is currently set up
> > internally (although it makes a good case for layered i/o, Dean *grin*).
> > It could be added somewhat easily for static or already-coded files,
> > though, which is what I suspect Dean meant when he referred to a mod_te.
>
> mod_te could also maintain a cache directory of gzipped files.
> mod_mmap_static is a good example of how to override the underlying
> default handler. I was actually referring to building a cache -- that's
> about the only way this would be useful in my opinion.
Hmm? I don't see why one would need a cache; the point of gzip/deflate is
to save bandwidth, not to process the request faster (although not having
to spend the CPU compressing a file for each request is a nice thing),
right?
> Layered i/o helps if you want to gzip mod_include, for example.
Yes. Or static files, too. And then they get cached (thinking about the
"magic cache" from the SF 2.0 discussions).
-- Alexei Kosut <ak...@stanford.edu> <http://www.stanford.edu/~akosut/>
Stanford University, Class of 2001 * Apache <http://www.apache.org> *
Re: request that we support TE
Posted by Dean Gaudet <dg...@arctic.org>.
On Tue, 14 Jul 1998, Alexei Kosut wrote:
> On Tue, 14 Jul 1998, Jim Gettys wrote:
>
> > > Is there something here I'm missing? Is he saying we should also implement
> > > other transfer-codings, e.g., deflate? In that case, of course, TE would
> > > make sense to implement...
> >
> >
> > Yes, I'm saying that transfer-coding deflate (and gzip, but it isn't as
> > efficient as deflate due to the unneeded file overhead) is a good thing...
> > To do so in a routine way that is cache safe, you need to deal with TE.
> > As I said, RFC2068 is buggy in this area, and TE is the fix.
>
> Ah. Okay, I get it now.
>
> Yes, I agree it would be good to support deflate. But I agree with Dean
> that it's non-trivial doing it with the way Apache is currently set up
> internally (although it makes a good case for layered i/o, Dean *grin*).
> It could be added somewhat easily for static or already-coded files,
> though, which is what I suspect Dean meant when he referred to a mod_te.
mod_te could also maintain a cache directory of gzipped files.
mod_mmap_static is a good example of how to override the underlying
default handler. I was actually referring to building a cache -- that's
about the only way this would be useful in my opinion.
Layered i/o helps if you want to gzip mod_include, for example.
Dean
Re: request that we support TE
Posted by Alexei Kosut <ak...@leland.Stanford.EDU>.
On Tue, 14 Jul 1998, Jim Gettys wrote:
> > Is there something here I'm missing? Is he saying we should also implement
> > other transfer-codings, e.g., deflate? In that case, of course, TE would
> > make sense to implement...
>
>
> Yes, I'm saying that transfer-coding deflate (and gzip, but it isn't as
> efficient as deflate due to the unneeded file overhead) is a good thing...
> To do so in a routine way that is cache safe, you need to deal with TE.
> As I said, RFC2068 is buggy in this area, and TE is the fix.
Ah. Okay, I get it now.
Yes, I agree it would be good to support deflate. But I agree with Dean
that it's non-trivial doing it with the way Apache is currently set up
internally (although it makes a good case for layered i/o, Dean *grin*).
It could be added somewhat easily for static or already-coded files,
though, which is what I suspect Dean meant when he referred to a mod_te.
-- Alexei Kosut <ak...@stanford.edu> <http://www.stanford.edu/~akosut/>
Stanford University, Class of 2001 * Apache <http://www.apache.org> *
Re: request that we support TE
Posted by Jim Gettys <jg...@pa.dec.com>.
> Sender: new-httpd-owner@apache.org
> From: Alexei Kosut <ak...@leland.Stanford.EDU>
> Date: Tue, 14 Jul 1998 12:42:20 -0700 (PDT)
> To: new-httpd@apache.org
> Subject: Re: request that we support TE
> -----
> On Tue, 14 Jul 1998, Dean Gaudet wrote:
>
> > While it's a good thing to support, it's not a walk in the park. This
> > comes down to needing a more generic form of caching, which is on the
> > plate for 2.0. Someone could do a mod_te for 1.3 with only moderate
> > effort in my opinion, it wouldn't be perfect, but it could do the job.
>
> I'm confused. I pulled up a copy of draft-ietf-http-vt11-spec-rev-03, and
> read over the TE section. AFAIK, Apache doesn't need to do a thing in
> order to implement it, because we support use any transfer codings except
> identity and chunked (although I guess we *could* parse the header to
> determine if the client perfers chunked over identity for a request where
> we could serve either, i.e., entities with content-lengths, but that seems
> trivial).
>
> Is there something here I'm missing? Is he saying we should also implement
> other transfer-codings, e.g., deflate? In that case, of course, TE would
> make sense to implement...
Yes, I'm saying that transfer-coding deflate (and gzip, but it isn't as
efficient as deflate due to the unneeded file overhead) is a good thing...
To do so in a routine way that is cache safe, you need to deal with TE.
As I said, RFC2068 is buggy in this area, and TE is the fix.
- Jim
--
Jim Gettys
Digital Industry Standards and Consortia
Compaq Computer Corporation
Visting Scientist, World Wide Web Consortium, M.I.T.
http://www.w3.org/People/Gettys/
jg@w3.org, jg@pa.dec.com
Re: request that we support TE
Posted by Dean Gaudet <dg...@arctic.org>.
On Tue, 14 Jul 1998, Alexei Kosut wrote:
> Is there something here I'm missing? Is he saying we should also implement
> other transfer-codings, e.g., deflate? In that case, of course, TE would
> make sense to implement...
Yes.
Dean
Re: request that we support TE
Posted by Alexei Kosut <ak...@leland.Stanford.EDU>.
On Tue, 14 Jul 1998, Dean Gaudet wrote:
> While it's a good thing to support, it's not a walk in the park. This
> comes down to needing a more generic form of caching, which is on the
> plate for 2.0. Someone could do a mod_te for 1.3 with only moderate
> effort in my opinion, it wouldn't be perfect, but it could do the job.
I'm confused. I pulled up a copy of draft-ietf-http-vt11-spec-rev-03, and
read over the TE section. AFAIK, Apache doesn't need to do a thing in
order to implement it, because we support use any transfer codings except
identity and chunked (although I guess we *could* parse the header to
determine if the client perfers chunked over identity for a request where
we could serve either, i.e., entities with content-lengths, but that seems
trivial).
Is there something here I'm missing? Is he saying we should also implement
other transfer-codings, e.g., deflate? In that case, of course, TE would
make sense to implement...
> ---------- Forwarded message ----------
> Date: Tue, 14 Jul 1998 07:51:43 -0700
> From: Jim Gettys <jg...@pa.dec.com>
> To: Dean Gaudet <dg...@arctic.org>
> Subject: TE in Apache....
>
>
> As you know, transfer coding in RFC2068 is somewhat broken; the draft
> standard revision includes a new TE header field to fix the problem
> (involving caching in proxies, where a compressed document might
> get served to an unsuspecting old client).
>
> Our Sigcomm paper shows it has significant good performance properties;
> Netscape 5, for example, has implemented and tested it (Jigsaw has
> an implementation to test against, as does Henrik Frystyk's libwww library).
>
> Is anyone working on this for Apache? If there were one thing
> I could pick to implement in Apache this instant, I'd seriously think about
> this one....
> - Jim Gettys
>
>
>
>
> ---------- Forwarded message ----------
> Date: Tue, 14 Jul 1998 12:03:09 -0700
> From: Jim Gettys <jg...@pa.dec.com>
> To: Dean Gaudet <dg...@arctic.org>
> Subject: Re: TE in Apache....
>
>
> See: http://www.w3.org/Protocols/HTTP/Forum/Reports/ for the implementation
> reports... The claim is that NS 5 have implemented and tested the TE
> header...
>
> I haven't looked very carefully at the Mozilla code so I don't know first
> hand; I've been going by the implementation report that Gagan Sakensa
> submitted.
>
> As to implementation, I'd probably do the opposite: convert all reasonable
> documents to .gz the first time I saw them, and then uncompress on
> the fly (compression is more expensive than decompression). Of course,
> this is presuming most stuff goes over the wire compressed, which it won't
> initially; the point is that "optimal" depends on whether you presume
> most compressable content goes across the wire compressed or not in
> the first place... :-).
> - Jim
>
>
-- Alexei Kosut <ak...@stanford.edu> <http://www.stanford.edu/~akosut/>
Stanford University, Class of 2001 * Apache <http://www.apache.org> *