You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Jeremie Miller <je...@netins.net> on 1998/09/18 16:52:15 UTC
HTTP Compression in Mozilla
I don't know if this has been pointed out yet, but I remember there being
some discussion about this awhile back, so some of you might find it
interesting:
http://www.mozilla.org/projects/apache/gzip/
Excerpt from page:
"This project aims to improve real and perceived web browsing performance
by having the server send compressed HTML files to the browser, and having
the browser uncompress before displaying. Assuming fast enough processors
on most machines these days, the user should end up seeing the document
sooner this way than sending uncompressed HTML. Also, since a majority of
network traffic these days is HTTP traffic, compressing all HTML sent via
HTTP should recover a significant amount of wasted network bandwidth. "
Jer
Re: HTTP Compression in Mozilla
Posted by Ben Laurie <be...@algroup.co.uk>.
Marc Slemko wrote:
>
> On Sat, 19 Sep 1998, Ben Laurie wrote:
>
> > >
> > > On Unix.
> >
> > On NT! Try it!
>
> Erm... are you sure?
>
> First, it doesn't even support it properly on Unix.
>
> eg. http://www.worldgate.com/~marcs/compressed.html.gz the compressed IMG
> won't display, if you view the image in a new window it will, then when
> you go back to the old one it will display inline until you reload when it
> will break again. That is 4.05 on Linux and 4.06 on FreeBSD.
>
> Are you sure it does on NT? What happens if you try to view the above
> page? jwz has stated several times that it only works on unix, although
> perhaps it works on NT if you have gzip installed?
The page seems to work - but the compressed image doesn't.
Just goes to show you shouldn't believe everything you read, even if jwz
wrote it.
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
and Technical Director|Email: ben@algroup.co.uk |
A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
WE'RE RECRUITING! http://www.aldigital.co.uk/
Re: HTTP Compression in Mozilla
Posted by Chris Tacy <ch...@enginered.com>.
page is viewed.
uncompressed image is fine.
compressed image is broken.
interesting though... if i "view image" it is still broken. but if i
enter full URL (http://www.worldgate.com/~marcs/apache_anim.gif.gz) it's
viewable. then going back to the page it's there until reload (very much
as in the Unix behavior reported.
hmmm....
-c
--
#################################################
chris tacy president and co-founder
fire engine red http://www.enginered.com/
Re: HTTP Compression in Mozilla
Posted by Rasmus Lerdorf <ra...@lerdorf.on.ca>.
> But do you see the page at all? If so, then it has to be working, unless
> you are viewing compressed.html. The logs say you are getting
> compressed.html.gz:
>
> siren.enginered.com - - [18/Sep/1998:18:30:16 -0600] "GET /~marcs/compressed.html.gz HTTP/1.0" 200 117
> siren.enginered.com - - [18/Sep/1998:18:30:17 -0600] "GET /~marcs/apache_anim.gif HTTP/1.0" 200 15634
> siren.enginered.com - - [18/Sep/1998:18:30:17 -0600] "GET /~marcs/apache_anim.gif.gz HTTP/1.0" 200 10705
>
> So it must be able to do something with gzipped files.
Netscape 4.06 on Win98 shows the page fine although the compressed image
is broken. IE and Opera are unable to show the page.
-Rasmus
Re: HTTP Compression in Mozilla
Posted by Marc Slemko <ma...@znep.com>.
On Fri, 18 Sep 1998, Chris Tacy wrote:
> NT4 sp3
> gzip and gunzip
> 4.5 pr2
>
> doesn't work - image is broken (even if viewed in new window).
But do you see the page at all? If so, then it has to be working, unless
you are viewing compressed.html. The logs say you are getting
compressed.html.gz:
siren.enginered.com - - [18/Sep/1998:18:30:16 -0600] "GET /~marcs/compressed.html.gz HTTP/1.0" 200 117
siren.enginered.com - - [18/Sep/1998:18:30:17 -0600] "GET /~marcs/apache_anim.gif HTTP/1.0" 200 15634
siren.enginered.com - - [18/Sep/1998:18:30:17 -0600] "GET /~marcs/apache_anim.gif.gz HTTP/1.0" 200 10705
So it must be able to do something with gzipped files.
Does it do the same without gzip installed?
Re: HTTP Compression in Mozilla
Posted by Chris Tacy <ch...@enginered.com>.
NT4 sp3
gzip and gunzip
4.5 pr2
doesn't work - image is broken (even if viewed in new window).
-c
Marc Slemko wrote:
>
> On Sat, 19 Sep 1998, Ben Laurie wrote:
>
> > >
> > > On Unix.
> >
> > On NT! Try it!
>
> Erm... are you sure?
>
> First, it doesn't even support it properly on Unix.
>
> eg. http://www.worldgate.com/~marcs/compressed.html.gz the compressed IMG
> won't display, if you view the image in a new window it will, then when
> you go back to the old one it will display inline until you reload when it
> will break again. That is 4.05 on Linux and 4.06 on FreeBSD.
>
> Are you sure it does on NT? What happens if you try to view the above
> page? jwz has stated several times that it only works on unix, although
> perhaps it works on NT if you have gzip installed?
--
#################################################
chris tacy president and co-founder
fire engine red http://www.enginered.com/
Hotmail and Apache
Posted by Chris Tacy <ch...@enginered.com>.
probably well known already, but...
Checking host: www.hotmail.com port 80
Operating system: * rare Xylan, or firewalled NetBSD
Web server software: Apache/1.2.1
--
#################################################
chris tacy president and co-founder
fire engine red http://www.enginered.com/
Re: HTTP Compression in Mozilla
Posted by Dean Gaudet <dg...@arctic.org>.
On Mon, 21 Sep 1998, Marc Slemko wrote:
> On Mon, 21 Sep 1998, Dean Gaudet wrote:
>
> > Dig through the archives in the last year, you'll find a thread that I
> > forwarded from mozilla-general. In that you'll find a few notes as to how
> > apache is broken in this area... take a look at the headers being sent
> > back, make sure they're correct before you blame NS.
>
> Wasn't that just related to negotiation?
I don't recall. Hence my suggestion to look at the headers/archives.
Dean
Re: HTTP Compression in Mozilla
Posted by Marc Slemko <ma...@worldgate.com>.
On Mon, 21 Sep 1998, Dean Gaudet wrote:
> Dig through the archives in the last year, you'll find a thread that I
> forwarded from mozilla-general. In that you'll find a few notes as to how
> apache is broken in this area... take a look at the headers being sent
> back, make sure they're correct before you blame NS.
Wasn't that just related to negotiation?
>
> Dean
>
> On Fri, 18 Sep 1998, Marc Slemko wrote:
>
> > On Sat, 19 Sep 1998, Ben Laurie wrote:
> >
> > > >
> > > > On Unix.
> > >
> > > On NT! Try it!
> >
> > Erm... are you sure?
> >
> > First, it doesn't even support it properly on Unix.
> >
> > eg. http://www.worldgate.com/~marcs/compressed.html.gz the compressed IMG
> > won't display, if you view the image in a new window it will, then when
> > you go back to the old one it will display inline until you reload when it
> > will break again. That is 4.05 on Linux and 4.06 on FreeBSD.
> >
> > Are you sure it does on NT? What happens if you try to view the above
> > page? jwz has stated several times that it only works on unix, although
> > perhaps it works on NT if you have gzip installed?
> >
> >
>
Re: HTTP Compression in Mozilla
Posted by Dean Gaudet <dg...@arctic.org>.
Dig through the archives in the last year, you'll find a thread that I
forwarded from mozilla-general. In that you'll find a few notes as to how
apache is broken in this area... take a look at the headers being sent
back, make sure they're correct before you blame NS.
Dean
On Fri, 18 Sep 1998, Marc Slemko wrote:
> On Sat, 19 Sep 1998, Ben Laurie wrote:
>
> > >
> > > On Unix.
> >
> > On NT! Try it!
>
> Erm... are you sure?
>
> First, it doesn't even support it properly on Unix.
>
> eg. http://www.worldgate.com/~marcs/compressed.html.gz the compressed IMG
> won't display, if you view the image in a new window it will, then when
> you go back to the old one it will display inline until you reload when it
> will break again. That is 4.05 on Linux and 4.06 on FreeBSD.
>
> Are you sure it does on NT? What happens if you try to view the above
> page? jwz has stated several times that it only works on unix, although
> perhaps it works on NT if you have gzip installed?
>
>
Re: HTTP Compression in Mozilla
Posted by Marc Slemko <ma...@znep.com>.
On Sat, 19 Sep 1998, Ben Laurie wrote:
> >
> > On Unix.
>
> On NT! Try it!
Erm... are you sure?
First, it doesn't even support it properly on Unix.
eg. http://www.worldgate.com/~marcs/compressed.html.gz the compressed IMG
won't display, if you view the image in a new window it will, then when
you go back to the old one it will display inline until you reload when it
will break again. That is 4.05 on Linux and 4.06 on FreeBSD.
Are you sure it does on NT? What happens if you try to view the above
page? jwz has stated several times that it only works on unix, although
perhaps it works on NT if you have gzip installed?
Re: HTTP Compression in Mozilla
Posted by Ben Laurie <be...@algroup.co.uk>.
Marc Slemko wrote:
>
> On Fri, 18 Sep 1998, Ben Laurie wrote:
>
> > Jeremie Miller wrote:
> > >
> > > I don't know if this has been pointed out yet, but I remember there being
> > > some discussion about this awhile back, so some of you might find it
> > > interesting:
> > >
> > > http://www.mozilla.org/projects/apache/gzip/
> > >
> > > Excerpt from page:
> > >
> > > "This project aims to improve real and perceived web browsing performance
> > > by having the server send compressed HTML files to the browser, and having
> > > the browser uncompress before displaying. Assuming fast enough processors
> > > on most machines these days, the user should end up seeing the document
> > > sooner this way than sending uncompressed HTML. Also, since a majority of
> > > network traffic these days is HTTP traffic, compressing all HTML sent via
> > > HTTP should recover a significant amount of wasted network bandwidth. "
> >
> > But, but... Netscape 4 already does this!
>
> On Unix.
On NT! Try it!
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
and Technical Director|Email: ben@algroup.co.uk |
A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
WE'RE RECRUITING! http://www.aldigital.co.uk/
Re: HTTP Compression in Mozilla
Posted by Jim Gettys <jg...@pa.dec.com>.
> Sender: new-httpd-owner@apache.org
> From: Marc Slemko <ma...@worldgate.com>
> Date: Fri, 18 Sep 1998 15:25:04 -0600 (MDT)
> To: new-httpd@apache.org
> Subject: Re: HTTP Compression in Mozilla
> -----
> On Fri, 18 Sep 1998, Jim Gettys wrote:
>
> > > On Unix.
> >
> > And I think the implementation may have been in NS4 UNIX by forking a
> > piped process and running zcat on the file, rather than internal using
> > zlib; this is non-trivial overhead....
>
> Exactly.
>
> >
> > Zlib makes this pretty easy to implement in a client; Henrik implemented
> > transfer encoding in an afternoon in libwww.
>
> Yup.
>
> > Note that I'm not claiming that a server implementation, except one that
> > was so stupid as to do the compression every time a document was asked
> > for, would be so simple for Apache right now; I had sympathy for Roy's
>
> The simple implementation is pretty trivial, as long as you don't object
> to having to compress them yourself manually each time you change the
> content or just have a cron job doing it.
I think there are people who would be willing to live with this class
of hack solution for now, until V2 of Apache makes the implementation
issues that Roy raised easier.
The other solution while Apache is based directly on a file systeem costs
extra system calls to make sure that any compressed form is more recent
than the uncompressed form. For most UNIX systems, stat's are pretty
cheap these days; unfortunately, for some other operating systems, the
equivalent to stat is expensive. This approach might be worth it for
some people, as it conserves link bandwidth which may be a bigger
issue than hits/second.
A remaining question is whether being able to compress dynamically generated
content on the fly is worth it (e.g. cgi script output). I don't
have any data on how expensive compression is at the moment, so hesitate
to make any prognostications on its value.
Note, btw, that deflate is preferable to gzip, as it gets rid of the
(useless to the web) file system data at the beginning of the document.
If I had the HTTP/1.1 spec to do again, I think I'd argue against
including gzip at all (the algorithms are the same; the only difference
is whether the file system headers are included, as I understand it).
- Jim
Re: HTTP Compression in Mozilla
Posted by Marc Slemko <ma...@worldgate.com>.
On Fri, 18 Sep 1998, Jim Gettys wrote:
> > On Unix.
>
> And I think the implementation may have been in NS4 UNIX by forking a
> piped process and running zcat on the file, rather than internal using
> zlib; this is non-trivial overhead....
Exactly.
>
> Zlib makes this pretty easy to implement in a client; Henrik implemented
> transfer encoding in an afternoon in libwww.
Yup.
> Note that I'm not claiming that a server implementation, except one that
> was so stupid as to do the compression every time a document was asked
> for, would be so simple for Apache right now; I had sympathy for Roy's
The simple implementation is pretty trivial, as long as you don't object
to having to compress them yourself manually each time you change the
content or just have a cron job doing it.
Re: HTTP Compression in Mozilla
Posted by Jim Gettys <jg...@pa.dec.com>.
> Sender: new-httpd-owner@apache.org
> From: Paul Sutton <pa...@c2.net>
> Date: Mon, 21 Sep 1998 15:21:50 +0100 (BST)
> To: new-httpd@apache.org
> Subject: Re: HTTP Compression in Mozilla
> -----
> On Mon, 21 Sep 1998, Jim Gettys wrote:
> > The sad thing is that zlib is set up properly for streaming data through
> > it (just as you want for progressive HTML rendering). And zlib is
> > freely available (I think it is under a GPL).
>
> It's totally unencumbered:
>
> "Permission is granted to anyone to use this software for any purpose,
> including commercial applications, and to alter it and redistribute it
> freely, subject to the following restrictions: [three reasonable
> restrictions]"
>
> (from README in zlib 1.0.4, which may be a bit old)
>
> Paul
Ah, an MIT/X style copyright; I like those much better than the GPL; truly
unencumbered. But then again, I drafted the original X copyright in the
first place, so this isn't too surprising :-).
- Jim
Re: HTTP Compression in Mozilla
Posted by Paul Sutton <pa...@c2.net>.
On Mon, 21 Sep 1998, Jim Gettys wrote:
> The sad thing is that zlib is set up properly for streaming data through
> it (just as you want for progressive HTML rendering). And zlib is
> freely available (I think it is under a GPL).
It's totally unencumbered:
"Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions: [three reasonable
restrictions]"
(from README in zlib 1.0.4, which may be a bit old)
Paul
Re: HTTP Compression in Mozilla
Posted by Jim Gettys <jg...@pa.dec.com>.
I've been corresponding with Eric Bina; it certainly sounded like
Mozilla intends to support TE, from our mail.
- Jim
Re: HTTP Compression in Mozilla
Posted by Dmitry Khrustalev <di...@bog.msu.su>.
>
> Neat! Given that it isn't implemented in many browsers right now,
> I'm gratified it is saving you that much bandwidth....
>
> I don't suppose you'd be interested in implementing TE as well are you?
> Henrik's libwww and clients can be used to test it.
> Claim is that Mozilla supports it as well, according to Netscape's
> implementation report (we haven't looked...)...
As I understand, mozilla does not send TE yet. Other than that, this
is just a matter of finding implementation to test against.
>
> BTW, deflate is a bit more efficient that gzip as it does include the
> file headers.
Deflate is 18 bytes less. However IE4 does not support deflate. IE5
does.
-Dima
Re: HTTP Compression in Mozilla
Posted by Jim Gettys <jg...@pa.dec.com>.
> Sender: new-httpd-owner@apache.org
> From: Dmitry Khrustalev <di...@bog.msu.su>
> Date: Tue, 22 Sep 1998 17:31:27 +0400 (MSD)
> To: Jim Gettys <jg...@pa.dec.com>
> Cc: new-httpd@apache.org, Marc Slemko <ma...@znep.com>
> Subject: Re: HTTP Compression in Mozilla
> -----
> well, i'm running httpd with streaming gzip compression on www.rbc.ru now
>
> total traffic for 17 hours is 3.87 GB
> compression saved 0.645 GB.
>
> responses are selected for compression using this algorithm:
>
> r->proto_num >= HTTP_VERSION(1,1)) &&
> r->status == HTTP_OK &&
> !ap_table_get(r->headers_in, "Via") &&
> ae = ap_table_get(r->headers_in, "Accept-Encoding") &&
> ap_find_token(r->pool, ae, "gzip") &&
> ce == NULL &&
> r->byterange == 0 &&
> ct != NULL &&
> !strncasecmp(ct, "text/", 5))
>
> where ct is content-type of response and ce is content-encoding.
>
> -Dima
>
>
Neat! Given that it isn't implemented in many browsers right now,
I'm gratified it is saving you that much bandwidth....
I don't suppose you'd be interested in implementing TE as well are you?
Henrik's libwww and clients can be used to test it.
Claim is that Mozilla supports it as well, according to Netscape's
implementation report (we haven't looked...)...
BTW, deflate is a bit more efficient that gzip as it does include the
file headers.
- Jim
Re: HTTP Compression in Mozilla
Posted by Dmitry Khrustalev <di...@bog.msu.su>.
well, i'm running httpd with streaming gzip compression on www.rbc.ru now
total traffic for 17 hours is 3.87 GB
compression saved 0.645 GB.
responses are selected for compression using this algorithm:
r->proto_num >= HTTP_VERSION(1,1)) &&
r->status == HTTP_OK &&
!ap_table_get(r->headers_in, "Via") &&
ae = ap_table_get(r->headers_in, "Accept-Encoding") &&
ap_find_token(r->pool, ae, "gzip") &&
ce == NULL &&
r->byterange == 0 &&
ct != NULL &&
!strncasecmp(ct, "text/", 5))
where ct is content-type of response and ce is content-encoding.
-Dima
On Mon, 21 Sep 1998, Jim Gettys wrote:
> The sad thing is that zlib is set up properly for streaming data through
> it (just as you want for progressive HTML rendering). And zlib is
> freely available (I think it is under a GPL).
>
> This is why Henrik found it so easy to implement transfer coding
> in libwww; it really was "an afternoon hack".
> - Jim
>
>
Re: HTTP Compression in Mozilla
Posted by Jim Gettys <jg...@pa.dec.com>.
The sad thing is that zlib is set up properly for streaming data through
it (just as you want for progressive HTML rendering). And zlib is
freely available (I think it is under a GPL).
This is why Henrik found it so easy to implement transfer coding
in libwww; it really was "an afternoon hack".
- Jim
Re: HTTP Compression in Mozilla
Posted by Dmitry Khrustalev <di...@bog.msu.su>.
On Fri, 18 Sep 1998, Jim Gettys wrote:
> > >
> > > But, but... Netscape 4 already does this!
> >
> > On Unix.
>
> And I think the implementation may have been in NS4 UNIX by forking a
> piped process and running zcat on the file, rather than internal using
> zlib; this is non-trivial overhead....
>
Reality is even worse: if you have several compressed frames it will
fork several zcat processes and lock up.
-Dima
Re: HTTP Compression in Mozilla
Posted by Jim Gettys <jg...@pa.dec.com>.
> Sender: new-httpd-owner@apache.org
> From: Marc Slemko <ma...@znep.com>
> Date: Fri, 18 Sep 1998 13:18:49 -0700 (PDT)
> To: new-httpd@apache.org
> Subject: Re: HTTP Compression in Mozilla
> -----
> On Fri, 18 Sep 1998, Ben Laurie wrote:
>
> > Jeremie Miller wrote:
> > >
> > > I don't know if this has been pointed out yet, but I remember there being
> > > some discussion about this awhile back, so some of you might find it
> > > interesting:
> > >
> > > http://www.mozilla.org/projects/apache/gzip/
> > >
> > > Excerpt from page:
> > >
> > > "This project aims to improve real and perceived web browsing performance
> > > by having the server send compressed HTML files to the browser, and having
> > > the browser uncompress before displaying. Assuming fast enough processors
> > > on most machines these days, the user should end up seeing the document
> > > sooner this way than sending uncompressed HTML. Also, since a majority of
> > > network traffic these days is HTTP traffic, compressing all HTML sent via
> > > HTTP should recover a significant amount of wasted network bandwidth. "
> >
> > But, but... Netscape 4 already does this!
>
> On Unix.
And I think the implementation may have been in NS4 UNIX by forking a
piped process and running zcat on the file, rather than internal using
zlib; this is non-trivial overhead....
Zlib makes this pretty easy to implement in a client; Henrik implemented
transfer encoding in an afternoon in libwww.
Note that I'm not claiming that a server implementation, except one that
was so stupid as to do the compression every time a document was asked
for, would be so simple for Apache right now; I had sympathy for Roy's
arguments against implementing transfer coding right now, even if I didn't
agree with him. In Jigsaw, the implementation was also very easy, but
it already has an object cache on disk, and therefore finds it easy
to do the compression when a document is changed and save the result.
- Jim
Re: HTTP Compression in Mozilla
Posted by Marc Slemko <ma...@znep.com>.
On Fri, 18 Sep 1998, Ben Laurie wrote:
> Jeremie Miller wrote:
> >
> > I don't know if this has been pointed out yet, but I remember there being
> > some discussion about this awhile back, so some of you might find it
> > interesting:
> >
> > http://www.mozilla.org/projects/apache/gzip/
> >
> > Excerpt from page:
> >
> > "This project aims to improve real and perceived web browsing performance
> > by having the server send compressed HTML files to the browser, and having
> > the browser uncompress before displaying. Assuming fast enough processors
> > on most machines these days, the user should end up seeing the document
> > sooner this way than sending uncompressed HTML. Also, since a majority of
> > network traffic these days is HTTP traffic, compressing all HTML sent via
> > HTTP should recover a significant amount of wasted network bandwidth. "
>
> But, but... Netscape 4 already does this!
On Unix.
Re: HTTP Compression in Mozilla
Posted by Ben Laurie <be...@algroup.co.uk>.
Jeremie Miller wrote:
>
> I don't know if this has been pointed out yet, but I remember there being
> some discussion about this awhile back, so some of you might find it
> interesting:
>
> http://www.mozilla.org/projects/apache/gzip/
>
> Excerpt from page:
>
> "This project aims to improve real and perceived web browsing performance
> by having the server send compressed HTML files to the browser, and having
> the browser uncompress before displaying. Assuming fast enough processors
> on most machines these days, the user should end up seeing the document
> sooner this way than sending uncompressed HTML. Also, since a majority of
> network traffic these days is HTTP traffic, compressing all HTML sent via
> HTTP should recover a significant amount of wasted network bandwidth. "
But, but... Netscape 4 already does this!
Cheers,
Ben.
--
Ben Laurie |Phone: +44 (181) 735 0686| Apache Group member
Freelance Consultant |Fax: +44 (181) 735 0689|http://www.apache.org/
and Technical Director|Email: ben@algroup.co.uk |
A.L. Digital Ltd, |Apache-SSL author http://www.apache-ssl.org/
London, England. |"Apache: TDG" http://www.ora.com/catalog/apache/
WE'RE RECRUITING! http://www.aldigital.co.uk/
Re: HTTP Compression in Mozilla
Posted by Jim Gettys <jg...@pa.dec.com>.
Yup. That is why I raised the issue about implementing it a while back.
Our performance work convinced me not to drop TE from HTTP/1.1 last year,
at a time when I was looking for things not getting implemented that
should be removed...
Our experience can be found at:
http://www.w3.org/Protocols/HTTP/Performance/#Compression
- Jim