suggestion: strip comments from served html pages

[Tobias Skytte <to...@itservices.co.mz>]

Hi,

Many webpages include alot of comments in the html code. Sometimes this  
can me more than 50% of the file! At the same time the internet is getting  
more and more congested. Why not introduce and option to remove the  
comments from the served file? The person web-browsing any given site has  
absolitely no use for those comments anyway (unless he want to steal the  
code offcourse....).
Also, this option should be enabled by default, since many admins probably  
wont bother to enable it.

Best regards,
Tobias Skytte

----------------------------------------
IT Services(tm)
GETT Lda, C.P.106, Av. Eduardo Mondlane 757
Nampula, Mocambique
Tel: +258-(0)6-217002, Cell: +258-82-601-845
email: tobias@itservices.co.mz
www: www.itservices.co.mz

Re: suggestion: strip comments from served html pages

[André Malo <nd...@perlig.de>]

* "Tobias Skytte" <to...@itservices.co.mz> wrote:

> Many webpages include alot of comments in the html code. Sometimes this  
> can me more than 50% of the file! At the same time the internet is getting  
> more and more congested. Why not introduce and option to remove the  
> comments from the served file?

Because it makes more sense to strip them before putting them online. And what
is a comment?

<style type="text/css"><!--
   foo {bar}
--></style>

? Yes, if it's XHTML, No if it's HTML. Well, that is solvable. We'd just need
an XHTML parser with HTML specialities, tag soup recignizer etc. pp.

Still sounds good?

> Also, this option should be enabled by default, since many admins probably  
> wont bother to enable it.

You're actually suggesting to modify served content by default?! I'm
proposing to add your mail address as default for all related bugzilla
entries ;-)

nd
-- 
Gib' mal folgendes in die Kommandozeile ein (und einen Moment warten):

net send localhost "Buuuh!"
Na, erschreckt?                              -- Markus Becker in mpdsh

Re: suggestion: strip comments from served html pages

[Nick Kew <ni...@webthing.com>]

On Sun, 7 Nov 2004, [ISO-8859-15] Andr� Malo wrote:

> * Nick Kew <ni...@webthing.com> wrote:
>
> > BTW, the "what is a comment" problem is easier than it looks, as both
> > <script> and <style> are declared in HTML as having CDATA content.
> > That makes it trivial to distinguish them from "inert" comments.
>
> but in xhtml it's PCDATA, which makes them real xml comments...

So long as we're in web-browser-compatible land, we can parse XHTML with
an HTML parser that knows about CDATA.  And when we move out of it,
we're also leaving commented <script> and <style> contents behind.

I'll grant there are other pathological edge-cases due to the ways
people abuse markup.  That's one very good reason none of the modules
I mentioned defaults to stripping comments:-)

-- 
Nick Kew

Re: suggestion: strip comments from served html pages

[André Malo <nd...@perlig.de>]

* Nick Kew <ni...@webthing.com> wrote:

> BTW, the "what is a comment" problem is easier than it looks, as both
> <script> and <style> are declared in HTML as having CDATA content.
> That makes it trivial to distinguish them from "inert" comments.

but in xhtml it's PCDATA, which makes them real xml comments...

nd
-- 
Winnetous Erbe: <http://pub.perlig.de/books.html#apache2>

Re: suggestion: strip comments from served html pages

[Nick Kew <ni...@webthing.com>]

On Sun, 7 Nov 2004, Cliff Woolley wrote:

> > > Why not introduce and option to remove the
> > > comments from the served file?
> >
> > You have that option from mod_xmlns, mod_proxy_html or mod_publisher,
> > to name but three.  Along with caveats about why it's not necessarily
> > a good idea.
>
> Ultimately I suspect that the biggest problem (assuming that the "what's a
> comment and what isn't" problem is solvable) is that this kind of parsing
> takes a disproportionately large amount of CPU time with respect to the
> amount of network bandwidth it saves.

Agreed.  The reason those modules offer it is that they're already
parsing the markup, so there's no additional overhead.  mod_include
could offer it for the same reason.

OTOH, the fact that we have - and people use - mod_deflate demonstrates
that there is a demand for byte count reduction.  mod_deflate uses a
great deal more CPU, and achieves a great deal more savings, than any
of the above.

>	  I mean really, how much bandwidth
> from *html* are we talking about?  Compared to images?  If you start
> parsing the html, you lose any ability to do zero-copy and the like, not
> to mention the fact that the CPU has to examine every single byte of the
> input and process and/or copy it.  Yuck.

I implemented that for a Client who is serving slow devices (mobile phones
at 9600 baud and with a lot of latency per connection).  That involved
compressing both HTML and a lot of other contents, including images
(the details are content-negotiated).  My opinion was that the gains
from manipulating HTML were not worth the extra hassle over mod_deflate,
but the Client - one of the best-known names in the business - took
the view that it was worthwhile.

BTW, the "what is a comment" problem is easier than it looks, as both
<script> and <style> are declared in HTML as having CDATA content.
That makes it trivial to distinguish them from "inert" comments.

-- 
Nick Kew

Re: suggestion: strip comments from served html pages

[Cliff Woolley <jw...@virginia.edu>]

On Sun, 7 Nov 2004, Nick Kew wrote:

> On Sun, 7 Nov 2004, Tobias Skytte wrote:
>
> > Why not introduce and option to remove the
> > comments from the served file?
>
> You have that option from mod_xmlns, mod_proxy_html or mod_publisher,
> to name but three.  Along with caveats about why it's not necessarily
> a good idea.

Yes, I'm aware of at least one more 3rd-party module that does the same
thing.  I can't remember the name of it, but it was discussed on this list
a while back (maybe a year ago now).

Ultimately I suspect that the biggest problem (assuming that the "what's a
comment and what isn't" problem is solvable) is that this kind of parsing
takes a disproportionately large amount of CPU time with respect to the
amount of network bandwidth it saves.  I mean really, how much bandwidth
from *html* are we talking about?  Compared to images?  If you start
parsing the html, you lose any ability to do zero-copy and the like, not
to mention the fact that the CPU has to examine every single byte of the
input and process and/or copy it.  Yuck.

Re: suggestion: strip comments from served html pages

[Nick Kew <ni...@webthing.com>]

On Sun, 7 Nov 2004, Tobias Skytte wrote:

	 Why not introduce and option to remove the
> comments from the served file?

You have that option from mod_xmlns, mod_proxy_html or mod_publisher,
to name but three.  Along with caveats about why it's not necessarily
a good idea.

-- 
Nick Kew