Whitespace strip filter for httpd v2.1

[Graham Leggett <mi...@sharp.fm>]

Hi all,

I have attached a small filter module that strips leading whitespace 
from text files. This would typically be used to remove the "indenting" 
whitespace found inside HTML files, resulting in a significant reduction 
in network traffic for some sites.

I didn't bother to include trailing whitespace removal, as this involved 
buffering (leading whitespace removal requires no buffering).

Comments?

Regards,
Graham
--

Re: Whitespace strip filter for httpd v2.1

[Graham Leggett <mi...@sharp.fm>]

Ian Holsman wrote:

> In a nutshell,
> it is very hard to strip whitespace out of HTML, and have the resulting 
> file still be as valid as the old one. Browsers are very forgiving on 
> something and not on others. for example,
> you should not strip code inside of javascript, or 'pre' tags.

The filter does not try to be content safe - otherwise the resulting 
processing burden would mean "just use deflate and save yourself the 
hassle". It is assumed that an admin deploying the filter knows what 
they are doing when they do so. We ship with a module that changes case 
of the content - that will very likely break javascript too, but again, 
it assumes the user knows that it's for.

Regards,
Graham
--

Re: Whitespace strip filter for httpd v2.1

[Ian Holsman <li...@holsman.net>]

Cliff Woolley wrote:
> On Mon, 22 Nov 2004, Graham Leggett wrote:
> 
> 
>>I have attached a small filter module that strips leading whitespace
>>from text files. This would typically be used to remove the "indenting"
>>whitespace found inside HTML files, resulting in a significant reduction
>>in network traffic for some sites. I didn't bother to include trailing
>>whitespace removal, as this involved buffering (leading whitespace
>>removal requires no buffering).
> 
> 
> Well, we've had people submit modules in the past that stripped the
> extraneous whitespace from HTML files, and we rejected them.  Check the
> archives for messages from a guy named fabio rohrich, who wrote a module
> called mod_blank to see why that might have been.  I don't completely
> remember the details.

In a nutshell,
it is very hard to strip whitespace out of HTML, and have the resulting 
file still be as valid as the old one. Browsers are very forgiving on 
something and not on others. for example,
you should not strip code inside of javascript, or 'pre' tags.

> 
> --Cliff
> 

Re: Whitespace strip filter for httpd v2.1

[Cliff Woolley <jw...@virginia.edu>]

On Mon, 22 Nov 2004, Graham Leggett wrote:

> I have attached a small filter module that strips leading whitespace
> from text files. This would typically be used to remove the "indenting"
> whitespace found inside HTML files, resulting in a significant reduction
> in network traffic for some sites. I didn't bother to include trailing
> whitespace removal, as this involved buffering (leading whitespace
> removal requires no buffering).

Well, we've had people submit modules in the past that stripped the
extraneous whitespace from HTML files, and we rejected them.  Check the
archives for messages from a guy named fabio rohrich, who wrote a module
called mod_blank to see why that might have been.  I don't completely
remember the details.

--Cliff

Re: Whitespace strip filter for httpd v2.1

[Graham Leggett <mi...@sharp.fm>]

Bill Stoddard wrote:

> +1 in concept but code not reviewed. mod_deflate would effectivly do the 
> same thing, right? What user base would are we targeting with this 
> module (looking for an answer a bit more specific than 'web clients that 
> don't support gzip decoding').

Bandwidth reduction for servers that cannot afford the horsepower for a 
full deflate? Whitespace stripping is negligible in terms of horsepower, 
deflate (I understand) is not.

Regards,
Graham
--

Re: Whitespace strip filter for httpd v2.1

[Bill Stoddard <bi...@wstoddard.com>]

Graham Leggett wrote:

> Hi all,
> 
> I have attached a small filter module that strips leading whitespace 
> from text files. This would typically be used to remove the "indenting" 
> whitespace found inside HTML files, resulting in a significant reduction 
> in network traffic for some sites.
> 
> I didn't bother to include trailing whitespace removal, as this involved 
> buffering (leading whitespace removal requires no buffering).
> 
> Comments?

+1 in concept but code not reviewed. mod_deflate would effectivly do the same thing, right? What user base 
would are we targeting with this module (looking for an answer a bit more specific than 'web clients that 
don't support gzip decoding').

Bill

Re: Whitespace strip filter for httpd v2.1

[Geoffrey Young <ge...@modperlcookbook.org>]

Graham Leggett wrote:
> Hi all,
> 
> I have attached a small filter module that strips leading whitespace
> from text files. This would typically be used to remove the "indenting"
> whitespace found inside HTML files, resulting in a significant reduction
> in network traffic for some sites.
> 
> I didn't bother to include trailing whitespace removal, as this involved
> buffering (leading whitespace removal requires no buffering).
> 
> Comments?

this may be outside the scope of the example code, but filters like this
probably ought to consider the ETag header.  my personal opinion is that
because the content will not be exactly the same bitwise as, say, a file on
disk, any ETag header should be made weak to be compliant.  I think roy
didn't agree, but I never really understood his response last time I brought
this up.

anyway, if you start considering the ETag header, then you need to consider
what happens when default-handler returns 304 and short-circuits your filter
entirely.  in this case the ETag returned would be the strong ETag created
by ap_set_etag, even though the cached response was using a weak ETag. so,
you would probably want to create a filter_init hook to weaken any ETag.
but that ETag would possibly run in vain if your filter doesn't have any
whitespace to change, so...

anyway, I'm bringing this up only as something to consider - it's a real
life problem for me that I have tried several times to figure out within the
current filter architecture but have come up short.  so, for a simple filter
like this to be fully RFC compliant would be a bit help to others that want
to apply filters in more complex settings.

fwiw

--Geoff