You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by TO...@aol.com on 2001/09/15 11:23:55 UTC

[SUBMIT] mod_gzip 2.0.26a ( Non-debug version )

Hello all...
Kevin Kiley here.

We decided not to wait any longer for a new BETA.

Attached is the current source code for mod_gzip 
for Apache 2.x series. It has been tested 
pretty heavily and seems to be working fine.

There is only 1 file... mod_gzip.c.

You are free to do whatever you like with this
submission.

This is the Non-debug filter version and it has been
adjusted to use ZLIB.

The full 'story-telling' debug version will be
sent following this upload for anyone who might
be interested in that cut of the code.

It contains all the same configuration directives,
Item Mapping and eligibility checks currently used in the
heavily tested and widely used mod_gzip 1.3.19.1a for the
Apache 1.x series.

Below is a sample configuration for using mod_gzip
for 2.x with mod_include.

It includes sample custom log records, a root server
definition, and a single virtual host definition.

The sample below does not contain all of the possible 
mod_gzip directives.

All of the configuration directives are fully
documented on the mod_gzip home page and a standard
Apache module documentation file is forthcoming.

[snip]

# SAMPLE MOD_GZIP CONFIGURATON FOR APACHE 2.X

#
# To use server-parsed HTML files
#
<FilesMatch "\.shtml(\..+)?$">
    SetOutputFilter INCLUDES GZIP
</FilesMatch>

LogFormat "%h %l %u %t \"%r\" %>s %b mod_gzip: %{mod_gzip_result}n 
In:%{mod_gzip_input_size}n 
Out:%{mod_gzip_output_size}n:%{mod_gzip_compression_ratio}npct." 
common_with_mod_gzip_info2
CustomLog logs/access.log common_with_mod_gzip_info2

mod_gzip_on Yes
mod_gzip_min_http 1000
mod_gzip_temp_dir c:\temp
mod_gzip_item_include file .*\.htm
mod_gzip_item_include mime text/*
mod_gzip_compression_level 3
mod_gzip_command_version mod_gzip_version

<VirtualHost 127.0.0.1:4010>

    ServerAdmin webmaster@127.0.0.1
    DocumentRoot "c:\httpd\release\htdocs2"
    ServerName 127.0.0.1
    ErrorLog logs/htdocs2/error.log
    CustomLog logs/htdocs2/access.log common_with_mod_gzip_info2

    mod_gzip_on Yes
    mod_gzip_min_http 1000
    mod_gzip_temp_dir c:\temp
    mod_gzip_item_include file .*\.htm
    mod_gzip_item_include mime text/*
    mod_gzip_compression_level 6
    mod_gzip_command_version mod_gzip_version_127

</VirtualHost>

[snip]

Yours...
Kevin Kiley

Re: [SUBMIT] mod_gzip 2.0.26a ( Non-debug version )

Posted by Cliff Woolley <cl...@yahoo.com>.
On Sun, 16 Sep 2001, Justin Erenkrantz wrote:

> - For Ian's mod_gz, I removed the dependency on zutil.h as it is static
>   and OS_CODE seems to defined as part of the RFC (I may be wrong here).
>   Most installations do not include zutil.h in their install (such as
>   Solaris), so we can't expect it to be there.  This *may* require
>   adding the zlib license for those chunk of lines.  It seems to be a flaw
>   in zlib that they require an internal definition for actual use.

We can do it either way, I don't care.  If we snarf in the OS_CODE stuff
from zutil.h, then we include the ZLIB license in mod_gzip.c.  If we use
zutil.h, we just have a general acknowledgement of the zlib authors in
mod_gzip.c and the full ZLIB license in zutil.h itself (as it already is,
I'm sure).

> - The str* funcs are thread-safe on any platform we care about.
>   If the libc implementations aren't thread-safe, there are other
>   problems that we'll have in the core that mod_gzip's thread-safety
>   will be the least of our problems.

Agreed.

> - npp isn't completely bogus - I did essentially the same thing in
>   flood.  I did:
>     r->uri ? r->uri : ""
>   instead.  No need for a separate function though.

I should have been more explicit.  It's not bogus to do a conditional like
the one you just displayed.  I thought it was excessive to make it a whole
separate function that's only used in one place.  I thought it was bogus
to set it to the string "NULL" instead of the empty string, though
coincidentally a string that says "NULL" works out fine in the one place
where this function is used.  Then there the fact that I can't think of a
way that r->uri would ever be NULL.  Can it really?  Even when there is no
r->uri, we set it to the empty string.  That was that whole INTERNALLY
GENERATED bogosity that we scratched our heads over for a while and then
Ryan fixed it by setting it to "".  Under what circumstances will
r->uri==NULL?

--Cliff


--------------------------------------------------------------
   Cliff Woolley
   cliffwoolley@yahoo.com
   Charlottesville, VA



Re: [SUBMIT] mod_gzip 2.0.26a ( Non-debug version )

Posted by Justin Erenkrantz <je...@ebuilt.com>.
On Sat, Sep 15, 2001 at 03:15:58PM -0400, Cliff Woolley wrote:
> On Sat, 15 Sep 2001 TOKILEY@aol.com wrote:
> 
> > We decided not to wait any longer for a new BETA.
> >
> > Attached is the current source code for mod_gzip
> > for Apache 2.x series. It has been tested
> > pretty heavily and seems to be working fine.
> >
> > There is only 1 file... mod_gzip.c.
> >
> > You are free to do whatever you like with this
> > submission.
> 
> Thanks Kevin.
> 
> I took the liberty of applying a good dose of Apache stylistic rules
> (might have still missed some things, but I tried).  I also removed some
> more debug stuff (the r->notes things) since none of it ever seemed to be
> really used.  I removed the ZLIB license since I don't think any of this
> code actually came from ZLIB.  The fact that it uses zutil.h is immaterial
> AFAIK.  If I'm wrong, somebody tell me and I'll put the ZLIB license back
> in.  I also removed some of the verbose comments... I think it's easier to
> read with fewer comments in some places.  I took out the version numbering
> since it'd be hard to keep that in-sync if this were in the actual
> httpd-2.0 tree.  It makes sense for a 3rd party module, but not for an
> official module.  I cleaned up a few logic things, but I tried my best not
> to change any functionality.  I did, however, strip out the handling of
> the deprecated commands... if they're deprecated, that can be documented,
> but IMO there's no need to handle them just to print out an error that
> says "this isn't supported anymore".  Where I had problems with/questions
> about things, I put in an #error so I couldn't forget to take care of
> them.  (They should be easily taken care of.)  Because of the #error
> additions and the fact that I'm out of time to work on this for the day, I
> haven't even tried compiling this, so there's the possibility I made some
> stupid mistakes.  I'll get back to it later no doubt, but if someone else
> would care to take the next turn at looking over it, I'd appreciate it.
> 
> To save bandwidth, my version is here:
> http://www.apache.org/~jwoolley/mod_gzip.c

Quick comments.

- For Ian's mod_gz, I removed the dependency on zutil.h as it is static
  and OS_CODE seems to defined as part of the RFC (I may be wrong here).
  Most installations do not include zutil.h in their install (such as
  Solaris), so we can't expect it to be there.  This *may* require 
  adding the zlib license for those chunk of lines.  It seems to be a flaw 
  in zlib that they require an internal definition for actual use.

- I dunno what the _set variables are either.  Do we need them?

- The str* funcs are thread-safe on any platform we care about.
  If the libc implementations aren't thread-safe, there are other
  problems that we'll have in the core that mod_gzip's thread-safety
  will be the least of our problems.

- npp isn't completely bogus - I did essentially the same thing in
  flood.  I did:
    r->uri ? r->uri : ""
  instead.  No need for a separate function though.

- I think the IMAP_* stuff can be handled better by the filter directive
  syntax.  Ideally, mod_gz{ip} would be configured as a filter for
  certain mime types or extensions.  Not sure we need to complicate things 
  by allowing inclusion/exclusion via other config syntaxes.

- I don't understand why we need do_command.

- I don't think we need the eligibility checks either.  The mod_mime
  and core should handle this.

- I don't think we need the handler either.

Enough for now.  I need to go to bed.  I'm jet-lagged...  

I'll test it out later this week.  Once I test this implementation,
I'll see which one I like better.  =-)  -- justin


Re: [SUBMIT] mod_gzip 2.0.26a ( Non-debug version )

Posted by Cliff Woolley <cl...@yahoo.com>.
On Sat, 15 Sep 2001, Ian Holsman wrote:

> * functions should be static

Oops, I meant to fix those.

> * why are you defining your own strncmp??

I had the same question.  strncmp, strnicmp, strcpy, strcat, strlen are
all reimplemented, plus "stringcontains" (strstr) and "strendswith".  I
commented on this in the code... I have no idea why they are
reimplemented here.  Kevin?  Comments?

> * Caching should be removed (there is another caching module there
>    it should use that), failing that, maybe it should be split out to
>    a different filter
> * logging should be via the logging optional function
> * flushing should probably flush the zlib buffer before sending it out
> * only check if gzip is on ONCE per request. (ie if (!f->ctx)) and if
>    you don't want it enabled remove it from the filter chain.
> * remove the de-chunking, you won't see this anyway
> * remove the 'enable' flag. if the user has setoutputfilter'd it he wants it
> * the filter should be a HTTP_HEADER not a content one.
> * the filter should only be run on a 'main' request (do you check this?)
>    otherwise you will have a gzip'ed included file in a non-gziped main file.

All good points.

Thanks,
Cliff

--------------------------------------------------------------
   Cliff Woolley
   cliffwoolley@yahoo.com
   Charlottesville, VA



Re: [SUBMIT] mod_gzip 2.0.26a ( Non-debug version )

Posted by Ian Holsman <ia...@cnet.com>.
Cliff Woolley wrote:

> On Sat, 15 Sep 2001 TOKILEY@aol.com wrote:
>  
> I took the liberty of applying a good dose of Apache stylistic rules
> (might have still missed some things, but I tried).  I also removed some
> more debug stuff (the r->notes things) since none of it ever seemed to be
> really used.  I removed the ZLIB license since I don't think any of this
> code actually came from ZLIB.  The fact that it uses zutil.h is immaterial
> AFAIK.  If I'm wrong, somebody tell me and I'll put the ZLIB license back
> in.  I also removed some of the verbose comments... I think it's easier to
> read with fewer comments in some places.  I took out the version numbering
> since it'd be hard to keep that in-sync if this were in the actual
> httpd-2.0 tree.  It makes sense for a 3rd party module, but not for an
> official module.  I cleaned up a few logic things, but I tried my best not
> to change any functionality.  I did, however, strip out the handling of
> the deprecated commands... if they're deprecated, that can be documented,
> but IMO there's no need to handle them just to print out an error that
> says "this isn't supported anymore".  Where I had problems with/questions
> about things, I put in an #error so I couldn't forget to take care of
> them.  (They should be easily taken care of.)  Because of the #error
> additions and the fact that I'm out of time to work on this for the day, I
> haven't even tried compiling this, so there's the possibility I made some
> stupid mistakes.  I'll get back to it later no doubt, but if someone else
> would care to take the next turn at looking over it, I'd appreciate it.
> 
> To save bandwidth, my version is here:
> http://www.apache.org/~jwoolley/mod_gzip.c
> 
> --Cliff


additional comments (my 2c)
* Caching should be removed (there is another caching module there
   it should use that), failing that, maybe it should be split out to
   a different filter
* functions should be static
* why are you defining your own strncmp??
* logging should be via the logging optional function
* flushing should probably flush the zlib buffer before sending it out
* only check if gzip is on ONCE per request. (ie if (!f->ctx)) and if you don't want
   it enabled remove it from the filter chain.
* remove the de-chunking, you won't see this anyway
* remove the 'enable' flag. if the user has setoutputfilter'd it he wants it
* the filter should be a HTTP_HEADER not a content one.
* the filter should only be run on a 'main' request (do you check this?)
   otherwise you will have a gzip'ed included file in a non-gziped main file.

..Ian



> 
> --------------------------------------------------------------
>    Cliff Woolley
>    cliffwoolley@yahoo.com
>    Charlottesville, VA
> 
> 




Re: [SUBMIT] mod_gzip 2.0.26a ( Non-debug version )

Posted by Cliff Woolley <cl...@yahoo.com>.
On Sat, 15 Sep 2001 TOKILEY@aol.com wrote:

> We decided not to wait any longer for a new BETA.
>
> Attached is the current source code for mod_gzip
> for Apache 2.x series. It has been tested
> pretty heavily and seems to be working fine.
>
> There is only 1 file... mod_gzip.c.
>
> You are free to do whatever you like with this
> submission.

Thanks Kevin.

I took the liberty of applying a good dose of Apache stylistic rules
(might have still missed some things, but I tried).  I also removed some
more debug stuff (the r->notes things) since none of it ever seemed to be
really used.  I removed the ZLIB license since I don't think any of this
code actually came from ZLIB.  The fact that it uses zutil.h is immaterial
AFAIK.  If I'm wrong, somebody tell me and I'll put the ZLIB license back
in.  I also removed some of the verbose comments... I think it's easier to
read with fewer comments in some places.  I took out the version numbering
since it'd be hard to keep that in-sync if this were in the actual
httpd-2.0 tree.  It makes sense for a 3rd party module, but not for an
official module.  I cleaned up a few logic things, but I tried my best not
to change any functionality.  I did, however, strip out the handling of
the deprecated commands... if they're deprecated, that can be documented,
but IMO there's no need to handle them just to print out an error that
says "this isn't supported anymore".  Where I had problems with/questions
about things, I put in an #error so I couldn't forget to take care of
them.  (They should be easily taken care of.)  Because of the #error
additions and the fact that I'm out of time to work on this for the day, I
haven't even tried compiling this, so there's the possibility I made some
stupid mistakes.  I'll get back to it later no doubt, but if someone else
would care to take the next turn at looking over it, I'd appreciate it.

To save bandwidth, my version is here:
http://www.apache.org/~jwoolley/mod_gzip.c

--Cliff

--------------------------------------------------------------
   Cliff Woolley
   cliffwoolley@yahoo.com
   Charlottesville, VA