Source reorganisation

[Paul Sutton <pa...@ukweb.com>]

There has been some discussion about re-organising the source files. To
give us something concrete to talk about, and possibly implement, I've
attempted to do this. I've also implemented some of the requested features
such as making it easier to install new (third-party) modules, and to
allow modules to influence the configuration process.

The system implements the following changes:

  1. Source code re-organisation, with the "core" source and header files
     moved into /src/core, os-specific files in /src/os/OSNAME, modules
     in /src/modules/SOMETHING and support in /src/support. OS-specific
     source and header files can be added.

  2. Modules are easier to add and better organised. They can be placed
     in _any_ subdirectory of /src/modules, and a suitable Makefile will
     be auto-created if necessary. For example, all the default modules
     could go into /src/modules/standard, all the auth modules into
     /src/modules/auth, etc. Third-party modules can be dropped into any
     new or existing directory. 

  3. Modules can contain additional information used by "Configure". For
     example, they can request extra libraries or add extra CFLAGS. This
     can be specified either within the C-file itself (as a specially
     formatted comment), or in a separted "module definition file", which
     could be distributed with binary (.o or .a) modules.

Besides moving the source files around, the main changes for this are to
Configure itself. Since it does involve moving most of the source files, I
cannot really send a working patch for the whole change. But I could send
a patch for Configure if this seems like a good idea.

I think it makes the Apache source tree easier to manage, allows us to
start on OS abstraction, makes it easier to provide additonal modules and
provides a better level of organisation to the source. It should also have
_no_ effect on the stabilty of Apache itself, since there are no code
changes.

//pcs

Re: Source reorganisation

[Dean Gaudet <dg...@arctic.org>]

I'm neutral on the tree changes.  But I'm +1 on the Configure features
allowing modules to have methods for tweaking CFLAGS and such. 

When the tree changes happen (I'm assuming they will) it's probably the
right time to start a second repository.  cp the entire repository and
then mv the actual ,v files around please so we don't lose history.

One thing I don't like about tree changes is that simple greps become find
-name '*.[ch]' | xargs grep... which I find a little longer to type ;) 

Dean

On Sun, 29 Jun 1997, Paul Sutton wrote:

> There has been some discussion about re-organising the source files. To
> give us something concrete to talk about, and possibly implement, I've
> attempted to do this. I've also implemented some of the requested features
> such as making it easier to install new (third-party) modules, and to
> allow modules to influence the configuration process.
> 
> The system implements the following changes:
> 
>   1. Source code re-organisation, with the "core" source and header files
>      moved into /src/core, os-specific files in /src/os/OSNAME, modules
>      in /src/modules/SOMETHING and support in /src/support. OS-specific
>      source and header files can be added.
> 
>   2. Modules are easier to add and better organised. They can be placed
>      in _any_ subdirectory of /src/modules, and a suitable Makefile will
>      be auto-created if necessary. For example, all the default modules
>      could go into /src/modules/standard, all the auth modules into
>      /src/modules/auth, etc. Third-party modules can be dropped into any
>      new or existing directory. 
> 
>   3. Modules can contain additional information used by "Configure". For
>      example, they can request extra libraries or add extra CFLAGS. This
>      can be specified either within the C-file itself (as a specially
>      formatted comment), or in a separted "module definition file", which
>      could be distributed with binary (.o or .a) modules.
> 
> Besides moving the source files around, the main changes for this are to
> Configure itself. Since it does involve moving most of the source files, I
> cannot really send a working patch for the whole change. But I could send
> a patch for Configure if this seems like a good idea.
> 
> I think it makes the Apache source tree easier to manage, allows us to
> start on OS abstraction, makes it easier to provide additonal modules and
> provides a better level of organisation to the source. It should also have
> _no_ effect on the stabilty of Apache itself, since there are no code
> changes.
> 
> //pcs
> 
> 
> 
> 

Re: Source reorganisation

[Paul Sutton <pa...@ukweb.com>]

On Sun, 29 Jun 1997, Alexei Kosut wrote:
> However, there is one thing that's always bugged me about Apache
> vs. some other Unix packages, and this has to do with both the
> organization of the source and the configuration issue, is this:
> To compile the server, Apache requires the user to go into the src/
> directory, and muck with things there, and the httpd binary ends up
> there. IMHO, Configure and the Makefile it produces should be in the
> top-level directory, as should the resultant httpd binary. Again IMHO,
> the user should never be forced to even see source files unless they
> plan to read or change them (which I think most Apache users do
> not).

I agree. But the thing that has bugged me the most is that Apache lacks a
concept of installation after compilation. I _think_ this was the fault of
NCSA, which for some reason I never understood assumed you'd be running
the server from the source tree. This is (IMO) an appallingly bad idea. I
never want to run files from the source area except while testing or
debugging. I want my source to have a clearly defined installation stage
which moves the binaries (httpd, htpasswd) to the right places, creates
appropriate support directories, etc.

//pcs

Re: Source reorganisation

[Marc Slemko <ma...@worldgate.com>]

On Sun, 29 Jun 1997, Alexei Kosut wrote:

> I'm not against these changes in 1.3, but I personally feel they'd go
> better into 2.0, when we *are* making code changes. For one thing, the
> origanlization of the reoganization needs some thinking about (the
> exact proposal you made has some problems, as others have pointed
> out).

I am wary about 1.3 getting out of hand.  Development is good, but 1.3 was
supposed to be a quick plop Windows in, some new non-core features, and
off we go.

> However, there is one thing that's always bugged me about Apache
> vs. some other Unix packages, and this has to do with both the
> organization of the source and the configuration issue, is this:
> To compile the server, Apache requires the user to go into the src/
> directory, and muck with things there, and the httpd binary ends up
> there. IMHO, Configure and the Makefile it produces should be in the
> top-level directory, as should the resultant httpd binary. Again IMHO,
> the user should never be forced to even see source files unless they
> plan to read or change them (which I think most Apache users do
> not).

I'm wondering if that is better left to an install script in the top level
directory that can do things like make the server, copy the binary,
install the tree, perhaps even a quick edit of the config files, etc.  I
don't know that it is necessary to take the configuration script and
makefile out of the subdirectory to achieve the goal of hiding them from
users who don't care.

> 
> But as I said, this is more a 2.0 thing. I'd still like to see a 1.3b1
> within a month.
> 
> -- 
> ________________________________________________________________________
> Alexei Kosut <ak...@nueva.pvt.k12.ca.us>      The Apache HTTP Server
> URL: http://www.nueva.pvt.k12.ca.us/~akosut/   http://www.apache.org/
> 

Re: Source reorganisation

[Alexei Kosut <ak...@nueva.pvt.k12.ca.us>]

On Sun, 29 Jun 1997, Paul Sutton wrote:

[...]

> I think it makes the Apache source tree easier to manage, allows us to
> start on OS abstraction, makes it easier to provide additonal modules and
> provides a better level of organisation to the source. It should also have
> _no_ effect on the stabilty of Apache itself, since there are no code
> changes.

I'm not against these changes in 1.3, but I personally feel they'd go
better into 2.0, when we *are* making code changes. For one thing, the
origanlization of the reoganization needs some thinking about (the
exact proposal you made has some problems, as others have pointed
out).

However, there is one thing that's always bugged me about Apache
vs. some other Unix packages, and this has to do with both the
organization of the source and the configuration issue, is this:
To compile the server, Apache requires the user to go into the src/
directory, and muck with things there, and the httpd binary ends up
there. IMHO, Configure and the Makefile it produces should be in the
top-level directory, as should the resultant httpd binary. Again IMHO,
the user should never be forced to even see source files unless they
plan to read or change them (which I think most Apache users do
not).

But as I said, this is more a 2.0 thing. I'd still like to see a 1.3b1
within a month.

-- 
________________________________________________________________________
Alexei Kosut <ak...@nueva.pvt.k12.ca.us>      The Apache HTTP Server
URL: http://www.nueva.pvt.k12.ca.us/~akosut/   http://www.apache.org/

Re: Source reorganisation

[Paul Sutton <pa...@ukweb.com>]

On Sun, 29 Jun 1997, Marc Slemko wrote:
> On Sun, 29 Jun 1997, Paul Sutton wrote:
> >   1. Source code re-organisation, with the "core" source and header files
> >      moved into /src/core, os-specific files in /src/os/OSNAME, modules
> >      in /src/modules/SOMETHING and support in /src/support. OS-specific
> >      source and header files can be added.
> 
> What is an OS?  Is Unix an OS or is FreeBSD an OS?  I don't see the need
> or desire for seperate subdirectories per Unix variety, as someone has
> suggested sometime.  We just don't have the volume to make it worthwhile.

Umm, I can see advantages to treating these as different OSes. But my
assumption was that the are only three OSes at the moment: win32, unix and
emx. FreeBSD, Linux, etc are 'variants' of the unix OS, while winnt and
win95 are variants of win32. 

> >   2. Modules are easier to add and better organised. They can be placed
> >      in _any_ subdirectory of /src/modules, and a suitable Makefile will
> >      be auto-created if necessary. For example, all the default modules
> >      could go into /src/modules/standard, all the auth modules into
> >      /src/modules/auth, etc. Third-party modules can be dropped into any
> >      new or existing directory. 
> 
> Sounds ok, but if you are defining "default modules" as those compiled in
> by default, I'm not sure that distinction is worthwhile.  If you mean
> those distributed by default, sure.

The arrangement of the modules is arbitrary - the new Configure can handle
any modules in any directories. It will auto-create Makefiles as necessary
in directories under /src/modules. So the paths above a just examples. For
my testing I put all the modules distributed with Apache in
/src/modules/standard, except for the authentication modules which went
into /src/modules/auth. The auth modules are a functionally distint set of
modules, but of course they could all be put into a single directory as
well. 

This scheme would also let us distribute "experimental" modules more
clearly marked as such, by putting them in /src/modules/experimental. 

> >   3. Modules can contain additional information used by "Configure". For
> >      example, they can request extra libraries or add extra CFLAGS. This
> >      can be specified either within the C-file itself (as a specially
> >      formatted comment), or in a separted "module definition file", which
> >      could be distributed with binary (.o or .a) modules.
> 
> Sounds good.  Seperate module definition file would be best.  Could even
> make a standard format that includes author, description, etc.

Yes, that is a good idea. The format can of course be adjusted, but a
typical module definition might look like this: 

Name:        dbm_auth_module
Author:      Apache Group
Description: Basic authentication with passwords stored DBM file
StartConfigure
  LIBS=$LIBS $DBMLIB
EndConfigure

The Name: gives the module structure name (needed for Configure and
currently given in Configuration), the next two lines are comments, and
the StartConfigure...EndConfigure block is executed by Configure after it
has determined the OS specific settings. I would prefer not to have shell
commands in the definition, but modules need to be able to influence the
configuration in a generic way (e.g. mod_status needs to check the value
of the STATUS rule). 

This whole block can be put into a separate file (i.e. 
mod_auth_dbm.module), or added into the source code inside a comment). 

//pcs

Re: Source reorganisation

[Marc Slemko <ma...@znep.com>]

On Sun, 29 Jun 1997, Paul Sutton wrote:

> There has been some discussion about re-organising the source files. To
> give us something concrete to talk about, and possibly implement, I've
> attempted to do this. I've also implemented some of the requested features
> such as making it easier to install new (third-party) modules, and to
> allow modules to influence the configuration process.
> 
> The system implements the following changes:
> 
>   1. Source code re-organisation, with the "core" source and header files
>      moved into /src/core, os-specific files in /src/os/OSNAME, modules
>      in /src/modules/SOMETHING and support in /src/support. OS-specific
>      source and header files can be added.

What is an OS?  Is Unix an OS or is FreeBSD an OS?  I don't see the need
or desire for seperate subdirectories per Unix variety, as someone has
suggested sometime.  We just don't have the volume to make it worthwhile.

> 
>   2. Modules are easier to add and better organised. They can be placed
>      in _any_ subdirectory of /src/modules, and a suitable Makefile will
>      be auto-created if necessary. For example, all the default modules
>      could go into /src/modules/standard, all the auth modules into
>      /src/modules/auth, etc. Third-party modules can be dropped into any
>      new or existing directory. 

Sounds ok, but if you are defining "default modules" as those compiled in
by default, I'm not sure that distinction is worthwhile.  If you mean
those distributed by default, sure.

>   3. Modules can contain additional information used by "Configure". For
>      example, they can request extra libraries or add extra CFLAGS. This
>      can be specified either within the C-file itself (as a specially
>      formatted comment), or in a separted "module definition file", which
>      could be distributed with binary (.o or .a) modules.

Sounds good.  Seperate module definition file would be best.  Could even
make a standard format that includes author, description, etc.

> Besides moving the source files around, the main changes for this are to
> Configure itself. Since it does involve moving most of the source files, I
> cannot really send a working patch for the whole change. But I could send
> a patch for Configure if this seems like a good idea.
> 
> I think it makes the Apache source tree easier to manage, allows us to
> start on OS abstraction, makes it easier to provide additonal modules and
> provides a better level of organisation to the source. It should also have
> _no_ effect on the stabilty of Apache itself, since there are no code
> changes.

I'm thinking this is best left until after 1.3.  Right now, and in the
forseeable future, it is nice to be able to do a diff between HEAD and
1.2.0 to see all the things that have come in, find the bogus ones, etc.