You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by "Roy T. Fielding" <fi...@ebuilt.com> on 2000/11/27 13:04:13 UTC
cvs indecision
I've been trying to figure out the least annoying way of fixing
the cvs repositories for httpd-docs and apache-2.0 (and indirectly
the one for apr, though it is not currently broken). Unfortunately,
what I've found is that all of the methods require a clean checkout
be forced.
To fix the docs, what I need to do in /home/cvs/* is
recursively, for each rcs ,v file in apache-2.0/htdocs/Attic
-- find the corresponding file in httpd-docs-2.0/htdocs
-- compare the tags in the old file to the new one
-- use rcs -nTAG:rev to restore each missing tag
mv apache-2.0/htdocs /home/fielding/save
mv httpd-docs-2.0/htdocs apache-2.0/htdocs
edit CVSROOT/modules accordingly
chgrp -R apcvs apache-2.0/htdocs
chgrp -R apcvs apache-2.0/src/include
chgrp -R apcvs apache-2.0/src/os/unix
chgrp -R apcvs apr/include
[note: I believe the last three are bogus, but that is what the
httpd-docs-2.0 module is trying to do. I think that any
changes to the include files should require someone in the
apr or server group to commit, so I would prefer to leave
those directories under the core groups.]
To "fix" apr so that it is automatically checked-out with apache-2.0
mv apache-2.0/src/lib/apr /home/fielding/save
add the following to the CVSROOT/modules file
apache-2.0 apache-2.0 &apr-httpd-2.0
apr-httpd-2.0 -d src/lib/apr apr
[note: removing the existing apr is necessary to avoid the overlap
error message we currently get from htdocs. Since apr still
has the history from apache-2.0, this will work fine.]
I could have done the above this [last] night, but I need to fly to Austin
today and will probably be away from e-mail until late Tuesday night. Not a
good idea to mess with things just before a trip. What really stopped
me though was that figuring this stuff out reminded me of how completely
f*cked the current source tree layout is for apache-2.0, and that it
would take the same amount of effort to just fix the whole thing as it
would to restore the broken bits above.
What I am thinking of doing is rebuilding the repository as if it were
a standard source tree (not a dual build/source tree), as follows:
httpd-2.0/
build/ [src/build and src/helpers]
docs/
docroot/ [containing index files in htdocs]
conf/ [what is in conf now]
icons/ [just the small ones]
manual/ [what is now htdocs/manual]
include/
modules/
aaa/ [access authentication and auth DBs]
generators/ [mod_asis, mod_cgid, etc.]
filters/ [mod_include, etc.]
loggers/ [content generators]
mappers/ [mod_alias, mod_rewrite, etc.]
metadata/ [mod_cern_meta, mod_user_track, etc.]
cache/
dav/
proxy/
experimental/
server/ [src/main]
mpm/
http/
ssl/
main.c
buildmark.c
srclib/
apr/ [autocheckout? via &apr-httpd-2.0]
aputil/ [what greg just created and src/ap]
expat-lite/ [autocheckout?]
pcre/ [autocheckout?]
sdbm/ [autocheckout?]
openssl/ [autocheckout?]
zlib/ [autocheckout?]
support/
test/
ABOUT_APACHE
README
INSTALL
LICENSE
NOTICE
CHANGES
KEYS
Announcement
Apache.dsp
ApacheCore.def
ApacheCore.dsp
ApacheCoreDll.dsp
Makefile.in
Makefile.win
acinclude.m4
buildconf
configure.in
In other words, remove the vestigial directories from Apache 1.0
(logs, htdocs, etc), eliminate the src layer, stop using the standard
names of install directories for our own source code (conf, lib, etc.),
segregate the modules by function rather than some arbitrary
notion of their status, and prepare for new server protocols.
Note that this is a new repository name. I would only copy over the
current rcs files. The old repository would be left as-is so that
we can still do checkouts of prior versions and have the files
end up in the right (older) directories. It would significantly
speed up checkouts and updates, since most of the dirs and files in
the current tree are nothing but Attic space.
It also reduces coupling between the core and modules directories
(mpm should never have been moved there) and improves cohesion.
Of course, the negative is that we'd have to then fix the build system.
Comments? [I won't be able to respond much until Wednesday]
Yikes, it took me five hours just to write that plan.
....Roy
Re: cvs reorg (was: Re: cvs indecision)
Posted by rb...@covalent.net.
> > There is nothing sacred about srclib -- they are just libraries of
> > code used by more than one module (whether that be a mod_whatever or
> > some other Apache project).
>
> Can we please codify that in a document about the repository layout? You've
> probably missed it, but there has been recent traffic on new-httpd about the
> purpose of src/lib/. Is it:
>
> 1) completely separable (from Apache) packages
> or
> 2) little bundles of functionality used by Apache, possibly by others
>
> Your statement above tends towards (2).
Personally, when I read Roy's statement, I saw it leaning more towards
(1), which just goes to show that we all see what we want to see. :-)
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: cvs reorg (was: Re: cvs indecision)
Posted by rb...@covalent.net.
On Fri, 1 Dec 2000, Sander van Zoest wrote:
> On Thu, 30 Nov 2000, Greg Stein wrote:
>
> > > Regardless, mod_echo
> > > would be a sibling to http [wtf was it doing in modules/standard?].
> > Oh. But wait. The reason why I ask about a "protocols" section under
> > modules/ is because things like mod_echo are modules. There isn't any reason
> > why HTTP wouldn't be handled as a module (and SSL, too)
>
> I would be really in favor of pulling out the whole http_protocol.c out
> into a protocol module and the rest of http_* more generic so other protocols
> can also get the benefits of log,request and vhost and almost for free.
>
> if you want to write a procotol module like gopher or maybe smtp, you would
> want to be able to re-use the request_rec, vhost, acls and logging. Right
> now that isn't as easy as you would hope, cause the http protocol isn't
> a module.
This should be the goal, but having tried this once, it is not a trivial
thing to do, and should not be a part of this re-org. The big problem is
that we don't have the abstraction perfect yet, and there are a lot of
things that are HTTP specific in the http_* files.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: cvs reorg (was: Re: cvs indecision)
Posted by Sander van Zoest <sa...@covalent.net>.
On Thu, 30 Nov 2000, Greg Stein wrote:
> > Regardless, mod_echo
> > would be a sibling to http [wtf was it doing in modules/standard?].
> Oh. But wait. The reason why I ask about a "protocols" section under
> modules/ is because things like mod_echo are modules. There isn't any reason
> why HTTP wouldn't be handled as a module (and SSL, too)
I would be really in favor of pulling out the whole http_protocol.c out
into a protocol module and the rest of http_* more generic so other protocols
can also get the benefits of log,request and vhost and almost for free.
if you want to write a procotol module like gopher or maybe smtp, you would
want to be able to re-use the request_rec, vhost, acls and logging. Right
now that isn't as easy as you would hope, cause the http protocol isn't
a module.
Just some thoughts,
--
Sander van Zoest [sander@covalent.net]
Covalent Technologies, Inc. http://www.covalent.net/
(415) 536-5218 http://www.vanzoest.com/sander/
cvs reorg (was: Re: cvs indecision)
Posted by Greg Stein <gs...@lyra.org>.
On Thu, Nov 30, 2000 at 06:08:48PM -0800, Roy T. Fielding wrote:
> I hate travelling -- got back from Austin exhausted and sickly.
eek. my sympathies...
> On Tue, Nov 28, 2000 at 12:46:47PM -0800, Greg Stein wrote:
> > On Mon, Nov 27, 2000 at 04:04:13AM -0800, Roy T. Fielding wrote:
> > >...
> > > docs/
> > > docroot/ [containing index files in htdocs]
> > > conf/ [what is in conf now]
> > > icons/ [just the small ones]
> > > manual/ [what is now htdocs/manual]
> >
> > Toss or keep cgi-bin/ ?
>
> Keep, but renamed. How about cgi-examples? (or just cgi)
I like cgi-examples. That clarifies their purpose, and their optional use.
>...
> > A protocols section? e.g. where does mod_echo go? And would we put HTTP as a
> > sibling of echo? (rather than server/http/ as you have below)
>
> I was trying to simplify things so that people working on the front-end
> of the server (the core) would not have to jump separate hierarchies.
> Perhaps it would be better to have
>
> protocols/
> echo/
> http/
> ssl/
> server/ [src/main]
> mpm/
> main.c
> buildmark.c
>
> but then people will start complaining about dav and proxy protocols.
hehe. "but that is just HTTP!" will become a FAQ :-)
But maybe this says that dav and proxy should go there?
> I would prefer to keep front-end modules in the server directory and
> back-end modules in the modules directory.
Seems fine, depending on how you define those terms... is dav a front-end
protocol? or a back-end implementation? (by virtue of leveraging HTTP and
using the modules mechanism)
> Regardless, mod_echo
> would be a sibling to http [wtf was it doing in modules/standard?].
No clue. There was a suggestion somewhere to move it out of there. It is
just that nobody got around to it. Guess we have a chance now :-) Back to
the original point... it would seem that it would go into server/echo/.
Oh. But wait. The reason why I ask about a "protocols" section under
modules/ is because things like mod_echo are modules. There isn't any reason
why HTTP wouldn't be handled as a module (and SSL, too)
So, the question becomes: do we organize solely by function, or do they go
into modules/ because of their implementation mechanism?
> Maybe the directory should be called back-end instead of modules?
Nope. You have modules to do logging, filters, possibly config, etc. I'm not
sure that you can classify all of them as "backend". Also, this kind of
distinguishing characteristic would need to be explained somewhere: "what is
the frontend? how is that different/separate from the backend?"
> My opinion would be different if we distributed the header files based
> on purpose, since then it might make sense to put all of the protocol
> definitions in the same directory as client and server implementations.
> However, since header files are all in the include directory, there
> really isn't much commonality between the front-end and the back-end.
Well, not quite. mod_dav.h lives in modules/dav/main/, and mod_include.h
currently lives in modules/standard/. Each of these need to be exported for
third-party modules to use.
At the moment, this is done by the "install-include" target (to include
mod_dav.h; mod_include.h isn't published yet)
>...
> > There is no canonical SDBM distro, let alone a CVS repository for it. The
> > SDBM in Apache CVS is "ours" for all intents and purposes, and falls under
> > our continued maintenance/development.
>
> I meant that it would make more sense for these to be their own cvs
> modules on apache.org, so that they could be used by multiple projects.
> We would still be maintaining them as "ours", perhaps only as a vendor
> branch for things like openssl and zlib, but I don't see any reason
> to keep them in the httpd cvs module directly.
This is fine. We're ramping up an "aprutil" library right now. That would
hold the portable functionality, but useful for app-building. Things like
MD5 (from APR), and from Apache: src/ap/, src/lib/aputil/, and some pieces
of src/main/. (aprutil took over from where aputil was going to go; so the
new aputil is already obsolete :-)
[ and note: we probably do *not* want to start screwing with CVS vendor
branches. bad, painful juj (based on my experience while we did that in
SVN for a while; painful even given that Karl "Karl Knows CVS" Fogel was
managing the process ]
> There is nothing sacred about srclib -- they are just libraries of
> code used by more than one module (whether that be a mod_whatever or
> some other Apache project).
Can we please codify that in a document about the repository layout? You've
probably missed it, but there has been recent traffic on new-httpd about the
purpose of src/lib/. Is it:
1) completely separable (from Apache) packages
or
2) little bundles of functionality used by Apache, possibly by others
Your statement above tends towards (2).
>...
> I need to restore the old tags to the docs first before I can do any
> moving of files, and that will fill my time tonight. Let's try to
> get the directories finalized by tomorrow. I can do whatever move is
> desired after that. The main thing is that we get the directories
> right, since it is the ridiculous number of empty directories and
> overlapping cvs modules that is killing checkouts right now.
No rush. I much prefer the new organization and clarification about the
repository layout. Whenever it happens is Goodness(tm).
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: cvs indecision
Posted by "Roy T. Fielding" <fi...@ebuilt.com>.
I hate travelling -- got back from Austin exhausted and sickly.
On Tue, Nov 28, 2000 at 12:46:47PM -0800, Greg Stein wrote:
> On Mon, Nov 27, 2000 at 04:04:13AM -0800, Roy T. Fielding wrote:
> >...
> > What I am thinking of doing is rebuilding the repository as if it were
> > a standard source tree (not a dual build/source tree), as follows:
> >
> > httpd-2.0/
> > build/ [src/build and src/helpers]
> > docs/
> > docroot/ [containing index files in htdocs]
> > conf/ [what is in conf now]
> > icons/ [just the small ones]
> > manual/ [what is now htdocs/manual]
>
> Toss or keep cgi-bin/ ?
Keep, but renamed. How about cgi-examples? (or just cgi)
> > include/
> > modules/
> > aaa/ [access authentication and auth DBs]
> > generators/ [mod_asis, mod_cgid, etc.]
> > filters/ [mod_include, etc.]
> > loggers/ [mod_log_config, etc.]
> > mappers/ [mod_alias, mod_rewrite, etc.]
> > metadata/ [mod_cern_meta, mod_user_track, etc.]
> > cache/
> > dav/
> > proxy/
> > experimental/
>
> A protocols section? e.g. where does mod_echo go? And would we put HTTP as a
> sibling of echo? (rather than server/http/ as you have below)
I was trying to simplify things so that people working on the front-end
of the server (the core) would not have to jump separate hierarchies.
Perhaps it would be better to have
protocols/
echo/
http/
ssl/
server/ [src/main]
mpm/
main.c
buildmark.c
but then people will start complaining about dav and proxy protocols.
I would prefer to keep front-end modules in the server directory and
back-end modules in the modules directory. Regardless, mod_echo
would be a sibling to http [wtf was it doing in modules/standard?].
Maybe the directory should be called back-end instead of modules?
My opinion would be different if we distributed the header files based
on purpose, since then it might make sense to put all of the protocol
definitions in the same directory as client and server implementations.
However, since header files are all in the include directory, there
really isn't much commonality between the front-end and the back-end.
> > srclib/
> > apr/ [autocheckout? via &apr-httpd-2.0]
> > aputil/ [what greg just created and src/ap]
> > expat-lite/ [autocheckout?]
> > pcre/ [autocheckout?]
>
> We have local mods to the above two packages, so an autocheckout isn't going
> to work for these.
>
> > sdbm/ [autocheckout?]
>
> There is no canonical SDBM distro, let alone a CVS repository for it. The
> SDBM in Apache CVS is "ours" for all intents and purposes, and falls under
> our continued maintenance/development.
I meant that it would make more sense for these to be their own cvs
modules on apache.org, so that they could be used by multiple projects.
We would still be maintaining them as "ours", perhaps only as a vendor
branch for things like openssl and zlib, but I don't see any reason
to keep them in the httpd cvs module directly.
There is nothing sacred about srclib -- they are just libraries of
code used by more than one module (whether that be a mod_whatever or
some other Apache project).
> > openssl/ [autocheckout?]
> > zlib/ [autocheckout?]
> > support/
> > test/
> >
>
> Where would src/os/ go? I would suggest server/os/. Note that it continues
> to exist because we have os-specific functionality that lives in there (for
> example, Win32 service registration and handling). This is app-level
> functionality, rather than portability stuff, so it goes here rather than in
> APR.
I don't believe in os-specific directories. Anything that is necessary
for portability should be in APR, and anything else should be located
according to function. This may mean that there are foo_win32.c files
that are selectively compiled, but one of the main things that makes
the core configuration a mess right now is the lack of abstraction.
I need to restore the old tags to the docs first before I can do any
moving of files, and that will fill my time tonight. Let's try to
get the directories finalized by tomorrow. I can do whatever move is
desired after that. The main thing is that we get the directories
right, since it is the ridiculous number of empty directories and
overlapping cvs modules that is killing checkouts right now.
....Roy
Re: suexec and ap_os_create_privileged_process (was: Re: cvs indecision)
Posted by Greg Stein <gs...@lyra.org>.
On Thu, Nov 30, 2000 at 07:26:38PM -0800, Roy T. Fielding wrote:
> > Well, the problem is that it uses SUEXEC for its operation, and that is
> > quite Apache specific. Many platforms just default to apr_create_process(),
> > but Unix-ish platforms use SUEXEC.
>
> Oh. In that case, I'd move it to server/mpm/common/.
Sounds good to me!
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: suexec and ap_os_create_privileged_process (was: Re: cvs indecision)
Posted by "Roy T. Fielding" <fi...@ebuilt.com>.
> Well, the problem is that it uses SUEXEC for its operation, and that is
> quite Apache specific. Many platforms just default to apr_create_process(),
> but Unix-ish platforms use SUEXEC.
Oh. In that case, I'd move it to server/mpm/common/.
....Roy
suexec and ap_os_create_privileged_process (was: Re: cvs indecision)
Posted by Greg Stein <gs...@lyra.org>.
On Thu, Nov 30, 2000 at 06:13:36PM -0800, Roy T. Fielding wrote:
> On Tue, Nov 28, 2000 at 03:03:30PM -0600, William A. Rowe, Jr. wrote:
> >
> > My only q is what to do with ap_os_create_privileged_process ???
>
> It sounds like an APR function to me.
Well, the problem is that it uses SUEXEC for its operation, and that is
quite Apache specific. Many platforms just default to apr_create_process(),
but Unix-ish platforms use SUEXEC.
Not quite sure whether this would go into server/, somewhere in a srclib/
package, or part of support/ (where suexec is).
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: cvs indecision
Posted by rb...@covalent.net.
On Thu, 30 Nov 2000, Roy T. Fielding wrote:
> On Tue, Nov 28, 2000 at 03:03:30PM -0600, William A. Rowe, Jr. wrote:
> >
> > My only q is what to do with ap_os_create_privileged_process ???
>
> It sounds like an APR function to me.
This is hard to do, because on Unix this is done with a separate
program. This means that APR on Unix would be a library and a secure
program. I'm not really sure we want to do that.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: cvs indecision
Posted by "Roy T. Fielding" <fi...@ebuilt.com>.
On Tue, Nov 28, 2000 at 03:03:30PM -0600, William A. Rowe, Jr. wrote:
>
> My only q is what to do with ap_os_create_privileged_process ???
It sounds like an APR function to me.
....Roy
Re: cvs indecision
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
From: "Greg Stein" <gs...@lyra.org>
To: <ne...@apache.org>
>
> Where would src/os/ go? I would suggest server/os/. Note that it continues
> to exist because we have os-specific functionality that lives in there (for
> example, Win32 service registration and handling). This is app-level
> functionality, rather than portability stuff, so it goes here rather than in
> APR.
nak. Service registration and handling will all be in the mpm/winnt mpm tree.
Once we finish our job in apr there is -no- platform specific code remaining.
All that remains are the canon name functions I'm working on and hope to wrap
asap.
Sure, there are platform specific module (isapi) but these don't necessarily
belong in the plat tree.
My only q is what to do with ap_os_create_privileged_process ???
Re: cvs indecision
Posted by rb...@covalent.net.
> Where would src/os/ go? I would suggest server/os/. Note that it continues
> to exist because we have os-specific functionality that lives in there (for
> example, Win32 service registration and handling). This is app-level
> functionality, rather than portability stuff, so it goes here rather than in
> APR.
This needs to go away. The few things left in os/win32 should be moved to
the MPM directory or removed completely.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: cvs indecision
Posted by Greg Stein <gs...@lyra.org>.
On Mon, Nov 27, 2000 at 04:04:13AM -0800, Roy T. Fielding wrote:
>...
> What I am thinking of doing is rebuilding the repository as if it were
> a standard source tree (not a dual build/source tree), as follows:
>
> httpd-2.0/
> build/ [src/build and src/helpers]
> docs/
> docroot/ [containing index files in htdocs]
> conf/ [what is in conf now]
> icons/ [just the small ones]
> manual/ [what is now htdocs/manual]
Toss or keep cgi-bin/ ?
> include/
> modules/
> aaa/ [access authentication and auth DBs]
> generators/ [mod_asis, mod_cgid, etc.]
> filters/ [mod_include, etc.]
> loggers/ [content generators]
> mappers/ [mod_alias, mod_rewrite, etc.]
> metadata/ [mod_cern_meta, mod_user_track, etc.]
> cache/
> dav/
> proxy/
> experimental/
A protocols section? e.g. where does mod_echo go? And would we put HTTP as a
sibling of echo? (rather than server/http/ as you have below)
> server/ [src/main]
> mpm/
> http/
> ssl/
> main.c
> buildmark.c
> srclib/
> apr/ [autocheckout? via &apr-httpd-2.0]
> aputil/ [what greg just created and src/ap]
> expat-lite/ [autocheckout?]
> pcre/ [autocheckout?]
We have local mods to the above two packages, so an autocheckout isn't going
to work for these.
> sdbm/ [autocheckout?]
There is no canonical SDBM distro, let alone a CVS repository for it. The
SDBM in Apache CVS is "ours" for all intents and purposes, and falls under
our continued maintenance/development.
> openssl/ [autocheckout?]
> zlib/ [autocheckout?]
> support/
> test/
>
Where would src/os/ go? I would suggest server/os/. Note that it continues
to exist because we have os-specific functionality that lives in there (for
example, Win32 service registration and handling). This is app-level
functionality, rather than portability stuff, so it goes here rather than in
APR.
>...
> In other words, remove the vestigial directories from Apache 1.0
> (logs, htdocs, etc), eliminate the src layer, stop using the standard
> names of install directories for our own source code (conf, lib, etc.),
> segregate the modules by function rather than some arbitrary
> notion of their status, and prepare for new server protocols.
+1
>...
> Note that this is a new repository name.
To follow "convention", I presume this would be httpd-2.0.
[ oh, I see it at the root of your diagram... cool ]
>...
> Of course, the negative is that we'd have to then fix the build system.
That isn't a big deal. The failure mode is easy to diagnose and resolve.
"whoops, it didn't compile/link". Poke/prod. "ooh. looks good now."
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: cvs indecision
Posted by "Roy T. Fielding" <fi...@ebuilt.com>.
> EXcept that the API documentation is embedded in the code (.h files),
> so you thereby would either lock the documenters out, or
> else have to give them commit access to more than they have/had
> now/before.
Or they can submit patches to the include files, like anyone else.
I want to make this perfectly clear: NOBODY gets write access to the
Apache source code files without individual approval of the httpd PMC.
....Roy
Re: cvs indecision
Posted by Rodent of Unusual Size <Ke...@Golux.Com>.
"Roy T. Fielding" wrote:
>
> chgrp -R apcvs apache-2.0/src/include
> chgrp -R apcvs apache-2.0/src/os/unix
> chgrp -R apcvs apr/include
>
> [note: I believe the last three are bogus, but that is what the
> httpd-docs-2.0 module is trying to do. I think that any
> changes to the include files should require someone in the
> apr or server group to commit, so I would prefer to leave
> those directories under the core groups.]
EXcept that the API documentation is embedded in the code (.h files),
so you thereby would either lock the documenters out, or
else have to give them commit access to more than they have/had
now/before.
--
#ken P-)}
Ken Coar <http://Golux.Com/coar/>
Apache Software Foundation <http://www.apache.org/>
"Apache Server for Dummies" <http://Apache-Server.Com/>
"Apache Server Unleashed" <http://ApacheUnleashed.Com/>