Let's get rid of .htaccess files :-)

[Randy Terbush <ra...@zyzzyva.com>]

Hmmm. I think I'm on to something here....

The following patch provides a very different version of the ConfigDir
functionality that I have presented in the past. While this probably
has some bugs and shortcomings at this stage, I think it would allow
us to ditch the use of .htaccess files completely.

By adding 'ConfigDir conf/db/' to your top level config file,
the server will recursively parse that directory tree looking
for (in the case of this patch) ".oO" files and parsing them
for config directives. By wrapping the contents of these files
with <Location ..> tags, we can preconfigure the entire server
setup at startup. It could even be possible to point the ConfigDir
directive at the DocumentRoot and allow it to parse the installed
.htaccess files.

What did I forget?

Index: src/http_core.c
===================================================================
RCS file: /export/home/cvs/apache/src/http_core.c,v
retrieving revision 1.18
diff -c -r1.18 http_core.c
*** http_core.c	1996/06/12 18:14:31	1.18
--- http_core.c	1996/07/18 03:59:55
***************
*** 612,617 ****
--- 612,647 ----
      return errmsg;
  }
  
+ char *read_configdir (cmd_parms *cmd, void *dummy, char *dir)
+ {
+     char *ndir;
+     DIR *vdir;
+     struct DIR_TYPE *vdir_entry;
+     
+ 
+     dir = server_root_relative (cmd->pool, dir);
+     
+     if (!is_directory (dir))
+ 	fprintf (stderr, "ConfigDir must be a valid directory: %s", dir);
+     
+     if ((vdir = opendir (dir)) != NULL) {
+ 	while ((vdir_entry = readdir (vdir)) != NULL) {
+ 	    ndir = pstrcat (cmd->pool, dir, vdir_entry->d_name, NULL);
+ 	    if (is_directory (ndir)) {
+ 		if ((strcmp (vdir_entry->d_name, "..")) > 0) {
+ 		    ndir = pstrcat (cmd->pool, ndir, "/", NULL);
+ 		    read_configdir (cmd, dummy, ndir);
+ 		}
+ 	    }
+ 	    else if (!(strcmp(vdir_entry->d_name, ".oO")))
+ 		process_resource_config (cmd->server, ndir, cmd->pool, cmd->temp_pool);
+ 	}
+     }
+     
+     closedir (vdir);
+     return NULL;
+ }
+ 
  char *set_server_string_slot (cmd_parms *cmd, void *dummy, char *arg)
  {
      /* This one's pretty generic... */
***************
*** 849,854 ****
--- 879,885 ----
        "a port number or a numeric IP address and a port number"},
  { "<VirtualHost", virtualhost_section, NULL, RSRC_CONF, RAW_ARGS, NULL },
  { "</VirtualHost>", end_virtualhost_section, NULL, RSRC_CONF, NO_ARGS, NULL },
+ { "ConfigDir", read_configdir, NULL, RSRC_CONF, TAKE1, "directory containing config database" },
  { NULL },
  };
  

Re: Let's get rid of .htaccess files :-)

[Paul Richards <p....@elsevier.co.uk>]

rasmus@madhaus.utcs.utoronto.ca writes:
 > > The obvious optimistation to make first would be to parse them all
 > > at startup and hold them in an internal structure and then do a stat
 > > on the file at each access rather than a full parse. I suspect that in
 > > itself would be a considerable performance boost. If the file's newer
 > > than it's cached contents then re-parse it.
 > 
 > On large systems, I am not sure it is a good idea to go hog-wild like that
 > and search the entire disk for these things.  You would need to search 
 > through every ~user/public_html tree as well, and on a lot of boxes this
 > could be many thousand directories.  Perhaps an option to read a list of
 > top-level directories to search would speed this up a bit.

Ok, a simple caching mechanism then, seems we're back to the ideas that
were floating around 3-4 months ago.

Re: Let's get rid of .htaccess files :-)

[Paul Richards <p....@elsevier.co.uk>]

Randy Terbush writes:
 > 
 > You would only need to parse the tree at startup and SIGHUP.
 > It would be very easy to have a separate daemon watching the
 > conf/ for changes and sending a SIGHUP when needed. The conf/
 > could even be a DBM format file, however I prefer the idea
 > of leaving it accessible by basic system tools like 'vi'.
 > Hell, I could even imagine an Emacs mode for configuring the
 > server! :-)

Hmm, I like this idea, only why bother with lots of files. Why not just
have on access.conf file that has auth information for a URL (with regex
matching). This would be parsed once at startup so it's complexity is
less of an issue and if you make a change you have to re-hup. You could
also provide an online tool to edit the internal access table directly
for immediate changes and have the server write out a new access file 
if you wished (or periodically perhaps, make it all configurable).

This would be a really nice feature, if it's suitably abstracted the
access file can be replaced by anything we want, like an Oracle database
which is what I'm doing a lot of work with at the moment.

In fact, authentication is top of my agenda at the moment, I need to do
far more complex things than just access authentication, such as credit
accounting and so forth, different access to different bits of
information that are all served up in a single dynamically generated
page using cgi etc.

The "standard" access methods are woefully inedaquate.

Re: Let's get rid of .htaccess files :-)

[ra...@madhaus.utcs.utoronto.ca]

>  > I guess the question would then be as to how you identify .htaccess
>  > files. Do you look for them at startup? If you have a lot of
>  > directories, that could take a while. I suppose it could read each in
>  > once, the first time. (it'd have to also store then which directories
>  > *didn't* have .htaccess files, to avoid the extra stat for each request).
> 
> The obvious optimistation to make first would be to parse them all
> at startup and hold them in an internal structure and then do a stat
> on the file at each access rather than a full parse. I suspect that in
> itself would be a considerable performance boost. If the file's newer
> than it's cached contents then re-parse it.

On large systems, I am not sure it is a good idea to go hog-wild like that
and search the entire disk for these things.  You would need to search 
through every ~user/public_html tree as well, and on a lot of boxes this
could be many thousand directories.  Perhaps an option to read a list of
top-level directories to search would speed this up a bit.

-Rasmus

Re: Let's get rid of .htaccess files :-)

[Paul Richards <p....@elsevier.co.uk>]

Alexei Kosut writes:
 > 
 > I guess the question would then be as to how you identify .htaccess files.
 > Do you look for them at startup? If you have a lot of directories, that
 > could take a while. I suppose it could read each in once, the first time.
 > (it'd have to also store then which directories *didn't* have .htaccess
 > files, to avoid the extra stat for each request).

The obvious optimistation to make first would be to parse them all
at startup and hold them in an internal structure and then do a stat
on the file at each access rather than a full parse. I suspect that in
itself would be a considerable performance boost. If the file's newer
than it's cached contents then re-parse it.

Re: Let's get rid of .htaccess files :-)

[Alexei Kosut <ak...@organic.com>]

The problem with .htaccess files, as I understand them, is that they need
to be searched for and parsed for each and every request. This is
neccessarily not a great things for heavily loaded servers.

On the other hand, they make it a lot easier to reconfigure the server,
directory by directory, and if someone doesn't have access to the server
config, they can still make an .htaccess file, and do whatever they want
in it.

How about an option that would let you find and read in all the
.htaccess files at startup, pretending they were <Directory> sections.
That way, they'd all be there, ready to go. It would never have to check
for them again. If they got changed, you'd have to provide some way to
restart the server, but combined with Ben's graceful restart patches, this
might not be a bad thing. At least as an option.

I guess the question would then be as to how you identify .htaccess files.
Do you look for them at startup? If you have a lot of directories, that
could take a while. I suppose it could read each in once, the first time.
(it'd have to also store then which directories *didn't* have .htaccess
files, to avoid the extra stat for each request).

Hmm.

-- Alexei Kosut <ak...@organic.com>            The Apache HTTP Server 
   http://www.nueva.pvt.k12.ca.us/~akosut/      http://www.apache.org/