You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Graham Leggett <mi...@sharp.fm> on 2001/08/15 17:34:17 UTC
Apache config files and alternate config sources
Hi all,
Part two in the grand LDAP plan is to support the storing of
configuration data in an LDAP directory, somewhat along the lines of
what has been achieved with qmail+ldap+control. This allows multiple
machines (probably in a redundant configuration) to derive their config
from a common datasource rather than relying on files on each machine's
filesystem. It also allows the LDAP schema checking mechanism to do a
kind of syntax checking on the LDAP config, which ensures that a bogus
Apache config cannot be applied in the first place - a Good Thing(TM) :)
In order to achieve this technically, I have a number of questions.
It's been a while since I looked at the way the config files are read in
v2.0 - so my understanding could be way off - however as I understand it
a config file is opened by Apache, and it is read line by line, feeding
non-comment lines to be processed by the config subsystem. The Include
directive is handled roughly as a kind of recursion, where a new file is
opened and the loop continued.
To support LDAP (and potentially other methods, such as the Windows
Registry), I propose a new hook. This would call one or more functions
whose job it is to recognise the filename/urlname of the configuration,
read that object in, and then render each line of config appropriately
as it is done now. For example the "file" module would recognise the URL
"conf/httpd.conf", while the "ldap" module would recognise URL
"ldap:///blah" and the "registry" module would recognise the URL
"registry:///some/key".
A default config file on an Apache system might include the line:
"Include ldap:///blah" which includes an LDAP object at that point, or
Apache could be started with "httpd -f ldap:///blah".
Does this sound like the right approach?
Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm "There's a moon
over Bourbon Street
tonight..."
Re: Apache config files and alternate config sources
Posted by Graham Leggett <mi...@sharp.fm>.
Ryan Bloom wrote:
> This has been discussed before, and was the original goal for the
> configuration tree approach.
Ok.
> The decision in the past has always been
> that Apache config files are stored text files. If you want to store them
> somewhere else, then create a pre-processor to grab the config data from
> some other location, and store it in a text file.
This will work - perhaps such a tool could be made and put in the
support directory.
This sounds good for LDAP, because a downed LDAP server will not affect
the Apache servers if the config files are all pulled and made text, but
it seems very klunky for supporting the windows registry.
> In fact, this idea was vetoed
> the last time Greg Stein and I brought it up. Take a look through the list
> archives from just after the London ApacheCon, at about the time that the
> util_cfgtree.c file was added to CVS.
I remember this discussion, but not the outcome. Will hunt down the
discussion in the archives...
Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm "There's a moon
over Bourbon Street
tonight..."
Re: Apache config files and alternate config sources
Posted by Ryan Bloom <rb...@covalent.net>.
On Thursday 16 August 2001 02:46, Graham Leggett wrote:
> Ryan Bloom wrote:
> > If you use a pre-processor, then you can make a sane decision. If the
> > LDAP server isn't responding, don't tell Apache to restart. If you get
> > half-way through updating the config file on disk, don't try to restart
> > Apache. If you mix the two, you open yourself up to more problems during
> > error conditions.
> >
> > The code is easier if we keep these things separate, and we allow for a
> > more robust server IMNSHO.
>
> Hmmmm... you're 100% right - but I'm trying to figure out a way for all
> of this to work elegantly that does not require any user confusion.
>
> What could be done is that such a preprocessor stage be built into the
> apachectl script. The graceful restart process would then be this:
>
> 1) run preprocessor on httpd.conf and generate a new file called
> httpd.conf-<date> containing all registry keys, LDAP objects, external
> files, whatever. If this fails - throw an error and don't restart.
>
> 2) run "httpd -f httpd.conf-<date> -t" to check if the syntax is
> correct. If this fails - throw an error and don't restart.
>
> 3) run actual restart or graceful restart.
>
> This way the user does not have to know or care about the checking
> process - they make a config file that has Include directives that point
> to either a windows registry or LDAP server and everything magically
> works. There is no change to the Apache webserver code, only the
> apachectl script.
>
> The <date> bit allows a bit of audit trail to be created, but if this is
> too much having a single file is fine. Thoughts?
++1.
Ryan
______________________________________________________________
Ryan Bloom rbb@apache.org
Covalent Technologies rbb@covalent.net
--------------------------------------------------------------
Re: Apache config files and alternate config sources
Posted by Graham Leggett <mi...@sharp.fm>.
Ryan Bloom wrote:
> If you use a pre-processor, then you can make a sane decision. If the LDAP
> server isn't responding, don't tell Apache to restart. If you get half-way through
> updating the config file on disk, don't try to restart Apache. If you mix the two,
> you open yourself up to more problems during error conditions.
> The code is easier if we keep these things separate, and we allow for a more
> robust server IMNSHO.
Hmmmm... you're 100% right - but I'm trying to figure out a way for all
of this to work elegantly that does not require any user confusion.
What could be done is that such a preprocessor stage be built into the
apachectl script. The graceful restart process would then be this:
1) run preprocessor on httpd.conf and generate a new file called
httpd.conf-<date> containing all registry keys, LDAP objects, external
files, whatever. If this fails - throw an error and don't restart.
2) run "httpd -f httpd.conf-<date> -t" to check if the syntax is
correct. If this fails - throw an error and don't restart.
3) run actual restart or graceful restart.
This way the user does not have to know or care about the checking
process - they make a config file that has Include directives that point
to either a windows registry or LDAP server and everything magically
works. There is no change to the Apache webserver code, only the
apachectl script.
The <date> bit allows a bit of audit trail to be created, but if this is
too much having a single file is fine. Thoughts?
Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm "There's a moon
over Bourbon Street
tonight..."
Re: Apache config files and alternate config sources
Posted by Ryan Bloom <rb...@covalent.net>.
On Wednesday 15 August 2001 11:11, Graham Leggett wrote:
> Ryan Bloom wrote:
> > This has been discussed before, and was the original goal for the
> > configuration tree approach. The decision in the past has always been
> > that Apache config files are stored text files. If you want to store
> > them somewhere else, then create a pre-processor to grab the config data
> > from some other location, and store it in a text file. In fact, this
> > idea was vetoed the last time Greg Stein and I brought it up. Take a
> > look through the list archives from just after the London ApacheCon, at
> > about the time that the util_cfgtree.c file was added to CVS.
>
> I took a look at the archives, and specifically the "Configuration
> modules..." thread. It seems the discussion started with the suggestion
> for config to be made modular, and a roadmap was plotted for doing it.
> Wouldn't-it-be-nice-if scope creep set in, and suddenly the discussion
> turned to making the config XML based. Then the suggestion that Apache
> configs could be made in different languages, and all hell broke
> loose...
>
> To rewind back from that, what I am trying to propose is extremely
> simple:
>
> o Apache wants config directives in the form "ApacheDirective [Value]\n"
> or "<ApacheContainer [Value]>\n", one by one in a long list.
>
> o A simple-config-module hook gets called by the root of the config, and
> by the Include directive that says "hook - give me some config
> statements for this URL/file".
>
> o The module opens the file / fetches the LDAP object / reads the
> registry, creates lines from this info that look like "ApacheDirective
> [Value]\n" or "<ApacheContainer [Value]>\n"
>
> o Each of these lines is handed to the core config, which does all the
> magic.
>
> The above is pretty much how it works now (or it did when I last looked
> at it), only the loop that reads each line from a file is not in a
> module :)
>
> I'd like to start getting some code together - is it worth me putting
> something like this together? I think that the concerns raised in the
> 2000/04 thread are different to what I am proposing here.
I disagree that the concerns are different. The web server doesn't need to
know how to contact an LDAP directory. If you want to store the config in a
different place, do it with a pre-processor. This also allows us to handle
graceful restarts better.
What happens in the instance that a module tries to read the config from an
LDAP server during a restart, if the LDAP server has died? By the time Apache
knows the LDAP server has died, we have already cleared the pconf pool,
and started killing child processes. That means we can't just continue to
use the old config. We also can't get the new config, so the server stops
working.
If you use a pre-processor, then you can make a sane decision. If the LDAP
server isn't responding, don't tell Apache to restart. If you get half-way through
updating the config file on disk, don't try to restart Apache. If you mix the two,
you open yourself up to more problems during error conditions.
The code is easier if we keep these things separate, and we allow for a more
robust server IMNSHO.
Ryan
______________________________________________________________
Ryan Bloom rbb@apache.org
Covalent Technologies rbb@covalent.net
--------------------------------------------------------------
Re: Apache config files and alternate config sources
Posted by Graham Leggett <mi...@sharp.fm>.
Ryan Bloom wrote:
> This has been discussed before, and was the original goal for the
> configuration tree approach. The decision in the past has always been
> that Apache config files are stored text files. If you want to store them
> somewhere else, then create a pre-processor to grab the config data from
> some other location, and store it in a text file. In fact, this idea was vetoed
> the last time Greg Stein and I brought it up. Take a look through the list
> archives from just after the London ApacheCon, at about the time that the
> util_cfgtree.c file was added to CVS.
I took a look at the archives, and specifically the "Configuration
modules..." thread. It seems the discussion started with the suggestion
for config to be made modular, and a roadmap was plotted for doing it.
Wouldn't-it-be-nice-if scope creep set in, and suddenly the discussion
turned to making the config XML based. Then the suggestion that Apache
configs could be made in different languages, and all hell broke
loose...
To rewind back from that, what I am trying to propose is extremely
simple:
o Apache wants config directives in the form "ApacheDirective [Value]\n"
or "<ApacheContainer [Value]>\n", one by one in a long list.
o A simple-config-module hook gets called by the root of the config, and
by the Include directive that says "hook - give me some config
statements for this URL/file".
o The module opens the file / fetches the LDAP object / reads the
registry, creates lines from this info that look like "ApacheDirective
[Value]\n" or "<ApacheContainer [Value]>\n"
o Each of these lines is handed to the core config, which does all the
magic.
The above is pretty much how it works now (or it did when I last looked
at it), only the loop that reads each line from a file is not in a
module :)
I'd like to start getting some code together - is it worth me putting
something like this together? I think that the concerns raised in the
2000/04 thread are different to what I am proposing here.
Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm "There's a moon
over Bourbon Street
tonight..."
Re: Apache config files and alternate config sources
Posted by Ryan Bloom <rb...@covalent.net>.
This has been discussed before, and was the original goal for the
configuration tree approach. The decision in the past has always been
that Apache config files are stored text files. If you want to store them
somewhere else, then create a pre-processor to grab the config data from
some other location, and store it in a text file. In fact, this idea was vetoed
the last time Greg Stein and I brought it up. Take a look through the list
archives from just after the London ApacheCon, at about the time that the
util_cfgtree.c file was added to CVS.
Ryan
On Wednesday 15 August 2001 08:34, Graham Leggett wrote:
> Hi all,
>
> Part two in the grand LDAP plan is to support the storing of
> configuration data in an LDAP directory, somewhat along the lines of
> what has been achieved with qmail+ldap+control. This allows multiple
> machines (probably in a redundant configuration) to derive their config
> from a common datasource rather than relying on files on each machine's
> filesystem. It also allows the LDAP schema checking mechanism to do a
> kind of syntax checking on the LDAP config, which ensures that a bogus
> Apache config cannot be applied in the first place - a Good Thing(TM) :)
>
> In order to achieve this technically, I have a number of questions.
>
> It's been a while since I looked at the way the config files are read in
> v2.0 - so my understanding could be way off - however as I understand it
> a config file is opened by Apache, and it is read line by line, feeding
> non-comment lines to be processed by the config subsystem. The Include
> directive is handled roughly as a kind of recursion, where a new file is
> opened and the loop continued.
>
> To support LDAP (and potentially other methods, such as the Windows
> Registry), I propose a new hook. This would call one or more functions
> whose job it is to recognise the filename/urlname of the configuration,
> read that object in, and then render each line of config appropriately
> as it is done now. For example the "file" module would recognise the URL
> "conf/httpd.conf", while the "ldap" module would recognise URL
> "ldap:///blah" and the "registry" module would recognise the URL
> "registry:///some/key".
>
> A default config file on an Apache system might include the line:
> "Include ldap:///blah" which includes an LDAP object at that point, or
> Apache could be started with "httpd -f ldap:///blah".
>
> Does this sound like the right approach?
>
> Regards,
> Graham
----------------------------------------
Content-Type: application/x-pkcs7-signature; charset="us-ascii"; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Description: S/MIME Cryptographic Signature
----------------------------------------
--
______________________________________________________________
Ryan Bloom rbb@apache.org
Covalent Technologies rbb@covalent.net
--------------------------------------------------------------
Re: Apache config files and alternate config sources
Posted by Graham Leggett <mi...@sharp.fm>.
Aaron Bannert wrote:
> The attack is the same, but the result is different. Named virtual hosts
> only really affect how the client contacts the server, and everything
> else happens in HTTP (in the Host: header). You can not prevent someone
> from altering their own DNS entries maliciously. OTOH, DNS-trusted
> runtime-config would allow an attacker to configure your httpd with
> whatever LDAP config they wanted, including SuEXEC, piped logs, etc.
You're 100% correct - which is why your network would be suitably
secured with private networks, connections based on IP address or names
defined in /etc/hosts, all the provisions normally installed at any
secure LDAP based email installation. :)
Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm "There's a moon
over Bourbon Street
tonight..."
Re: Apache config files and alternate config sources
Posted by Aaron Bannert <aa...@clove.org>.
On Wed, Aug 15, 2001 at 06:08:56PM +0200, Graham Leggett wrote:
> Aaron Bannert wrote:
>
> > One of the biggest dangers in this kind of a thing (and it is rather similiar
> > to depending on a remote DTD in XML) is that you are now implicitly trusting
> > DNS for authenticity. A poisoned DNS entry could be catastrophic.
>
> We face that problem today with the virtualhost directives, etc (any
> directive that can take a DNS name as an argument).
The attack is the same, but the result is different. Named virtual hosts
only really affect how the client contacts the server, and everything
else happens in HTTP (in the Host: header). You can not prevent someone
from altering their own DNS entries maliciously. OTOH, DNS-trusted
runtime-config would allow an attacker to configure your httpd with
whatever LDAP config they wanted, including SuEXEC, piped logs, etc.
This is really just a hypothetical demonstration, I may be going overboard
here. :)
-aaron
Re: Apache config files and alternate config sources
Posted by Graham Leggett <mi...@sharp.fm>.
Aaron Bannert wrote:
> One of the biggest dangers in this kind of a thing (and it is rather similiar
> to depending on a remote DTD in XML) is that you are now implicitly trusting
> DNS for authenticity. A poisoned DNS entry could be catastrophic.
We face that problem today with the virtualhost directives, etc (any
directive that can take a DNS name as an argument).
> To me it sounds like the main thing we are trying to accomplish here is
> to allow for centralized configuration, which is useful in things like
> server farms or for rapid deployment of cloned or slightly mutated
> configurations. What other things are we trying to solve with this?
This is exactly what it is for.
Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm "There's a moon
over Bourbon Street
tonight..."
Re: Apache config files and alternate config sources
Posted by Aaron Bannert <aa...@clove.org>.
On Wed, Aug 15, 2001 at 05:34:17PM +0200, Graham Leggett wrote:
> Hi all,
>
> Part two in the grand LDAP plan is to support the storing of
> configuration data in an LDAP directory, somewhat along the lines of
> what has been achieved with qmail+ldap+control. This allows multiple
> machines (probably in a redundant configuration) to derive their config
> from a common datasource rather than relying on files on each machine's
> filesystem. It also allows the LDAP schema checking mechanism to do a
> kind of syntax checking on the LDAP config, which ensures that a bogus
> Apache config cannot be applied in the first place - a Good Thing(TM) :)
>
> In order to achieve this technically, I have a number of questions.
>
> It's been a while since I looked at the way the config files are read in
> v2.0 - so my understanding could be way off - however as I understand it
> a config file is opened by Apache, and it is read line by line, feeding
> non-comment lines to be processed by the config subsystem. The Include
> directive is handled roughly as a kind of recursion, where a new file is
> opened and the loop continued.
>
> To support LDAP (and potentially other methods, such as the Windows
> Registry), I propose a new hook. This would call one or more functions
> whose job it is to recognise the filename/urlname of the configuration,
> read that object in, and then render each line of config appropriately
> as it is done now. For example the "file" module would recognise the URL
> "conf/httpd.conf", while the "ldap" module would recognise URL
> "ldap:///blah" and the "registry" module would recognise the URL
> "registry:///some/key".
>
> A default config file on an Apache system might include the line:
> "Include ldap:///blah" which includes an LDAP object at that point, or
> Apache could be started with "httpd -f ldap:///blah".
>
> Does this sound like the right approach?
One of the biggest dangers in this kind of a thing (and it is rather similiar
to depending on a remote DTD in XML) is that you are now implicitly trusting
DNS for authenticity. A poisoned DNS entry could be catastrophic.
To me it sounds like the main thing we are trying to accomplish here is
to allow for centralized configuration, which is useful in things like
server farms or for rapid deployment of cloned or slightly mutated
configurations. What other things are we trying to solve with this?
-aaron