You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Jan Lehnardt <ja...@apache.org> on 2011/07/18 18:14:43 UTC

Configuration Load Order

Hey all,

we found an oddity with the way we read config files and directories on startup.

From the 0.10.0 docs (http://wiki.apache.org/couchdb/Breaking_changes#Modular_Configuration_Directories):

> CouchDB now loads configuration from the following places (glob(7) syntax) in order:
> 
> 	• PREFIX/default.ini
> 	• PREFIX/default.d/*
> 	• PREFIX/local.ini
> 	• PREFIX/local.d/*


The way I've understand the config system is that system defaults go into default.ini and default.d/* and local overrides go into local.ini and local.d/*.

The reason (I believe) *.d/ exist is that in a distribution some special configuration can be added. For the sake of example, I think GeoCouch is a good example to think about this. A CouchDB installation with GeoCouch could have the following config files:

$etcdir/default.ini # system defaults
$etcdir/default.d/geocouch.ini # geocouch system defaults
$etcdir/local.ini # user settings overriding defaults in default.ini
$etcdir/local.d/geocouch.ini # user settings overriding defaults in default.d/geocouch.ini.

Now if one uses the /_config API, new settings are stored in local.d/geocouch.ini rather than local.ini (as I'd expect).

I'd therefore consider the load order to be wrong and propose to change it so that local.ini is the very last item in the config chain.

Alternatively, I'd like to know that there are flaws in my assumptions above :)

Cheers
Jan
-- 



Re: Configuration Load Order

Posted by Robert Newson <rn...@apache.org>.
+1

On 18 July 2011 17:14, Jan Lehnardt <ja...@apache.org> wrote:
> Hey all,
>
> we found an oddity with the way we read config files and directories on startup.
>
> From the 0.10.0 docs (http://wiki.apache.org/couchdb/Breaking_changes#Modular_Configuration_Directories):
>
>> CouchDB now loads configuration from the following places (glob(7) syntax) in order:
>>
>>       • PREFIX/default.ini
>>       • PREFIX/default.d/*
>>       • PREFIX/local.ini
>>       • PREFIX/local.d/*
>
>
> The way I've understand the config system is that system defaults go into default.ini and default.d/* and local overrides go into local.ini and local.d/*.
>
> The reason (I believe) *.d/ exist is that in a distribution some special configuration can be added. For the sake of example, I think GeoCouch is a good example to think about this. A CouchDB installation with GeoCouch could have the following config files:
>
> $etcdir/default.ini # system defaults
> $etcdir/default.d/geocouch.ini # geocouch system defaults
> $etcdir/local.ini # user settings overriding defaults in default.ini
> $etcdir/local.d/geocouch.ini # user settings overriding defaults in default.d/geocouch.ini.
>
> Now if one uses the /_config API, new settings are stored in local.d/geocouch.ini rather than local.ini (as I'd expect).
>
> I'd therefore consider the load order to be wrong and propose to change it so that local.ini is the very last item in the config chain.
>
> Alternatively, I'd like to know that there are flaws in my assumptions above :)
>
> Cheers
> Jan
> --
>
>
>

Re: Configuration Load Order

Posted by Benoit Chesneau <bc...@gmail.com>.
On Tue, Jul 19, 2011 at 12:29 AM, Jan Lehnardt <ja...@apache.org> wrote:
>
> On 18 Jul 2011, at 20:42, Noah Slater wrote:
>>> What is wrong is the place where (some) _writes_ go. Don't know if possible,
>>> but, continuing in your example, changes in geocouch config should go to
>>> geocouch.ini, other changes to local.ini. Or maybe changes to existing key
>>> should go to the file from where it was effectively read, new keys to
>>> local.ini. Or maybe this is way too complicated :-)
>>
>> Also agreed.
>>
>> There is obviously a problem in where we write back to. But I don't necessarily think this means we need to break implicit compatibility with system behaviour expectations. Perhaps a way to solve it would be to write out to a generated.ini file that is loaded last.
>
> My understanding is that local.ini is that generated.ini.
>
> Ideally, we'd track which config setting came from which file and write to the appropriate "local" counterpart, but I'm not willing to write the necessary code for that.
>
> I'm willing to patch the read order to make sure changes get written to local.ini however.
>
> Cheers
> Jan
> --
>
>
>

Maybe generated conf could go in a couch.cfg file to make clear
distinction between what we can edit (and local.ini is one) and others
?

- benoît

Re: Configuration Load Order

Posted by Dave Cottlehuber <da...@muse.net.nz>.
On 18 August 2011 00:47, Robert Newson <rn...@apache.org> wrote:
> Oh, that's not what I'm +1 for then.
>
> I specifically mean that passwords are never held in any .ini file,
> but instead in a separate password file that contains nothing but
> passwords, and never plaintext passwords either. It is updated with a
> script which takes username and password, crunches it, and writes it
> to the file. I used 'htpasswd' as shorthand for all of the above but
> clearly was too terse, sorry.

+1 to htpasswd. self-mutilation is not a good look & contravenes
principle of least surprise.

> I quite like your proposal too, though, where only bootstrapping
> (database_dir, etc) are in config and the rest can be read from a
> special _config database (thus eliminating any manual alterations from
> outside _config).
>
> B.

+0.
I support minimal ini entries for bootstrapping. Personally when
admining stuff I used svn; storing in couch makes that a script
instead of a text file.  It's already easy for people to lose DB
access by fiddling with rewrite rules; we need to ensure its
reversable and intuitive in case of self-inflicted muppet moments. I'm
not convinced the coolness of having everything in a _config.couch
outweighs the convenience of vi on .ini.

A+
Dave

> On 17 August 2011 12:29, Noah Slater <ns...@apache.org> wrote:
>>
>> On 16 Aug 2011, at 23:07, Robert Newson wrote:
>>
>>> nice idea to have a separate htpasswd (-like) file. Passwords are
>>> special, let's treat them accordingly.
>>
>> Just to clarify.
>>
>> The proposal you're commenting on only suggest a separate password script that generates hashes which are then put into the regular ini files.
>

Re: Configuration Load Order

Posted by Robert Newson <rn...@apache.org>.
Oh, that's not what I'm +1 for then.

I specifically mean that passwords are never held in any .ini file,
but instead in a separate password file that contains nothing but
passwords, and never plaintext passwords either. It is updated with a
script which takes username and password, crunches it, and writes it
to the file. I used 'htpasswd' as shorthand for all of the above but
clearly was too terse, sorry.

I quite like your proposal too, though, where only bootstrapping
(database_dir, etc) are in config and the rest can be read from a
special _config database (thus eliminating any manual alterations from
outside _config).

B.

On 17 August 2011 12:29, Noah Slater <ns...@apache.org> wrote:
>
> On 16 Aug 2011, at 23:07, Robert Newson wrote:
>
>> nice idea to have a separate htpasswd (-like) file. Passwords are
>> special, let's treat them accordingly.
>
> Just to clarify.
>
> The proposal you're commenting on only suggest a separate password script that generates hashes which are then put into the regular ini files.

Re: Configuration Load Order

Posted by Noah Slater <ns...@apache.org>.
On 16 Aug 2011, at 23:07, Robert Newson wrote:

> nice idea to have a separate htpasswd (-like) file. Passwords are
> special, let's treat them accordingly.

Just to clarify.

The proposal you're commenting on only suggest a separate password script that generates hashes which are then put into the regular ini files.

Re: Configuration Load Order

Posted by Robert Newson <rn...@apache.org>.
nice idea to have a separate htpasswd (-like) file. Passwords are
special, let's treat them accordingly.

B.

On 16 August 2011 23:03, Randall Leeds <ra...@gmail.com> wrote:
> On Tue, Aug 16, 2011 at 11:33, Jan Lehnardt <ja...@apache.org> wrote:
>
>>
>> On Aug 16, 2011, at 8:31 PM, Noah Slater wrote:
>>
>> >
>> > On 16 Aug 2011, at 10:33, Benoit Chesneau wrote:
>> >
>> >> Imo we shouldn't at all provide plaintext passwords. Maybe a safer
>> >> option would be to let the admin create the first one via http or put
>> >> the hash in the a password.ini file manually. If we are enough kind we
>> >> could also provide a couchctl script allowing user management, config
>> >> changes ... ?
>> >
>> > This sounds like a decent proposal. Much like you have to use htpasswd to
>> generate passwords for Apache httpd, we could bundle a script that lets you
>> generate passwords for the CouchDB ini files, and then forbid the use of
>> plaintext. This solves both the technical problem (I think?) and helps us
>> re-enforce better security practices across the board.
>>
>> Agreed.
>>
>>
> Agreed also. We still have a question about load and save order.
> One idea would be to track the .ini file from whence an option came. If an
> option comes from a local.ini or local.d/ file it could be updated in place.
> If it comes from a default.ini or default.d/ file, updates should be placed
> in local.ini. This would make the most sense to me.
>
> I would also be in favor of enforcing a load order that supports a directory
> structure like:
> local.d/
>  010-stuff.ini
>  020-others.ini
>
> We don't need to ship anything like that by default. I think right now we
> take the load directories on the command line, no? It'd be nice if the order
> of resolution within those directories was well specified.
>
> -Randall
>

Re: Configuration Load Order

Posted by Noah Slater <ns...@apache.org>.
On 16 Aug 2011, at 23:03, Randall Leeds wrote:

> Agreed also. We still have a question about load and save order.
> One idea would be to track the .ini file from whence an option came. If an
> option comes from a local.ini or local.d/ file it could be updated in place.
> If it comes from a default.ini or default.d/ file, updates should be placed
> in local.ini. This would make the most sense to me.

I think our previous discussion holds on this.

CouchDB should not be modifying default.ini, default.d/*, local.ini, or local.d/* at all. Anything CouchDB needs to write back to the config files should be put in runtime.ini, or runtime.d/* only. Sorry, I couldn't help giving the whole "generated" thing a new name while I was at it. In this case, it means that someone might put a password hash in local.ini, and CouchDB would later write out an updated password hash to runtime.ini, but that is fine. The runtime.ini file is loaded after local.ini, so as long as it is in place, the setting there would override any previous values.

To be honest, getting back to a previous sub-thread of this thread, if CouchDB is needing to write back at all, that might be a good indication that the config value should be held in CouchDB in the first place. If you look at how MySQL uses its config files, for example, you see that all the values there are values that the database needs BEFORE IT INITIALISES and is ready to serve requests. Anything else is kept within the MySQL instance itself. If we were going to remove anything from the ini files that isn't needed for database initialisation, what would we loose? I am guessing the users and passwords would go, which would obviate this issue. MySQL keeps its user information in MySQL.


> I would also be in favor of enforcing a load order that supports a directory
> structure like:
> local.d/
>  010-stuff.ini
>  020-others.ini
> 
> We don't need to ship anything like that by default. I think right now we
> take the load directories on the command line, no? It'd be nice if the order
> of resolution within those directories was well specified.

This is how the system is designed.

The code that loads files from a directory is:

    for file in "$1"/*.ini; do
        if [ -r "$file" ]; then
          _add_config_file "$file"
        fi
    done

This should list files in some alphanum sort order, as demonstrated:

nslater@tumbolia:~$ echo test/*
test/b test/c test/d test/e test/f test/h test/i test/j

This means that with an example full config directory, the load order should look like:

default.ini
default.d/001_foo.ini
default.d/002_bar.ini
local.ini
local.d/001_foo.ini
local.d/002_bar.ini
runtime.ini


Re: Configuration Load Order

Posted by Robert Newson <rn...@apache.org>.
It makes sense to perform the PUT to _config, yes, though I was
thinking it just updated the file. As long as the PUT in_ config for
passwords explicitly writes to a dedicated password file, not any of
the .ini files. Or something.

B.

On 17 August 2011 18:15, Noah Slater <ns...@apache.org> wrote:
> To be clear, is the suggestion that this script simply be a wrapper around updating the password via CouchDB's HTTP API? Having gone back and forth on this thread for the past few days, I think it makes sense to keep user info in CouchDB, and not in the ini files. If a user manages to lock themselves out of the server, they can start it with --admin-party and disable the auth system while they fix things up.
>
> On 17 Aug 2011, at 17:09, Robert Newson wrote:
>
>> It's also the sort of thing one would expect in a Debian postinst via Dialog.
>>
>> B.
>>
>> On 17 August 2011 17:04, Benoit Chesneau <bc...@gmail.com> wrote:
>>> On Wednesday, August 17, 2011, Jason Smith <jh...@iriscouch.com> wrote:
>>>> On Wed, Aug 17, 2011 at 10:22 PM, Robert Newson <rn...@apache.org>
>>> wrote:
>>>>> Jason,
>>>>>
>>>>> The --set-password thing is to ensure there are no plaintext passwords
>>>>> in the first place, which eliminates the oddness of couch rewriting a
>>>>> plaintext pwd to a digested pwd (and putting the output in a different
>>>>> file).
>>>>
>>>> Thanks for the clarification.
>>>>
>>>> If you can read a plaintext password from an .ini file, then you can
>>>> hit the HTTP API as the admin and make changes to the couch. So that
>>>> is privilege escalation.
>>>>
>>>> To answer Benoit's question, it is simpler to tell admins to use the
>>>> HTTP API (or Futon) to create the admin account. The password is
>>>> stored *somewhere* under the hood. IMHO it is less simple to add a
>>>> command-line tool as a requirement (or worse, as an alternative
>>>> option) to deploy Couch.
>>>>
>>>> --
>>> it all depends if you admin via a console.
>>>
>>> couchctl set-password username is a way easier than curl -XPUT
>>> http://blah/_users -D... -H...  . at the end if you are a good admin you
>>> will write this script. providing useful helpere don't break the kiss way
>>> here.
>>>
>>> benoît
>>>
>
>

Re: Configuration Load Order

Posted by Noah Slater <ns...@apache.org>.
To be clear, is the suggestion that this script simply be a wrapper around updating the password via CouchDB's HTTP API? Having gone back and forth on this thread for the past few days, I think it makes sense to keep user info in CouchDB, and not in the ini files. If a user manages to lock themselves out of the server, they can start it with --admin-party and disable the auth system while they fix things up.

On 17 Aug 2011, at 17:09, Robert Newson wrote:

> It's also the sort of thing one would expect in a Debian postinst via Dialog.
> 
> B.
> 
> On 17 August 2011 17:04, Benoit Chesneau <bc...@gmail.com> wrote:
>> On Wednesday, August 17, 2011, Jason Smith <jh...@iriscouch.com> wrote:
>>> On Wed, Aug 17, 2011 at 10:22 PM, Robert Newson <rn...@apache.org>
>> wrote:
>>>> Jason,
>>>> 
>>>> The --set-password thing is to ensure there are no plaintext passwords
>>>> in the first place, which eliminates the oddness of couch rewriting a
>>>> plaintext pwd to a digested pwd (and putting the output in a different
>>>> file).
>>> 
>>> Thanks for the clarification.
>>> 
>>> If you can read a plaintext password from an .ini file, then you can
>>> hit the HTTP API as the admin and make changes to the couch. So that
>>> is privilege escalation.
>>> 
>>> To answer Benoit's question, it is simpler to tell admins to use the
>>> HTTP API (or Futon) to create the admin account. The password is
>>> stored *somewhere* under the hood. IMHO it is less simple to add a
>>> command-line tool as a requirement (or worse, as an alternative
>>> option) to deploy Couch.
>>> 
>>> --
>> it all depends if you admin via a console.
>> 
>> couchctl set-password username is a way easier than curl -XPUT
>> http://blah/_users -D... -H...  . at the end if you are a good admin you
>> will write this script. providing useful helpere don't break the kiss way
>> here.
>> 
>> benoît
>> 


Re: Configuration Load Order

Posted by Robert Newson <rn...@apache.org>.
It's also the sort of thing one would expect in a Debian postinst via Dialog.

B.

On 17 August 2011 17:04, Benoit Chesneau <bc...@gmail.com> wrote:
> On Wednesday, August 17, 2011, Jason Smith <jh...@iriscouch.com> wrote:
>> On Wed, Aug 17, 2011 at 10:22 PM, Robert Newson <rn...@apache.org>
> wrote:
>>> Jason,
>>>
>>> The --set-password thing is to ensure there are no plaintext passwords
>>> in the first place, which eliminates the oddness of couch rewriting a
>>> plaintext pwd to a digested pwd (and putting the output in a different
>>> file).
>>
>> Thanks for the clarification.
>>
>> If you can read a plaintext password from an .ini file, then you can
>> hit the HTTP API as the admin and make changes to the couch. So that
>> is privilege escalation.
>>
>> To answer Benoit's question, it is simpler to tell admins to use the
>> HTTP API (or Futon) to create the admin account. The password is
>> stored *somewhere* under the hood. IMHO it is less simple to add a
>> command-line tool as a requirement (or worse, as an alternative
>> option) to deploy Couch.
>>
>> --
> it all depends if you admin via a console.
>
> couchctl set-password username is a way easier than curl -XPUT
> http://blah/_users -D... -H...  . at the end if you are a good admin you
> will write this script. providing useful helpere don't break the kiss way
> here.
>
> benoît
>

Re: Configuration Load Order

Posted by Benoit Chesneau <bc...@gmail.com>.
On Wednesday, August 17, 2011, Jason Smith <jh...@iriscouch.com> wrote:
> On Wed, Aug 17, 2011 at 10:22 PM, Robert Newson <rn...@apache.org>
wrote:
>> Jason,
>>
>> The --set-password thing is to ensure there are no plaintext passwords
>> in the first place, which eliminates the oddness of couch rewriting a
>> plaintext pwd to a digested pwd (and putting the output in a different
>> file).
>
> Thanks for the clarification.
>
> If you can read a plaintext password from an .ini file, then you can
> hit the HTTP API as the admin and make changes to the couch. So that
> is privilege escalation.
>
> To answer Benoit's question, it is simpler to tell admins to use the
> HTTP API (or Futon) to create the admin account. The password is
> stored *somewhere* under the hood. IMHO it is less simple to add a
> command-line tool as a requirement (or worse, as an alternative
> option) to deploy Couch.
>
> --
it all depends if you admin via a console.

couchctl set-password username is a way easier than curl -XPUT
http://blah/_users -D... -H...  . at the end if you are a good admin you
will write this script. providing useful helpere don't break the kiss way
here.

benoît

Re: Configuration Load Order

Posted by Jason Smith <jh...@iriscouch.com>.
On Wed, Aug 17, 2011 at 10:22 PM, Robert Newson <rn...@apache.org> wrote:
> Jason,
>
> The --set-password thing is to ensure there are no plaintext passwords
> in the first place, which eliminates the oddness of couch rewriting a
> plaintext pwd to a digested pwd (and putting the output in a different
> file).

Thanks for the clarification.

If you can read a plaintext password from an .ini file, then you can
hit the HTTP API as the admin and make changes to the couch. So that
is privilege escalation.

To answer Benoit's question, it is simpler to tell admins to use the
HTTP API (or Futon) to create the admin account. The password is
stored *somewhere* under the hood. IMHO it is less simple to add a
command-line tool as a requirement (or worse, as an alternative
option) to deploy Couch.

-- 
Iris Couch

Re: Configuration Load Order

Posted by Robert Newson <rn...@apache.org>.
Jason,

The --set-password thing is to ensure there are no plaintext passwords
in the first place, which eliminates the oddness of couch rewriting a
plaintext pwd to a digested pwd (and putting the output in a different
file).

This is only for admin passwords. User passwords are held in _users as
normal (though many are concerned that digested passwords are visible,
and rightly so).

And, as ever, holding your ego together is our primary concern but I
think it can survive an improvement to how admin passwords are managed
in couchdb. :)

B.

On 17 August 2011 15:58, Jason Smith <jh...@iriscouch.com> wrote:
> On Wed, Aug 17, 2011 at 9:22 PM, Robert Newson <rn...@apache.org> wrote:
>> <distilled from IRC chat>
>>
>> A separate password file as described above, but can only be updated thus;
>>
>> # couchdb --set-password admin
>> Password: foo
>> Password updated.
>
> What problem is this solving exactly? This thread started because you
> edit foo.ini and subsequent changes go to bar.ini.
>
> That foo.ini happens to hold plaintext passwords instead of, say, TCP
> nodelay only underscores the problem. But plaintext vs. hashed
> passwords is a totally different matter.
>
> But regarding passwords, would you humor me and please re-state the
> requirements?
>
> I think it is a solution looking for a problem. Are we talking about
> moving *all* passwords to this file (ignoring _user doc .salt and
> .password_sha)? Or are we keeping those in sync now? Or is this just
> admin passwords? But only admins can see (hashed) passwords over HTTP.
> On Unix filesystems, if you have permission to read
> /etc/couchdb/local.ini then you very likely have permission to read
> /var/lib/couchdb/everything.couch, so what is the point?
>
> Regarding --set-password and couchctl, unless I am missing some
> serious requirement (possible), it sounds like CouchDB is poised to
> get much more complex soon. I spend all my free time bragging about
> how simple it is so that would be quite a blow to my ego.
>
> Thanks.
>
> --
> Iris Couch
>

Re: Configuration Load Order

Posted by Benoit Chesneau <bc...@gmail.com>.
On Wednesday, August 17, 2011, Jason Smith <jh...@iriscouch.com> wrote:
> On Wed, Aug 17, 2011 at 9:22 PM, Robert Newson <rn...@apache.org> wrote:
>> <distilled from IRC chat>
>>
>> A separate password file as described above, but can only be updated
thus;
>>
>> # couchdb --set-password admin
>> Password: foo
>> Password updated.
>
> What problem is this solving exactly? This thread started because you
> edit foo.ini and subsequent changes go to bar.ini.

because the biggest pb are passwords. local.ini could be then used for what
it should be: local configuration updated via http or not.
>
> That foo.ini happens to hold plaintext passwords instead of, say, TCP
> nodelay only underscores the problem. But plaintext vs. hashed
> passwords is a totally different matter.
>
> But regarding passwords, would you humor me and please re-state the
> requirements?
>
> I think it is a solution looking for a problem. Are we talking about
> moving *all* passwords to this file (ignoring _user doc .salt and
> .password_sha)? Or are we keeping those in sync now? Or is this just
> admin passwords? But only admins can see (hashed) passwords over HTTP.
> On Unix filesystems, if you have permission to read
> /etc/couchdb/local.ini then you very likely have permission to read
> /var/lib/couchdb/everything.couch, so what is the point?

be safer and more logical. passwords shouldn't be put in plain text at all.
>
> Regarding --set-password and couchctl, unless I am missing some
> serious requirement (possible), it sounds like CouchDB is poised to
> get much more complex soon. I spend all my free time bragging about
> how simple it is so that would be quite a blow to my ego.
>
> Thanks.
>
what is the argument against smplicity here?

- benoit

Re: Configuration Load Order

Posted by Jason Smith <jh...@iriscouch.com>.
On Wed, Aug 17, 2011 at 9:22 PM, Robert Newson <rn...@apache.org> wrote:
> <distilled from IRC chat>
>
> A separate password file as described above, but can only be updated thus;
>
> # couchdb --set-password admin
> Password: foo
> Password updated.

What problem is this solving exactly? This thread started because you
edit foo.ini and subsequent changes go to bar.ini.

That foo.ini happens to hold plaintext passwords instead of, say, TCP
nodelay only underscores the problem. But plaintext vs. hashed
passwords is a totally different matter.

But regarding passwords, would you humor me and please re-state the
requirements?

I think it is a solution looking for a problem. Are we talking about
moving *all* passwords to this file (ignoring _user doc .salt and
.password_sha)? Or are we keeping those in sync now? Or is this just
admin passwords? But only admins can see (hashed) passwords over HTTP.
On Unix filesystems, if you have permission to read
/etc/couchdb/local.ini then you very likely have permission to read
/var/lib/couchdb/everything.couch, so what is the point?

Regarding --set-password and couchctl, unless I am missing some
serious requirement (possible), it sounds like CouchDB is poised to
get much more complex soon. I spend all my free time bragging about
how simple it is so that would be quite a blow to my ego.

Thanks.

-- 
Iris Couch

Re: Configuration Load Order

Posted by Robert Newson <rn...@apache.org>.
<distilled from IRC chat>

A separate password file as described above, but can only be updated thus;

# couchdb --set-password admin
Password: foo
Password updated.
#

Whether this password is written to a separate file or to a future
_config database is immaterial, but separate file is proposed for the
first implementation.

In the case of the latter, the issue of locking yourself out is
raised, so we propose adding;

# couchdb --admin-party
Apache CouchDB <version> (LogLevel=info) is starting.
Apache CouchDB has started in admin party mode! Time to be scared!
[info] [<0.31.0>] Apache CouchDB has started on http://127.0.0.1:5984/
#

B.

On 17 August 2011 15:16, Noah Slater <ns...@apache.org> wrote:
>
> On 17 Aug 2011, at 15:07, Jason Smith wrote:
>
>> The .ini file that edits itself. There are probably some servers out
>> there which do that, but as you know, sysadmins expect /etc files to
>> stay put! Many use CVS to manage them, as you noted.
>>
>> Hence, I like the generated.ini idea because as a sysadmin, during
>> peacetime (couch is running fine) it is just an opaque file blob I
>> need to back up. During wartime (couch is crashed/misconfigured), I
>> can dig into it with emacs and troubleshoot.
>
> Gotcha. :)
>

Re: Configuration Load Order

Posted by Noah Slater <ns...@apache.org>.
On 17 Aug 2011, at 15:07, Jason Smith wrote:

> The .ini file that edits itself. There are probably some servers out
> there which do that, but as you know, sysadmins expect /etc files to
> stay put! Many use CVS to manage them, as you noted.
> 
> Hence, I like the generated.ini idea because as a sysadmin, during
> peacetime (couch is running fine) it is just an opaque file blob I
> need to back up. During wartime (couch is crashed/misconfigured), I
> can dig into it with emacs and troubleshoot.

Gotcha. :)

Re: Configuration Load Order

Posted by Jason Smith <jh...@iriscouch.com>.
On Wed, Aug 17, 2011 at 8:24 PM, Noah Slater <ns...@apache.org> wrote:
>
> On 17 Aug 2011, at 02:04, Jason Smith wrote:
>
>> The American quip goes: the professor who never even ran for dog
>> catcher presumes to tell the president how to do his job. Developers
>> who spend all day in ./utils/run pontificate about good daemon
>> behavior in an OS or distribution.
>
> Just to clear up the ad hominem before I continue. Most of your arguments are directed comments I have made. But of course, I built the original system, and used my experience as a Debian developer to inform my decisions about best practice. I've also been a sysadmin for a number of companies, and in fact, I have never used ./utils/run for anything, so…

Sorry, I'll cut that out on the dev@ list. While I addressed many of
your points later on, the motivation to chime in came directly from
Randall's post. Out of the blue comes discussion of couchctl, and
password management tools, and config file
convention-over-configuration. My feeling is make configuration
simpler than it is today.

Looks like you and I are in broad agreement on the details. Thanks for
the feedback!

>> I do not buy the "misbehaving Couch" scenario. Firstly, how common is
>> that? After installation and confirmation, daemons get pretty stable.
>> If a misconfiguration totally destroys the couch, well, they are still
>> plain text. As before, load emacs and go for it!
>
> The argument was that if daemon configuration lives inside the daemon, and you mess it up, then you wont be able to run the daemon to edit the daemon configuration. As long as there is a separation of concerns, this shouldn't be a problem. You shouldn't be able to hose you CouchDB installation through the configuration options hosted within CouchDB. Think of it in terms of a managed hosting environment. As the service provider, you want to be able to let your customers configure users and permissions and URL handlers, etc. But you don't, necessarily, want them to be messing about with the daemon options.

I have no difficulties thinking of it in terms of a managed hosting
environment :) and I wrote config_whitelist because users were
changing their listen address and port, taking their couches offline.
Totally agree.

>> Finally, I am basically happy with the Couch config. It's quirky but
>> not too bad. I only hope to share a fresh perspective: the viewpoint
>> of people for whom couch is just another daemon, like MySQL or httpd
>> or cron.
>
> Why is it quirky?

The .ini file that edits itself. There are probably some servers out
there which do that, but as you know, sysadmins expect /etc files to
stay put! Many use CVS to manage them, as you noted.

Hence, I like the generated.ini idea because as a sysadmin, during
peacetime (couch is running fine) it is just an opaque file blob I
need to back up. During wartime (couch is crashed/misconfigured), I
can dig into it with emacs and troubleshoot.

-- 
Iris Couch

Re: Configuration Load Order

Posted by Noah Slater <ns...@apache.org>.
On 17 Aug 2011, at 02:04, Jason Smith wrote:

> The American quip goes: the professor who never even ran for dog
> catcher presumes to tell the president how to do his job. Developers
> who spend all day in ./utils/run pontificate about good daemon
> behavior in an OS or distribution.

Just to clear up the ad hominem before I continue. Most of your arguments are directed comments I have made. But of course, I built the original system, and used my experience as a Debian developer to inform my decisions about best practice. I've also been a sysadmin for a number of companies, and in fact, I have never used ./utils/run for anything, so…

> My proposal is already implemented. Now I say promote HTTP config
> (Futon) over .ini files when possible. Integrators, packagers, and
> advanced sysadmins can attack the .ini files just as before.

The more I've reflected on this thread, the more I think that the ini files should be for pre-initialisation settings. Conversely, anything post-initialisation should probably be moved to a special CouchDB database. So it seems we are in agreement here.

> Now we are invoking use cases of versioning the config, and auditing
> it. Wow! 

In many setups, the basic text file configuration of a system is versioned and audited. Think again of the MySQL daemon configuration versus the users and permissions. Debian packages themselves are, of course, a great example of this. Each package ships with it a default configuration that gets the daemon up and running. It would be much harder to do this if that process involved starting CouchDB and then communicating with the database. It would, in the very least, involve writing a custom tool that speaks to CouchDB. For the basics, you want to be able to just move a file to the correct part of a filesystem.

> Config files that change themselves are bizarre and scary.

Agreed.

> Admins, passwords, and non-boostrappy configuration over HTTP seems
> more Couch-like, more "of the web," and more relaxed.

Agreed.

> Take a MySQL admin, or an admin of Drupal, Wordpress, Moodle, Joomla,
> or pretty much any big PHP application. Tell them this: "You have to
> get CouchDB up in the first place. So you edit some config files. Once
> it's up, you connect with your client/browser. It assumes you are an
> admin, and you complete installation over that interface." They would
> respond: "Yeah, sure."

Agreed.

> I do not buy the "misbehaving Couch" scenario. Firstly, how common is
> that? After installation and confirmation, daemons get pretty stable.
> If a misconfiguration totally destroys the couch, well, they are still
> plain text. As before, load emacs and go for it!

The argument was that if daemon configuration lives inside the daemon, and you mess it up, then you wont be able to run the daemon to edit the daemon configuration. As long as there is a separation of concerns, this shouldn't be a problem. You shouldn't be able to hose you CouchDB installation through the configuration options hosted within CouchDB. Think of it in terms of a managed hosting environment. As the service provider, you want to be able to let your customers configure users and permissions and URL handlers, etc. But you don't, necessarily, want them to be messing about with the daemon options.

> Finally, I am basically happy with the Couch config. It's quirky but
> not too bad. I only hope to share a fresh perspective: the viewpoint
> of people for whom couch is just another daemon, like MySQL or httpd
> or cron.

Why is it quirky?

Luckily, I have been building the system from that perspective from the start!

Re: Configuration Load Order

Posted by Jason Smith <jh...@iriscouch.com>.
On Wed, Aug 17, 2011 at 5:03 AM, Randall Leeds <ra...@gmail.com> wrote:
> On Tue, Aug 16, 2011 at 11:33, Jan Lehnardt <ja...@apache.org> wrote:
>
>>
>> On Aug 16, 2011, at 8:31 PM, Noah Slater wrote:
>>
>> >
>> > On 16 Aug 2011, at 10:33, Benoit Chesneau wrote:
>> >
>> >> Imo we shouldn't at all provide plaintext passwords. Maybe a safer
>> >> option would be to let the admin create the first one via http or put
>> >> the hash in the a password.ini file manually. If we are enough kind we
>> >> could also provide a couchctl script allowing user management, config
>> >> changes ... ?
>> >
>> > This sounds like a decent proposal. Much like you have to use htpasswd to
>> generate passwords for Apache httpd, we could bundle a script that lets you
>> generate passwords for the CouchDB ini files, and then forbid the use of
>> plaintext. This solves both the technical problem (I think?) and helps us
>> re-enforce better security practices across the board.
>>
>> Agreed.
>>
>>
> Agreed also. We still have a question about load and save order.
> One idea would be to track the .ini file from whence an option came. If an
> option comes from a local.ini or local.d/ file it could be updated in place.
> If it comes from a default.ini or default.d/ file, updates should be placed
> in local.ini. This would make the most sense to me.
>
> I would also be in favor of enforcing a load order that supports a directory
> structure like:
> local.d/
>  010-stuff.ini
>  020-others.ini

IMHO, this is madness.

The American quip goes: the professor who never even ran for dog
catcher presumes to tell the president how to do his job. Developers
who spend all day in ./utils/run pontificate about good daemon
behavior in an OS or distribution.

(I don't *really* believe this. I know several of you are responsible
for production couches, but that is the flash-bulb image in my mind.)
I don't feel strongly on the matter, just want to share a sysadmin's
perspective. Any of the proposals would be an improvement, so I'm
net-happy.

Some final apologist thoughts:

My proposal is already implemented. Now I say promote HTTP config
(Futon) over .ini files when possible. Integrators, packagers, and
advanced sysadmins can attack the .ini files just as before.

CouchDB stores versioned data, with a powerful validation and audit
tool (potentially, I'm thinking about validate_doc_update and log()).
Now we are invoking use cases of versioning the config, and auditing
it. Wow! My point is not that the config (or some of it) should be in
a database, but that the config should (1) *lose* complexity over
time, not gain it; and (2) be deprecated as an implementation detail,
or just for advanced users.

Config files that change themselves are bizarre and scary. If that's
what we've got, fine, but make it as simple as possible.

Admins, passwords, and non-boostrappy configuration over HTTP seems
more Couch-like, more "of the web," and more relaxed.

Take a MySQL admin, or an admin of Drupal, Wordpress, Moodle, Joomla,
or pretty much any big PHP application. Tell them this: "You have to
get CouchDB up in the first place. So you edit some config files. Once
it's up, you connect with your client/browser. It assumes you are an
admin, and you complete installation over that interface." They would
respond: "Yeah, sure."

I do not buy the "misbehaving Couch" scenario. Firstly, how common is
that? After installation and confirmation, daemons get pretty stable.
If a misconfiguration totally destroys the couch, well, they are still
plain text. As before, load emacs and go for it!

Finally, I am basically happy with the Couch config. It's quirky but
not too bad. I only hope to share a fresh perspective: the viewpoint
of people for whom couch is just another daemon, like MySQL or httpd
or cron.

-- 
Iris Couch

Re: Configuration Load Order

Posted by Randall Leeds <ra...@gmail.com>.
On Tue, Aug 16, 2011 at 11:33, Jan Lehnardt <ja...@apache.org> wrote:

>
> On Aug 16, 2011, at 8:31 PM, Noah Slater wrote:
>
> >
> > On 16 Aug 2011, at 10:33, Benoit Chesneau wrote:
> >
> >> Imo we shouldn't at all provide plaintext passwords. Maybe a safer
> >> option would be to let the admin create the first one via http or put
> >> the hash in the a password.ini file manually. If we are enough kind we
> >> could also provide a couchctl script allowing user management, config
> >> changes ... ?
> >
> > This sounds like a decent proposal. Much like you have to use htpasswd to
> generate passwords for Apache httpd, we could bundle a script that lets you
> generate passwords for the CouchDB ini files, and then forbid the use of
> plaintext. This solves both the technical problem (I think?) and helps us
> re-enforce better security practices across the board.
>
> Agreed.
>
>
Agreed also. We still have a question about load and save order.
One idea would be to track the .ini file from whence an option came. If an
option comes from a local.ini or local.d/ file it could be updated in place.
If it comes from a default.ini or default.d/ file, updates should be placed
in local.ini. This would make the most sense to me.

I would also be in favor of enforcing a load order that supports a directory
structure like:
local.d/
  010-stuff.ini
  020-others.ini

We don't need to ship anything like that by default. I think right now we
take the load directories on the command line, no? It'd be nice if the order
of resolution within those directories was well specified.

-Randall

Re: Configuration Load Order

Posted by Terry Smith <te...@gmail.com>.
On the subject of a tool to write passwords to CouchDB ini file... A while
back I discovered an inconsistency with respect to the password salt
deserialization. If password securing is still a salted hash when this tool
is made you'll need to take what I found into account because I've done
this.

Here's what I found.

A password's salt is defined as "(128 bit UUID generated by the crypto lib)"
from here
http://mail-archives.apache.org/mod_mbox/incubator-couchdb-dev/200810.mbox/%3cFEC771A6-9020-4C30-99A2-C4591B3066E1@apache.org%3e.
The salt starts out that way but then it is converted to a string
representation of the bytes via couch_util:to_hex(). When the hash of the
password + salt is taken the bytes for the character codes of the string
version of the 128 bit UUID are used as the salt. I'll use a 3 byte salt for
an example.

The bytes are 16#3f 16#db 16#ef

to_hex() returns "3fdbef"

list_to_binary() then converts the string into <<"3fdbef"">>... this is the
list of character codes for each character in the string. The bytes are
16#33, 16#66, 16#64, 16#62, 16#65, 16#66. This is the actual salt.

I discovered this because I am setting up an admin's name, hash and salt in
CouchDB's local.ini with an external program (because of my organization's
security requirements), but authentication in CouchDB failed with a
different hash result. I did not convert my 16 byte salt to a string and
then use those bytes (character codes) for the salt. Once I did
authentication in CouchDB succeeded.


--Terry




On Tue, Aug 16, 2011 at 2:33 PM, Jan Lehnardt <ja...@apache.org> wrote:

>
> On Aug 16, 2011, at 8:31 PM, Noah Slater wrote:
>
> >
> > On 16 Aug 2011, at 10:33, Benoit Chesneau wrote:
> >
> >> Imo we shouldn't at all provide plaintext passwords. Maybe a safer
> >> option would be to let the admin create the first one via http or put
> >> the hash in the a password.ini file manually. If we are enough kind we
> >> could also provide a couchctl script allowing user management, config
> >> changes ... ?
> >
> > This sounds like a decent proposal. Much like you have to use htpasswd to
> generate passwords for Apache httpd, we could bundle a script that lets you
> generate passwords for the CouchDB ini files, and then forbid the use of
> plaintext. This solves both the technical problem (I think?) and helps us
> re-enforce better security practices across the board.
>
> Agreed.
>
> Cheers
> Jan
> --
>
>

Re: Configuration Load Order

Posted by Jan Lehnardt <ja...@apache.org>.
On Aug 16, 2011, at 8:31 PM, Noah Slater wrote:

> 
> On 16 Aug 2011, at 10:33, Benoit Chesneau wrote:
> 
>> Imo we shouldn't at all provide plaintext passwords. Maybe a safer
>> option would be to let the admin create the first one via http or put
>> the hash in the a password.ini file manually. If we are enough kind we
>> could also provide a couchctl script allowing user management, config
>> changes ... ?
> 
> This sounds like a decent proposal. Much like you have to use htpasswd to generate passwords for Apache httpd, we could bundle a script that lets you generate passwords for the CouchDB ini files, and then forbid the use of plaintext. This solves both the technical problem (I think?) and helps us re-enforce better security practices across the board.

Agreed.

Cheers
Jan
-- 


Re: Configuration Load Order

Posted by Noah Slater <ns...@apache.org>.
On 16 Aug 2011, at 10:33, Benoit Chesneau wrote:

> Imo we shouldn't at all provide plaintext passwords. Maybe a safer
> option would be to let the admin create the first one via http or put
> the hash in the a password.ini file manually. If we are enough kind we
> could also provide a couchctl script allowing user management, config
> changes ... ?

This sounds like a decent proposal. Much like you have to use htpasswd to generate passwords for Apache httpd, we could bundle a script that lets you generate passwords for the CouchDB ini files, and then forbid the use of plaintext. This solves both the technical problem (I think?) and helps us re-enforce better security practices across the board.

Re: Configuration Load Order

Posted by Benoit Chesneau <bc...@gmail.com>.
On Mon, Aug 15, 2011 at 8:29 PM, Jan Lehnardt <ja...@apache.org> wrote:
>
> On Aug 15, 2011, at 7:36 PM, Noah Slater wrote:
>
>>
>> On 15 Aug 2011, at 18:32, Jan Lehnardt wrote:
>>
>>> 1. Write admin = password to local.ini
>>> 2. Restart CouchDB
>>> 3. Hash gets persisted to generated.ini
>>> 4. Plain text password remains in local.ini
>>
>> Which one of these steps is the problem? 4? What would you have happen in place of that? That the plain text password be removed? Could we not simply leave that up to the admin to remove it from the config? What if it is needed again at some point? If I put my plain text password in a config file that I had edited by hand on a server, I would not expect it to be removed by the software. If I was concerned about saving the plain text password in the first place, I would hope that the software in question would come with an interactive prompt that would ask me for my password and write the hash out to the file for me.
>
> I would expect that a plaintext admin password would never survive a server restart.
>
> If you want to change the admin-addition procedure to a startup prompt thing, I'd be happy to consider this, but currently we are stuck between a rock and a hard place because all the documentation out there suggests adding an admin to local.ini will do the trick, yet distributions that add config files to local.d/ will keep plaintext passwords around, contrary to what is documented. I consider this a bad user experience as well as a security issue.
>
> I was supporting that local.ini should come after local.d/*.ini, but dev@ overturned me here and came up with generated.ini, which I'd be fine with, except, it doesn't solve the original problem.
>
> Cheers
> Jan
> --

Imo we shouldn't at all provide plaintext passwords. Maybe a safer
option would be to let the admin create the first one via http or put
the hash in the a password.ini file manually. If we are enough kind we
could also provide a couchctl script allowing user management, config
changes ... ?


- benoît

Re: Configuration Load Order

Posted by Jan Lehnardt <ja...@apache.org>.
On Aug 15, 2011, at 7:36 PM, Noah Slater wrote:

> 
> On 15 Aug 2011, at 18:32, Jan Lehnardt wrote:
> 
>> 1. Write admin = password to local.ini
>> 2. Restart CouchDB
>> 3. Hash gets persisted to generated.ini
>> 4. Plain text password remains in local.ini
> 
> Which one of these steps is the problem? 4? What would you have happen in place of that? That the plain text password be removed? Could we not simply leave that up to the admin to remove it from the config? What if it is needed again at some point? If I put my plain text password in a config file that I had edited by hand on a server, I would not expect it to be removed by the software. If I was concerned about saving the plain text password in the first place, I would hope that the software in question would come with an interactive prompt that would ask me for my password and write the hash out to the file for me.

I would expect that a plaintext admin password would never survive a server restart.

If you want to change the admin-addition procedure to a startup prompt thing, I'd be happy to consider this, but currently we are stuck between a rock and a hard place because all the documentation out there suggests adding an admin to local.ini will do the trick, yet distributions that add config files to local.d/ will keep plaintext passwords around, contrary to what is documented. I consider this a bad user experience as well as a security issue.

I was supporting that local.ini should come after local.d/*.ini, but dev@ overturned me here and came up with generated.ini, which I'd be fine with, except, it doesn't solve the original problem.

Cheers
Jan
-- 


Re: Configuration Load Order

Posted by Noah Slater <ns...@apache.org>.
On 15 Aug 2011, at 18:32, Jan Lehnardt wrote:

> 1. Write admin = password to local.ini
> 2. Restart CouchDB
> 3. Hash gets persisted to generated.ini
> 4. Plain text password remains in local.ini

Which one of these steps is the problem? 4? What would you have happen in place of that? That the plain text password be removed? Could we not simply leave that up to the admin to remove it from the config? What if it is needed again at some point? If I put my plain text password in a config file that I had edited by hand on a server, I would not expect it to be removed by the software. If I was concerned about saving the plain text password in the first place, I would hope that the software in question would come with an interactive prompt that would ask me for my password and write the hash out to the file for me.

Re: Configuration Load Order

Posted by Noah Slater <ns...@apache.org>.
On 16 Aug 2011, at 02:20, Jason Smith wrote:

> Is it possible to deprecate the .ini files as a configuration tool? In
> other words, tell the world: "Configure CouchDB over HTTP via the
> /_config URLs, probably via Futon."

I think this proposal reaches too far.

Having the configuration in the ini files is good for a number of reasons. It allows you to configure CouchDB without actually running CouchDB. Following on from that, it allows you to rescue a CouchDB instance that is misbehaving, even if you are unable to access CouchDB. It lets sysadmins version the files, perform audits, as well as allowing them to easily integrate CouchDB within automatic deployment and configuration systems.

If there are certain types of things that people regularly want to do via CouchDB itself, such as URL handlers, or users, then I see no reason why this stuff shouldn't be moved to CouchDB itself. But this type of thing should probably be handled on a case-by-case basis. Anything which relates to the CouchDB server in a more general sense, should stay in the system configuration files.

Re: Configuration Load Order

Posted by Jason Smith <jh...@iriscouch.com>.
On Tue, Aug 16, 2011 at 12:32 AM, Jan Lehnardt <ja...@apache.org> wrote:
>
> On Jul 19, 2011, at 5:28 PM, Noah Slater wrote:
>
>>
>> On 19 Jul 2011, at 09:22, Matt Goodall wrote:
>>
>>> This makes sense to me. Personally, I don't think a
>>> generated.ini/generated.d pair is needed - just a single generated.ini would
>>> do.
>>>
>>> As well as ensuring changes are written to the last .ini file in the
>>> configuration chain a generated.ini would act very nicely as a per-instance
>>> configuration for when multiple CouchDB instances are run from the same,
>>> read-only installation, i.e.
>>>
>>>   default.ini/default.d  --  CouchDB default config, system-wide (R)
>>>   local.ini/local.d  --  local sysadmin's config, system-wide (R)
>>>   generated.ini  --  per-instance config (RW)
>>>
>>> As such, I would suggest naming generated.ini something more like
>>> instance.ini.
>>
>> I am +1 on all of this.
>
> This doesn't solve the problem that spawned this discussion:
>
> 1. Write admin = password to local.ini
> 2. Restart CouchDB
> 3. Hash gets persisted to generated.ini
> 4. Plain text password remains in local.ini

That is an excellent point.

IMO (and sysadmins responsible for couches would agree): the .ini
system, especially the automatic editing and overwriting by couch
itself, is brittle or at best confusing. CouchDB's raison d'être is to
store structured data which changes over time. And yet the config is a
totally separate, less mature, less coherent implementation.

You can talk about bootstrapping or human-access or backups, but
wearing my sysadmin hat, I don't care. All I know is the config files
change arbitrarily and incomprehensibly depending on the whims of
CouchDB. (Remember, I wrote the config_whitelist patch in part to
address this.)

Maybe the answer is not in code but in documentation.

Is it possible to deprecate the .ini files as a configuration tool? In
other words, tell the world: "Configure CouchDB over HTTP via the
/_config URLs, probably via Futon."

The .ini files become an irrelevant implementation detail. The fact
that one of them changes is of no concern. Is that possible?

The abstraction is slightly leaky:

1. Sysadmins still have to edit the bootstrapping config, such as the
listen address and port.
2. Sysadmins still have to back-up the .ini files because they do in
fact reflect changes to the config.

But I still think it's a net-win:

1. No changes to the code, just to the mental model of CouchDB
2. Nobody will ever put an admin account in foo.ini, but the hash
shows up in bar.ini

-- 
Iris Couch

Re: Configuration Load Order

Posted by Jan Lehnardt <ja...@apache.org>.
On Jul 19, 2011, at 5:28 PM, Noah Slater wrote:

> 
> On 19 Jul 2011, at 09:22, Matt Goodall wrote:
> 
>> This makes sense to me. Personally, I don't think a
>> generated.ini/generated.d pair is needed - just a single generated.ini would
>> do.
>> 
>> As well as ensuring changes are written to the last .ini file in the
>> configuration chain a generated.ini would act very nicely as a per-instance
>> configuration for when multiple CouchDB instances are run from the same,
>> read-only installation, i.e.
>> 
>>   default.ini/default.d  --  CouchDB default config, system-wide (R)
>>   local.ini/local.d  --  local sysadmin's config, system-wide (R)
>>   generated.ini  --  per-instance config (RW)
>> 
>> As such, I would suggest naming generated.ini something more like
>> instance.ini.
> 
> I am +1 on all of this.

This doesn't solve the problem that spawned this discussion:

1. Write admin = password to local.ini
2. Restart CouchDB
3. Hash gets persisted to generated.ini
4. Plain text password remains in local.ini

Cheers
Jan
-- 



Re: Configuration Load Order

Posted by Noah Slater <ns...@apache.org>.
On 19 Jul 2011, at 09:22, Matt Goodall wrote:

> This makes sense to me. Personally, I don't think a
> generated.ini/generated.d pair is needed - just a single generated.ini would
> do.
> 
> As well as ensuring changes are written to the last .ini file in the
> configuration chain a generated.ini would act very nicely as a per-instance
> configuration for when multiple CouchDB instances are run from the same,
> read-only installation, i.e.
> 
>    default.ini/default.d  --  CouchDB default config, system-wide (R)
>    local.ini/local.d  --  local sysadmin's config, system-wide (R)
>    generated.ini  --  per-instance config (RW)
> 
> As such, I would suggest naming generated.ini something more like
> instance.ini.

I am +1 on all of this.

Re: Configuration Load Order

Posted by Matt Goodall <ma...@gmail.com>.
On 19 July 2011 02:22, Noah Slater <ns...@apache.org> wrote:

>
> On 19 Jul 2011, at 00:57, Randall Leeds wrote:
>
> >> My understanding is that local.ini is that generated.ini.
>
> Nope.
>
> The default.ini file is for the default configuration we ship with CouchDB,
> or for downstream distributions to include a default config. The local.ini
> file is for the local system administrator to customise the setup, without
> changing the system defaults, so that any future updates can be made without
> over-writing local changes.
>

I agree with this.

Also, at least from my experience on Debian-based systems, files from a
something.d directory are *included by* the main something.ini file. So,
writing to local.ini seems wrong as it's not the end of the chain.


>
> What I am suggesting is that we add an additional generated.ini layer that
> sits on top of that, and lets CouchDB write out run-time configuration
> changes. Each layer acts as insulation from loosing changes in the previous
> layer.
>

This makes sense to me. Personally, I don't think a
generated.ini/generated.d pair is needed - just a single generated.ini would
do.

As well as ensuring changes are written to the last .ini file in the
configuration chain a generated.ini would act very nicely as a per-instance
configuration for when multiple CouchDB instances are run from the same,
read-only installation, i.e.

    default.ini/default.d  --  CouchDB default config, system-wide (R)
    local.ini/local.d  --  local sysadmin's config, system-wide (R)
    generated.ini  --  per-instance config (RW)

As such, I would suggest naming generated.ini something more like
instance.ini.

- Matt


>
> >> I'm willing to patch the read order to make sure changes get written to
> local.ini however.
>
> It is my current opinion that this breaks expected behaviour as an easy fix
> for an unrelated problem — namely, where to write config changes to.
>
>

Re: Configuration Load Order

Posted by Noah Slater <ns...@apache.org>.
On 19 Jul 2011, at 00:57, Randall Leeds wrote:

>> My understanding is that local.ini is that generated.ini.

Nope.

The default.ini file is for the default configuration we ship with CouchDB, or for downstream distributions to include a default config. The local.ini file is for the local system administrator to customise the setup, without changing the system defaults, so that any future updates can be made without over-writing local changes.

What I am suggesting is that we add an additional generated.ini layer that sits on top of that, and lets CouchDB write out run-time configuration changes. Each layer acts as insulation from loosing changes in the previous layer.

>> I'm willing to patch the read order to make sure changes get written to local.ini however.

It is my current opinion that this breaks expected behaviour as an easy fix for an unrelated problem — namely, where to write config changes to.


Re: Configuration Load Order

Posted by Randall Leeds <ra...@gmail.com>.
On Mon, Jul 18, 2011 at 15:29, Jan Lehnardt <ja...@apache.org> wrote:
>
> On 18 Jul 2011, at 20:42, Noah Slater wrote:
>>> What is wrong is the place where (some) _writes_ go. Don't know if possible,
>>> but, continuing in your example, changes in geocouch config should go to
>>> geocouch.ini, other changes to local.ini. Or maybe changes to existing key
>>> should go to the file from where it was effectively read, new keys to
>>> local.ini. Or maybe this is way too complicated :-)
>>
>> Also agreed.
>>
>> There is obviously a problem in where we write back to. But I don't necessarily think this means we need to break implicit compatibility with system behaviour expectations. Perhaps a way to solve it would be to write out to a generated.ini file that is loaded last.
>
> My understanding is that local.ini is that generated.ini.
>
> Ideally, we'd track which config setting came from which file and write to the appropriate "local" counterpart, but I'm not willing to write the necessary code for that.

I could maybe be convinced.

>
> I'm willing to patch the read order to make sure changes get written to local.ini however.
>
> Cheers
> Jan
> --
>
>
>

Re: Configuration Load Order

Posted by Jan Lehnardt <ja...@apache.org>.
On 18 Jul 2011, at 20:42, Noah Slater wrote:
>> What is wrong is the place where (some) _writes_ go. Don't know if possible,
>> but, continuing in your example, changes in geocouch config should go to
>> geocouch.ini, other changes to local.ini. Or maybe changes to existing key
>> should go to the file from where it was effectively read, new keys to
>> local.ini. Or maybe this is way too complicated :-)
> 
> Also agreed.
> 
> There is obviously a problem in where we write back to. But I don't necessarily think this means we need to break implicit compatibility with system behaviour expectations. Perhaps a way to solve it would be to write out to a generated.ini file that is loaded last.

My understanding is that local.ini is that generated.ini.

Ideally, we'd track which config setting came from which file and write to the appropriate "local" counterpart, but I'm not willing to write the necessary code for that.

I'm willing to patch the read order to make sure changes get written to local.ini however.

Cheers
Jan
-- 



Re: Configuration Load Order

Posted by Noah Slater <ns...@apache.org>.
On 18 Jul 2011, at 19:23, Ladislav Thon wrote:

> I encountered this behavior some time ago too, but I don't think that there
> is a problem in _read_ order. In fact, the current one is in line with what
> I and hopefully others would expect (*.d directories usually overrides the
> stuff above them).

We are in agreement.

It is my experience that *.d directories are usually read after other things. I modelled the current behaviour on my experience with other Sys-V system packages. I think that the read order behaves in a standard way, and I am hesitant to change it.

> What is wrong is the place where (some) _writes_ go. Don't know if possible,
> but, continuing in your example, changes in geocouch config should go to
> geocouch.ini, other changes to local.ini. Or maybe changes to existing key
> should go to the file from where it was effectively read, new keys to
> local.ini. Or maybe this is way too complicated :-)

Also agreed.

There is obviously a problem in where we write back to. But I don't necessarily think this means we need to break implicit compatibility with system behaviour expectations. Perhaps a way to solve it would be to write out to a generated.ini file that is loaded last.

Re: Configuration Load Order

Posted by Ladislav Thon <la...@gmail.com>.
Not a CouchDB dev, but I figured I'd throw something into this discussion.

I encountered this behavior some time ago too, but I don't think that there
is a problem in _read_ order. In fact, the current one is in line with what
I and hopefully others would expect (*.d directories usually overrides the
stuff above them).

What is wrong is the place where (some) _writes_ go. Don't know if possible,
but, continuing in your example, changes in geocouch config should go to
geocouch.ini, other changes to local.ini. Or maybe changes to existing key
should go to the file from where it was effectively read, new keys to
local.ini. Or maybe this is way too complicated :-)

Just my 0.017 cents.

LT