You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Jan Lehnardt <ja...@apache.org> on 2008/04/23 23:35:07 UTC
Runtime Configuration #1
Dear devs.
I made some progress in getting CouchDB to be
configurable at runtime. And I like you to try it out
and send in suggestions for improvements. The
patches can be found here:
http://pastie.textmate.org/private/evprkfb1jihq2c1fnrtm7w
apply that to trunk and here:
http://pastie.textmate.org/private/wmkxbcx7ua3l1vz76jwba
put that into a file at src/couchdb/couch_config.erl
then run ./bootstrap && ./configure && make && make install
Note that I changed the couch.ini file and you better install
CouchDB into a fresh directory.
What does this do:
There is a new module couch_config that handles storing
and retrieving configuration from the other modules in
CouchDB. On startup, it initializes itself with the values
from couch.ini. It includes a simple HTTP API as well.
It also moves code from couch_server_sup.erl to
couch_config.erl, making the former a tad simpler.
What it doesn't do:
For a new configuration value to take effect, the module
that uses it must be restarted. This does not yet happen.
So effectively, you'd still have to restart CouchDB to
change settings. This will, however, be implemented
so true runtime configuration is possible.
Another feature I plan on is committing changes back
to the couch.ini file.
In conclusion, this doesn't add any actual features at the
moment, but lays the groundwork for a set of cool features
that weren't possible otherwise. The patch aims to not
change behaviour in CouchDB, save renaming some
ini options. The only place I didn't succeed in getting
to work as it did before is the dump of configuration
variables on startup with a log level set to "debug", but
that should be an easy enough fix.
Damien suggested not starting a branch for this, but if
you'd prefer, I'm happy to set one up and maintain it
until the config work is mature enough for trunk.
I open for any suggestions here from implementation
to HTTP API, please be creative :)
Enjoy!
Jan
--
PS: Here's a teaser:
$ curl -X GET http://localhost:5984/_config/httpd/port
{"ok":true,"module":"httpd","key":"port","value":"5984"}
$ curl -X PUT http://localhost:5984/_config/httpd/port -d "5985"
{"ok":true,"module":"httpd","key":"port","value":"5985"}
$ curl -X GET http://localhost:5984/_config/httpd/port
{"ok":true,"module":"httpd","key":"port","value":"5985"}
$ curl -X DELETE http://localhost:5984/_config/httpd/port
{"ok":true,"module":"httpd","key":"port","old_value":"5985"}
$ curl -X GET http://localhost:5984/_config/httpd/port
{"ok":true,"module":"httpd","key":"port","value":"no_value"}
$ curl -X PUT http://localhost:5984/_config/httpd/port -d "5984"
{"ok":true,"module":"httpd","key":"port","value":"5984"}
Re: Runtime Configuration #1
Posted by Jan Lehnardt <ja...@apache.org>.
Heya,
a couple of combined replies on this thread:
On May 11, 2008, at 16:03, Noah Slater wrote:
> On Sat, May 10, 2008 at 04:00:20PM +0200, Christopher Lenz wrote:
>> I'd personally prefer to see branches for such things.
>
> +1
I have created branches/runtimeconfig for future work
>>
>> $ curl -X GET http://localhost:5984/_config/httpd/port
>> {"ok":true,"module":"httpd","key":"port","value":"no_value"}
>
> Should `"no_value"' not be `nil'?
fixed
On Apr 24, 2008, at 23:12, Damien Katz wrote:
> No special couchdb code should be in the config module. So the code
> for formating and dumping to std out and the code for creating a
> default couchdb configuration should still be in couch_server_sup or
> maybe couch_server. But not in couch_config.
fixed. In fact, the dumping is not yet re-introduced.
> Actually there should be no single "default settings" code body,
> each module should deal with missing configuration settings on its
> own. And its fine if they don't handle the case of missing settings
> data and just fail.
all gone.
> Make your changes and repost for review. Also, when submitting code
> for review here just go ahead an include the diffs right into the
> posting, it will make commenting on individual bits of code easier.
I created a branch. see above :)
This version introduces a new feature we've been discussing on IRC:
The final config system will be able to write changes that were
made through the HTTP API back to the couch.ini file for persistence.
To be on the safe side and don't leave a user hanging in case he
misconfigures his CouchDB, we want to use two ini files. One
with default values that we ship with and never change and another
one where the user can make his changes by hand and where the
changes via the HTTP API get written to. To accommodate this,
on startup, CouchDB (more specifically couch_server:start()) now
receives a list of full paths to any number of ini files that are
parsed.
By convention, the last specified ini file (if it is not the fist)
will be used
to write back the changes that are made though the HTTP API).
Noah: I mocked up bin/couchdb(.tpl(.in)) to suport this, all you need
to add is the parsing of the -c flag.
Please test the new branch and let me know what you think.
Note: this is still a moving target and things like auto-rewrite ini-
values
is not yet in.
Cheers
Jan
--
Re: Runtime Configuration #1
Posted by Noah Slater <ns...@apache.org>.
On Sat, May 10, 2008 at 04:00:20PM +0200, Christopher Lenz wrote:
> I'd personally prefer to see branches for such things.
+1
--
Noah Slater - The Apache Software Foundation <http://www.apache.org/>
Re: Runtime Configuration #1
Posted by Christopher Lenz <cm...@gmx.de>.
On 23.04.2008, at 23:35, Jan Lehnardt wrote:
> I made some progress in getting CouchDB to be
> configurable at runtime. And I like you to try it out
> and send in suggestions for improvements. The
> patches can be found here:
> http://pastie.textmate.org/private/evprkfb1jihq2c1fnrtm7w
> apply that to trunk and here:
> http://pastie.textmate.org/private/wmkxbcx7ua3l1vz76jwba
> put that into a file at src/couchdb/couch_config.erl
>
> then run ./bootstrap && ./configure && make && make install
>
> Note that I changed the couch.ini file and you better install
> CouchDB into a fresh directory.
[...]
>
> Damien suggested not starting a branch for this, but if
> you'd prefer, I'm happy to set one up and maintain it
> until the config work is mature enough for trunk.
I'd personally prefer to see branches for such things. It definitely
makes it easier for us as a team to refine the changes, without you
having to play the patch manager ;)
Cheers,
--
Christopher Lenz
cmlenz at gmx.de
http://www.cmlenz.net/
Re: Runtime Configuration #1
Posted by Damien Katz <da...@gmail.com>.
On Apr 26, 2008, at 6:08 PM, Jan Lehnardt wrote:
> Heya Damien,
> thanks for the review.
> On Apr 24, 2008, at 23:12, Damien Katz wrote:
>> Things looks pretty good overall. Some thoughts:
>>
>> No special couchdb code should be in the config module. So the code
>> for formating and dumping to std out and the code for creating a
>> default couchdb configuration should still be in couch_server_sup
>> or maybe couch_server. But not in couch_config.
>>
>> Actually there should be no single "default settings" code body,
>> each module should deal with missing configuration settings on its
>> own. And its fine if they don't handle the case of missing settings
>> data and just fail.
>
> At the moment there is no "default settings code body" apart from
> what was in couch_server_sup already; it just feeds the ini values
> into the couch_config instead of variables as before. Do you suggest
> getting rid of that? How/where to read the ini file then? Should
> couch_config expose functions to read ini values for the other
> modules to use?
>
>
Right, the single code body should go away, and each component should
do it's own thing if the setting is missing for some reason. If could
load its own default or just error out.
Re: Runtime Configuration #1
Posted by Jan Lehnardt <ja...@apache.org>.
Heya Damien,
thanks for the review.
On Apr 24, 2008, at 23:12, Damien Katz wrote:
> Things looks pretty good overall. Some thoughts:
>
> No special couchdb code should be in the config module. So the code
> for formating and dumping to std out and the code for creating a
> default couchdb configuration should still be in couch_server_sup or
> maybe couch_server. But not in couch_config.
>
> Actually there should be no single "default settings" code body,
> each module should deal with missing configuration settings on its
> own. And its fine if they don't handle the case of missing settings
> data and just fail.
At the moment there is no "default settings code body" apart from what
was in couch_server_sup already; it just feeds the ini values into the
couch_config instead of variables as before. Do you suggest getting
rid of that? How/where to read the ini file then? Should couch_config
expose functions to read ini values for the other modules to use?
> Make your changes and repost for review. Also, when submitting code
> for review here just go ahead an include the diffs right into the
> posting, it will make commenting on individual bits of code easier.
>
> Bigger question: When upgrading the server, how to populate the
> defaults in an existing ini file? What do we do now?
>
> On Apr 23, 2008, at 5:35 PM, Jan Lehnardt wrote:
>
>> Dear devs.
>> I made some progress in getting CouchDB to be
>> configurable at runtime. And I like you to try it out
>> and send in suggestions for improvements. The
>> patches can be found here:
>> http://pastie.textmate.org/private/evprkfb1jihq2c1fnrtm7w
>> apply that to trunk and here:
>> http://pastie.textmate.org/private/wmkxbcx7ua3l1vz76jwba
>> put that into a file at src/couchdb/couch_config.erl
>>
>> then run ./bootstrap && ./configure && make && make install
>>
>> Note that I changed the couch.ini file and you better install
>> CouchDB into a fresh directory.
>>
>> What does this do:
>> There is a new module couch_config that handles storing
>> and retrieving configuration from the other modules in
>> CouchDB. On startup, it initializes itself with the values
>> from couch.ini. It includes a simple HTTP API as well.
>>
>> It also moves code from couch_server_sup.erl to
>> couch_config.erl, making the former a tad simpler.
>>
>> What it doesn't do:
>> For a new configuration value to take effect, the module
>> that uses it must be restarted. This does not yet happen.
>> So effectively, you'd still have to restart CouchDB to
>> change settings. This will, however, be implemented
>> so true runtime configuration is possible.
>>
>> Another feature I plan on is committing changes back
>> to the couch.ini file.
>>
>> In conclusion, this doesn't add any actual features at the
>> moment, but lays the groundwork for a set of cool features
>> that weren't possible otherwise. The patch aims to not
>> change behaviour in CouchDB, save renaming some
>> ini options. The only place I didn't succeed in getting
>> to work as it did before is the dump of configuration
>> variables on startup with a log level set to "debug", but
>> that should be an easy enough fix.
>>
>> Damien suggested not starting a branch for this, but if
>> you'd prefer, I'm happy to set one up and maintain it
>> until the config work is mature enough for trunk.
>>
>> I open for any suggestions here from implementation
>> to HTTP API, please be creative :)
>>
>> Enjoy!
>>
>> Jan
>> --
>> PS: Here's a teaser:
>>
>> $ curl -X GET http://localhost:5984/_config/httpd/port
>> {"ok":true,"module":"httpd","key":"port","value":"5984"}
>> $ curl -X PUT http://localhost:5984/_config/httpd/port -d "5985"
>> {"ok":true,"module":"httpd","key":"port","value":"5985"}
>> $ curl -X GET http://localhost:5984/_config/httpd/port
>> {"ok":true,"module":"httpd","key":"port","value":"5985"}
>> $ curl -X DELETE http://localhost:5984/_config/httpd/port
>> {"ok":true,"module":"httpd","key":"port","old_value":"5985"}
>> $ curl -X GET http://localhost:5984/_config/httpd/port
>> {"ok":true,"module":"httpd","key":"port","value":"no_value"}
>> $ curl -X PUT http://localhost:5984/_config/httpd/port -d "5984"
>> {"ok":true,"module":"httpd","key":"port","value":"5984"}
>>
>>
>
>
Re: Runtime Configuration #1
Posted by Noah Slater <ns...@apache.org>.
On Thu, Apr 24, 2008 at 05:12:01PM -0400, Damien Katz wrote:
> Bigger question: When upgrading the server, how to populate the defaults
> in an existing ini file? What do we do now?
Well, no matter what the user does, if you install over the existing install the
configuration gets reset/wiped, so this isn't really a problem.
If the user wants to preserve the configuration they can install the new
configuration into a staging area and diff between the existing and new.
This is pretty standard stuff in the Autotools world, so I have been told. :)
--
Noah Slater - The Apache Software Foundation <http://www.apache.org/>
Re: Runtime Configuration #1
Posted by Damien Katz <da...@gmail.com>.
Things looks pretty good overall. Some thoughts:
No special couchdb code should be in the config module. So the code
for formating and dumping to std out and the code for creating a
default couchdb configuration should still be in couch_server_sup or
maybe couch_server. But not in couch_config.
Actually there should be no single "default settings" code body, each
module should deal with missing configuration settings on its own. And
its fine if they don't handle the case of missing settings data and
just fail.
Make your changes and repost for review. Also, when submitting code
for review here just go ahead an include the diffs right into the
posting, it will make commenting on individual bits of code easier.
Bigger question: When upgrading the server, how to populate the
defaults in an existing ini file? What do we do now?
On Apr 23, 2008, at 5:35 PM, Jan Lehnardt wrote:
> Dear devs.
> I made some progress in getting CouchDB to be
> configurable at runtime. And I like you to try it out
> and send in suggestions for improvements. The
> patches can be found here:
> http://pastie.textmate.org/private/evprkfb1jihq2c1fnrtm7w
> apply that to trunk and here:
> http://pastie.textmate.org/private/wmkxbcx7ua3l1vz76jwba
> put that into a file at src/couchdb/couch_config.erl
>
> then run ./bootstrap && ./configure && make && make install
>
> Note that I changed the couch.ini file and you better install
> CouchDB into a fresh directory.
>
> What does this do:
> There is a new module couch_config that handles storing
> and retrieving configuration from the other modules in
> CouchDB. On startup, it initializes itself with the values
> from couch.ini. It includes a simple HTTP API as well.
>
> It also moves code from couch_server_sup.erl to
> couch_config.erl, making the former a tad simpler.
>
> What it doesn't do:
> For a new configuration value to take effect, the module
> that uses it must be restarted. This does not yet happen.
> So effectively, you'd still have to restart CouchDB to
> change settings. This will, however, be implemented
> so true runtime configuration is possible.
>
> Another feature I plan on is committing changes back
> to the couch.ini file.
>
> In conclusion, this doesn't add any actual features at the
> moment, but lays the groundwork for a set of cool features
> that weren't possible otherwise. The patch aims to not
> change behaviour in CouchDB, save renaming some
> ini options. The only place I didn't succeed in getting
> to work as it did before is the dump of configuration
> variables on startup with a log level set to "debug", but
> that should be an easy enough fix.
>
> Damien suggested not starting a branch for this, but if
> you'd prefer, I'm happy to set one up and maintain it
> until the config work is mature enough for trunk.
>
> I open for any suggestions here from implementation
> to HTTP API, please be creative :)
>
> Enjoy!
>
> Jan
> --
> PS: Here's a teaser:
>
> $ curl -X GET http://localhost:5984/_config/httpd/port
> {"ok":true,"module":"httpd","key":"port","value":"5984"}
> $ curl -X PUT http://localhost:5984/_config/httpd/port -d "5985"
> {"ok":true,"module":"httpd","key":"port","value":"5985"}
> $ curl -X GET http://localhost:5984/_config/httpd/port
> {"ok":true,"module":"httpd","key":"port","value":"5985"}
> $ curl -X DELETE http://localhost:5984/_config/httpd/port
> {"ok":true,"module":"httpd","key":"port","old_value":"5985"}
> $ curl -X GET http://localhost:5984/_config/httpd/port
> {"ok":true,"module":"httpd","key":"port","value":"no_value"}
> $ curl -X PUT http://localhost:5984/_config/httpd/port -d "5984"
> {"ok":true,"module":"httpd","key":"port","value":"5984"}
>
>
Re: Runtime Configuration #1
Posted by Jan Lehnardt <ja...@apache.org>.
On May 11, 2008, at 16:03, Noah Slater wrote:
> On Wed, Apr 23, 2008 at 11:35:07PM +0200, Jan Lehnardt wrote:
>> $ curl -X GET http://localhost:5984/_config/httpd/port
>> {"ok":true,"module":"httpd","key":"port","value":"5984"}
>> $ curl -X PUT http://localhost:5984/_config/httpd/port -d "5985"
>> {"ok":true,"module":"httpd","key":"port","value":"5985"}
>
> Sweet.
>
>> $ curl -X DELETE http://localhost:5984/_config/httpd/port
>> {"ok":true,"module":"httpd","key":"port","old_value":"5985"}
>
> Why bother with `old_value'?
For a client's convenience.
>> $ curl -X GET http://localhost:5984/_config/httpd/port
>> {"ok":true,"module":"httpd","key":"port","value":"no_value"}
>
> Should `"no_value"' not be `nil'?
null, but yeah. I use the no_value atom in Erlang, but that
could/should probably morphed into null in JSON.
Thanks for your input!
Cheers
Jan
--
Re: Runtime Configuration #1
Posted by Noah Slater <ns...@apache.org>.
On Wed, Apr 23, 2008 at 11:35:07PM +0200, Jan Lehnardt wrote:
> $ curl -X GET http://localhost:5984/_config/httpd/port
> {"ok":true,"module":"httpd","key":"port","value":"5984"}
> $ curl -X PUT http://localhost:5984/_config/httpd/port -d "5985"
> {"ok":true,"module":"httpd","key":"port","value":"5985"}
Sweet.
> $ curl -X DELETE http://localhost:5984/_config/httpd/port
> {"ok":true,"module":"httpd","key":"port","old_value":"5985"}
Why bother with `old_value'?
> $ curl -X GET http://localhost:5984/_config/httpd/port
> {"ok":true,"module":"httpd","key":"port","value":"no_value"}
Should `"no_value"' not be `nil'?
--
Noah Slater - The Apache Software Foundation <http://www.apache.org/>