Config Validators

[Ulises <ul...@gmail.com>]

Hey all,

I'm in the process of writing configuration validators, i.e. a fun
that gets invoked when a change in a config parameter is requested via
PUT. The main machinery is in place in couch_config.erl and now
modules can register their validators through
couch_config:register_validator(fun f/3, {Section, Key}). Next time
couch_config receives a PUT with a value Value for {Section, Key} it
will invoke f(Section, Key, Value). In terms of code a module should
have

foo_validator("section", "key", Value) ->
  % do your validation here
  % if all went well return ok, else return {error, Msg}
  ok.

init( ... ) ->
  % do your init
  couch_config:register_validator(fun foo_validator/3, {"section", "key"}),
  % more stuff
  Reply.

It's fine and dandy and working. The issue to solve now is validating
parameter values before init (or before the module can actually start
doing useful stuff).

Here's the two options I came up with:

1) validate everything on load of parameter values (on couch_config)
and then signal a "config change" so that modules can reload and
change from a blank state to a configured state. This implies that
modules only register validators on init and then wait for signals
(somehow). That is: 2 inits. Not very nice really.

2) each module's init validates values coming from config (config
doesn't do any checking on load from file) and if any value is invalid
it deals with the error itself, registering validators still applies
for config changes. This would keep couch_config.erl simpler and
validating on init on each module is not such a big deal anyway as you
would ideally reuse the funs you've registered as validators.

If you have any suggestions please let me know, I'm planning on
implementing this tonight.

BTW, this whole validation thing was motivated by ticket 153
(https://issues.apache.org/jira/browse/COUCHDB-153).

U

Re: Config Validators

[Chris Anderson <jc...@gmail.com>]

On Tue, Jan 13, 2009 at 10:48 AM, Ulises <ul...@gmail.com> wrote:
>> And I choose a slightly modified form of 2, but have the config module
>> call each registered validator immediately and then close things down
>> if one fails.
>
> The problem is that validators are registered asynchronously so, from
> the config's point of view, you can't be sure you've ran them all once
> you're done loading the config. From the module's point of view,
> however, once you start pulling values out of the config you know
> they've been loaded. I think that checking that a value is valid in
> the init() of a module might be a better approach.
>
> Generally, though, does this whole validators thingie make sense?
>

I think it would make more sense for the couchdb modules which are
being configured to raise whatever errors they may have, upon
receiving a config change. It's just a matter of documenting those
errors better. Trying to second-guess the work of admins seems like
more headache than help. I think the more straightforward approach
will give us all the startup error messages we need, if we see it
through.

-- 
Chris Anderson
http://jchris.mfdz.com

Re: Config Validators

[Ulises <ul...@gmail.com>]

> Couch admins may not always be admins on the server itself. If the
> validations can be limited to checks that would otherwise result in a crash,
> they're not training wheels for admins, more like safety nets. You shouldn't
> run up against them unless you're otherwise *fucked*.

This would be the point of validators: checking that parameter values
are sane and won't make couchdb crash and burn.

Yes, admins should double and triple check config values (or at least
that's what good admins do) however sh*t happens and you might miss a
., assign too strict permissions to a dir, etc. I think the greatest
benefit will come from not allowing an admin feeding couch potentially
dangerous (as in making couchdb crash) config values when couchdb is
up and running.

This doesn't prescribe good error reporting on init though.

U

Re: Config Validators

[Dean Landolt <de...@deanlandolt.com>]

On Tue, Jan 13, 2009 at 6:44 PM, Paul Davis <pa...@gmail.com>wrote:

> On Tue, Jan 13, 2009 at 2:16 PM, Damien Katz <da...@apache.org> wrote:
> > I see the danger of the admin setting a value that then disables the
> server,
> > they could do that anyway by editing the ini file. I think instead of
> > validators, we should just contain a stern warning in Futon that any bad
> > change to the config could cause the server to crash and un-restartable
> and
> > to repair it they will need to manually edit the INI.
> >
>
> A stern warning doesn't catch typos, mis-behaving scripts etc. I'd
> rather have an error thrown when something unexpected happens instead
> of crashing a production environment. I very much disagree that we
> shouldn't make some effort to inform people that a change will break
> things catastrophically.
>
> > The problem is Config validators are trying to provide training wheels to
> > the users who need and desire it least, administrators. And we can't be
> > expected to validated every input to see if it will work properly before
> it
> > takes effect. Changing the config is inherently dangerous.
> >
>
> So basically we should just stand at the edge of the cliff and watch
> administrators jump off and go splat? I don't see why you think of it
> as training wheels for new users. Administrators are very capable of
> making mistakes. We may not be able to check every possible config
> permutation for safety, but we can catch quite a few silly errors.
>
> > I completely agree giving nice error messages on start up when possible
> is a
> > very good thing, but we don't need validators to do that, we just need to
> > add it to the code in each module's init code to print nice error
> messages
> > for common failures.
> >
>
> Config validators were intended as a way to take the error detection
> code out of the init function and put it somewhere that could be run
> when the config changes thus providing the entire facility for runtime
> checking. To me its a simple system with a good cost/benefit ratio.
>

Couch admins may not always be admins on the server itself. If the
validations can be limited to checks that would otherwise result in a crash,
they're not training wheels for admins, more like safety nets. You shouldn't
run up against them unless you're otherwise *fucked*.

Re: Config Validators

[Paul Davis <pa...@gmail.com>]

On Tue, Jan 13, 2009 at 2:16 PM, Damien Katz <da...@apache.org> wrote:
> I see the danger of the admin setting a value that then disables the server,
> they could do that anyway by editing the ini file. I think instead of
> validators, we should just contain a stern warning in Futon that any bad
> change to the config could cause the server to crash and un-restartable and
> to repair it they will need to manually edit the INI.
>

A stern warning doesn't catch typos, mis-behaving scripts etc. I'd
rather have an error thrown when something unexpected happens instead
of crashing a production environment. I very much disagree that we
shouldn't make some effort to inform people that a change will break
things catastrophically.

> The problem is Config validators are trying to provide training wheels to
> the users who need and desire it least, administrators. And we can't be
> expected to validated every input to see if it will work properly before it
> takes effect. Changing the config is inherently dangerous.
>

So basically we should just stand at the edge of the cliff and watch
administrators jump off and go splat? I don't see why you think of it
as training wheels for new users. Administrators are very capable of
making mistakes. We may not be able to check every possible config
permutation for safety, but we can catch quite a few silly errors.

> I completely agree giving nice error messages on start up when possible is a
> very good thing, but we don't need validators to do that, we just need to
> add it to the code in each module's init code to print nice error messages
> for common failures.
>

Config validators were intended as a way to take the error detection
code out of the init function and put it somewhere that could be run
when the config changes thus providing the entire facility for runtime
checking. To me its a simple system with a good cost/benefit ratio.

Paul Davis

> -Damien
>
>
>
> On Jan 13, 2009, at 1:48 PM, Ulises wrote:
>
>>> And I choose a slightly modified form of 2, but have the config module
>>> call each registered validator immediately and then close things down
>>> if one fails.
>>
>> The problem is that validators are registered asynchronously so, from
>> the config's point of view, you can't be sure you've ran them all once
>> you're done loading the config. From the module's point of view,
>> however, once you start pulling values out of the config you know
>> they've been loaded. I think that checking that a value is valid in
>> the init() of a module might be a better approach.
>>
>> Generally, though, does this whole validators thingie make sense?
>>
>> U
>
>
>

Re: Config Validators

[Ulises <ul...@gmail.com>]

> I completely agree giving nice error messages on start up when possible is a
> very good thing, but we don't need validators to do that, we just need to
> add it to the code in each module's init code to print nice error messages
> for common failures.

That's a fair and sensible point (together with the comment about the
wheelies :)

/me heads off to work on providing better error reporting on module startup

U

Re: Config Validators

[Damien Katz <da...@apache.org>]

I see the danger of the admin setting a value that then disables the  
server, they could do that anyway by editing the ini file. I think  
instead of validators, we should just contain a stern warning in Futon  
that any bad change to the config could cause the server to crash and  
un-restartable and to repair it they will need to manually edit the INI.

The problem is Config validators are trying to provide training wheels  
to the users who need and desire it least, administrators. And we  
can't be expected to validated every input to see if it will work  
properly before it takes effect. Changing the config is inherently  
dangerous.

I completely agree giving nice error messages on start up when  
possible is a very good thing, but we don't need validators to do  
that, we just need to add it to the code in each module's init code to  
print nice error messages for common failures.

-Damien



On Jan 13, 2009, at 1:48 PM, Ulises wrote:

>> And I choose a slightly modified form of 2, but have the config  
>> module
>> call each registered validator immediately and then close things down
>> if one fails.
>
> The problem is that validators are registered asynchronously so, from
> the config's point of view, you can't be sure you've ran them all once
> you're done loading the config. From the module's point of view,
> however, once you start pulling values out of the config you know
> they've been loaded. I think that checking that a value is valid in
> the init() of a module might be a better approach.
>
> Generally, though, does this whole validators thingie make sense?
>
> U

Re: Config Validators

[Ulises <ul...@gmail.com>]

> And I choose a slightly modified form of 2, but have the config module
> call each registered validator immediately and then close things down
> if one fails.

The problem is that validators are registered asynchronously so, from
the config's point of view, you can't be sure you've ran them all once
you're done loading the config. From the module's point of view,
however, once you start pulling values out of the config you know
they've been loaded. I think that checking that a value is valid in
the init() of a module might be a better approach.

Generally, though, does this whole validators thingie make sense?

U

Re: Config Validators

[Paul Davis <pa...@gmail.com>]

On Tue, Jan 13, 2009 at 11:42 AM, Damien Katz <da...@apache.org> wrote:
> What happens on startup after the user puts bad values into the ini file?

This is exactly the question at hand as addressed by COUCHDB-153.
Currently, if a directory structure doesn't exist or permissions are
wrong somewhere, CouchDB shits itself and dies a horrible flaming
death. All without providing the user with any indication on what the
error might've been.

I've personally spent a good half hour debugging the fact that I
forgot to `make dev` after a local-clean, so I can only imagine the
frustration for people not familiar with the system and/or the
language.

The proposed solution would be to allow methods to prevent possibly
catastrophic changes to the DB configuration thus creating a more
stable system at runtime as well as providing a notification at
startup when something appears to be screwed up. The question at hand
is about the mechanics of how the errors during startup should be
handled.

And I choose a slightly modified form of 2, but have the config module
call each registered validator immediately and then close things down
if one fails.

HTH,
Paul Davis

>
> -Damien
>
>
> On Jan 13, 2009, at 11:36 AM, Ulises wrote:
>
>>> My question is what problem does this solve?
>>
>> Users shooting themselves in the foot by providing wrong config
>> parameters, better error reporting for, say, when the dir in which the
>> dbs reside is non-writeable by the user couchdb is running as, etc.
>>
>> U
>
>

Re: Config Validators

[Ulises <ul...@gmail.com>]

> What happens on startup after the user puts bad values into the ini file?

Up to the module, use default values, refuse to start, die with a nice
error message, etc.

U

Re: Config Validators

[Damien Katz <da...@apache.org>]

What happens on startup after the user puts bad values into the ini  
file?

-Damien


On Jan 13, 2009, at 11:36 AM, Ulises wrote:

>> My question is what problem does this solve?
>
> Users shooting themselves in the foot by providing wrong config
> parameters, better error reporting for, say, when the dir in which the
> dbs reside is non-writeable by the user couchdb is running as, etc.
>
> U

Re: Config Validators

[Ulises <ul...@gmail.com>]

> My question is what problem does this solve?

Users shooting themselves in the foot by providing wrong config
parameters, better error reporting for, say, when the dir in which the
dbs reside is non-writeable by the user couchdb is running as, etc.

U

Re: Config Validators

[Damien Katz <da...@apache.org>]

My question is what problem does this solve?

-Damien


On Jan 13, 2009, at 9:59 AM, Ulises wrote:

> Hey all,
>
> I'm in the process of writing configuration validators, i.e. a fun
> that gets invoked when a change in a config parameter is requested via
> PUT. The main machinery is in place in couch_config.erl and now
> modules can register their validators through
> couch_config:register_validator(fun f/3, {Section, Key}). Next time
> couch_config receives a PUT with a value Value for {Section, Key} it
> will invoke f(Section, Key, Value). In terms of code a module should
> have
>
> foo_validator("section", "key", Value) ->
>  % do your validation here
>  % if all went well return ok, else return {error, Msg}
>  ok.
>
> init( ... ) ->
>  % do your init
>  couch_config:register_validator(fun foo_validator/3, {"section",  
> "key"}),
>  % more stuff
>  Reply.
>
> It's fine and dandy and working. The issue to solve now is validating
> parameter values before init (or before the module can actually start
> doing useful stuff).
>
> Here's the two options I came up with:
>
> 1) validate everything on load of parameter values (on couch_config)
> and then signal a "config change" so that modules can reload and
> change from a blank state to a configured state. This implies that
> modules only register validators on init and then wait for signals
> (somehow). That is: 2 inits. Not very nice really.
>
> 2) each module's init validates values coming from config (config
> doesn't do any checking on load from file) and if any value is invalid
> it deals with the error itself, registering validators still applies
> for config changes. This would keep couch_config.erl simpler and
> validating on init on each module is not such a big deal anyway as you
> would ideally reuse the funs you've registered as validators.
>
> If you have any suggestions please let me know, I'm planning on
> implementing this tonight.
>
> BTW, this whole validation thing was motivated by ticket 153
> (https://issues.apache.org/jira/browse/COUCHDB-153).
>
> U