ideas for dynamic configuration

[Ceki Gülcü <ce...@qos.ch>]

It is assumed that the reader is familiar with the concepts found in
commons-digester. See http://jakarta.apache.org/commons/digester.html
for more details.

Log4j, or more specifically the DOMConfigurator, can parse and
interpret configuration files written in XML. One can define appenders
and their layouts, add these to loggers, set the level of loggers and
so on. Unfortunately, the DOMConfigurator cannot deal with XML tags
that were not known to it at compile time.

A jakarta-commons project called commons-digester offers a very
interesting method for interpreting XML files. Instead of hard coding
the way XML files are interpreted, one programs the digester with a
set of rules (a pattern+an action). When the given pattern is matched
by the XML element being parsed, then the programmed action or actions
are invoked. Beside increased clarity and modularity of the code, the
digester mechanism opens the door for dynamic configuration, that is
the capability to learn new rules at run time, much like ANT's ability
to learn new tasks.

Log4j 1.3 will obviously contain hard-coded rules to enable it to
configure itself. The novelty will be in its capability to learn new
tricks. These new rules must be specified somewhere. The question is
where?  Where are these rules specified? There are at least two
possibilities.

1) In the XML configuration file

Nothing fancy. This is just the straightforward application of existing
digester usage patterns. The down side is that these rules must be
repeated in each configuration file.

2) Within the component being configured.

The component being configured informs the digester about the
additional rules it needs.  These rules are applied only while the
component is being processed (configured). They are removed just after
the processing of that particular component finishes.

This approach would require changes in the commons-digester
model. Instead of having a single set of rules, the set of rules would
be hierarchical. There would always be the global set of rules at the
top. One ore more child rule-sets (defined by the components) would be
added/removed according to interactions with the components.  The
search for patterns would proceed from child to parent rule-sets.

This is perhaps an overkill for log4j but I think it could be useful
in more complicated scenarios.


--
Ceki


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: ideas for dynamic configuration

[robert burrell donkin <ro...@mac.com>]

On Wednesday, May 15, 2002, at 11:32 AM, Ceki Gülcü wrote:

> At 22:04 14.05.2002 +0100, you wrote:

<snip>

>> the first strategy is to allow the subcomponents to configure themselves 
>> using bean-like rules. the idea is that the object to be configured must 
>> conform to certain patterns similar to getters and setters for the 
>> xml->object mapping. (xo and betwixt in the commons-sandbox were created 
>> with this philosophy in mind.)
>>
>> this strategy has the advantage of being more secure and offers less 
>> room for configuration conflicts. it's also much easier to explain to an 
>> implementor that these are the rules that their beans need to follow to 
>> make the configuration work properly than it is to say - go learn 
>> digester.
>
> Interesting approach. Do you have an idea of what would these rules
> be? Isn't this approach "undigesterly"? I mean that "assembly by
> discovery" does not really need the digester machinery.

digester is just a SAX wrapper. just about anything you can do with SAX 
you can do using digester. (and that means just about anything.)

there are various ways of using digester. this approach certainly goes 
against the original digester style. digester doesn't have any rules which 
are quite as radical as this at the moment but a lot of people seem to use 
digester in similar ways using naming patterns and extended wildcards to 
save writing lots of rules.

i tend to use the extended base rules and general pattern based mapping. i 
believe that scott has been working on esoteric stuff that is even more 
liberal. judging from the feedback on the list, most users of digester are 
pretty lazy and seem to prefer things to be as automatic as possible.

one powerful feature about digester is that it's easy to mix and match 
different approaches.

> Your go learn digester comment is an important one.  First, note that
> not all components need to interact with the digester, only the more 
> complex ones.
> Second, learning the digester model is not that difficult but given that 
> I
> find the digester model quite brilliant, my judgement may be somewhat
> clouded... :-)

brilliant? digester was designed by craig which says it all, i think.

this approach probably originates with jason. he was looking for a xml <-> 
object mapper for turbine. after taking soundings on the turbine lists, he 
didn't want a flexible system but rather one that was generic so that any 
bean matching certain naming conventions could be mapped. the point was 
that any coder can quickly learn a set of conventions whereas learning a 
more flexible - although more powerful - system takes quite a lot longer.

i worked a bit with him on an implementation based on digester a while ago 
but unfortunately fighting commons-logging flame wars took all my energy 
and jason ended up creating a version based on dom4j instead. it's called 
xo and it's in the sandbox if you want to take a look.

i've been wanting to try and write a rule that provides a digester 
compatible implementation of jason's heuristics for a while now.

betwixt also has some interesting ideas (which since it was dreamed up by 
james strachan shouldn't come as a surprise to anyone). this is primarily 
an object -> xml mapper which is inspired by the java bean specification. 
it uses digester for round tripping (but this isn't very well debugged at 
the moment) and so (when it's fixed up) could also be used for automagic 
xml -> object mapping.


i wasn't really advocating using them, i just though that you might find 
it useful to know the state of the 'current art' (as i see it).

- robert


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: ideas for dynamic configuration

[Ceki Gülcü <ce...@qos.ch>]

At 22:04 14.05.2002 +0100, you wrote:
>On Tuesday, May 14, 2002, at 02:00 PM, Ceki Gülcü wrote:
>
><snip>
>
>>2) Within the component being configured.
>>
>>The component being configured informs the digester about the
>>additional rules it needs.  These rules are applied only while the
>>component is being processed (configured). They are removed just after
>>the processing of that particular component finishes.
>>
>>This approach would require changes in the commons-digester
>>model. Instead of having a single set of rules, the set of rules would
>>be hierarchical. There would always be the global set of rules at the
>>top. One ore more child rule-sets (defined by the components) would be
>>added/removed according to interactions with the components.  The
>>search for patterns would proceed from child to parent rule-sets.
>>
>>This is perhaps an overkill for log4j but I think it could be useful
>>in more complicated scenarios.
>
>by a strange piece of synchronicity, functionality along these lines was 
>one reason i was going to give for trying to keep the version of digester 
>used by log4j as close to the base code as possible :)
>
>this is something for which there has been a need for a while now. there 
>are two general approaches to the problem that i'm aware of.
>
>the first strategy is to allow the subcomponents to configure themselves 
>using bean-like rules. the idea is that the object to be configured must 
>conform to certain patterns similar to getters and setters for the 
>xml->object mapping. (xo and betwixt in the commons-sandbox were created 
>with this philosophy in mind.)
>
>this strategy has the advantage of being more secure and offers less room 
>for configuration conflicts. it's also much easier to explain to an 
>implementor that these are the rules that their beans need to follow to 
>make the configuration work properly than it is to say - go learn digester.

Interesting approach. Do you have an idea of what would these rules
be? Isn't this approach "undigesterly"? I mean that "assembly by
discovery" does not really need the digester machinery.

Your go learn digester comment is an important one.  First, note that
not all components need to interact with the digester, only the more 
complex ones.
Second, learning the digester model is not that difficult but given that I
find the digester model quite brilliant, my judgement may be somewhat
clouded... :-)

>the other strategy (as ceki outlined above) is to allow objects to supply 
>the rules required to allow mapping by digester. we've had a patch sitting 
>around for a long time now which offers an implementation. it has some 
>good points but AFAIK it hasn't been committed. i originally had some 
>reservations about it's design but i did eventually get back to the 
>original author offering to commit it if he still felt strongly enough 
>about it. ceki's outlined implementation is much more like the way i'd 
>prefer to see it done.

The problem with my suggestion is that it requires some low level
changes to the digester code as much as the digester model. This
increases the complexity of the code. Another problem with the
localized rules approach is that digester learns new rules only to
forget them a little while later. There are probably simpler ways to
learn about new rules. For example, we could have them stored in an
XML file and have them parsed with DigesterLoader or some other
similar mechanism. See for example the
org.apache.commons.digester.xmlrules package.


>the major advantage of this approach is that it is powerful and flexible.

It also requires low level changes to the existing digester model
which is a disadvantage.

By the way, I will not have the time to work on this for a couple of
weeks. If anyone else wants to start coding this very critical piece,
they are welcome to it, as long as they are open to code reviews. :-)


--
Ceki


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: ideas for dynamic configuration

[robert burrell donkin <ro...@mac.com>]

On Tuesday, May 14, 2002, at 02:00 PM, Ceki Gülcü wrote:

<snip>

> 2) Within the component being configured.
>
> The component being configured informs the digester about the
> additional rules it needs.  These rules are applied only while the
> component is being processed (configured). They are removed just after
> the processing of that particular component finishes.
>
> This approach would require changes in the commons-digester
> model. Instead of having a single set of rules, the set of rules would
> be hierarchical. There would always be the global set of rules at the
> top. One ore more child rule-sets (defined by the components) would be
> added/removed according to interactions with the components.  The
> search for patterns would proceed from child to parent rule-sets.
>
> This is perhaps an overkill for log4j but I think it could be useful
> in more complicated scenarios.

by a strange piece of synchronicity, functionality along these lines was 
one reason i was going to give for trying to keep the version of digester 
used by log4j as close to the base code as possible :)

this is something for which there has been a need for a while now. there 
are two general approaches to the problem that i'm aware of.


the first strategy is to allow the subcomponents to configure themselves 
using bean-like rules. the idea is that the object to be configured must 
conform to certain patterns similar to getters and setters for the 
xml->object mapping. (xo and betwixt in the commons-sandbox were created 
with this philosophy in mind.)

this strategy has the advantage of being more secure and offers less room 
for configuration conflicts. it's also much easier to explain to an 
implementor that these are the rules that their beans need to follow to 
make the configuration work properly than it is to say - go learn digester.


the other strategy (as ceki outlined above) is to allow objects to supply 
the rules required to allow mapping by digester. we've had a patch sitting 
around for a long time now which offers an implementation. it has some 
good points but AFAIK it hasn't been committed. i originally had some 
reservations about it's design but i did eventually get back to the 
original author offering to commit it if he still felt strongly enough 
about it. ceki's outlined implementation is much more like the way i'd 
prefer to see it done.

the major advantage of this approach is that it is powerful and flexible.


of course, these strategies are not necessarily exclusive. digester is 
certainly flexible enough to allow rules to implement either strategy.

- robert


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>