Project Proposal: Configuration Utility

[Bruce Conrad <ba...@mindspring.com>]

I have developed a very useful Configuration utility that provides very 
simple, flexible and powerful functionality for managing one or more 
configurations in a java environment.

It allows for a combination of a hierarchy of configuration files, 
substitution variables and property variables. Methods are provided to get 
values stored in a configuration dictionary with a variety of types 
(String, array, integer, float, boolean, etc) and default values.

I've used it in several applications and I've had very good feedback from 
other developers. It solves some very basic problems common to any medium 
to large size application. I'd like to contribute it to the open source 
community and see it enhanced. Any suggestions on how to proceed with this 
would be welcome. (I hope this is the right place to post this. If not, my 
apologies).

A more complete description follows:

Configuration Object


This Configuration package provides very simple, flexible and powerful 
functionality for managing one or more configurations in a java environment.


THE PROBLEM

Most projects require configurations for multiple versions of that project. 
You may have multiple developers connecting to separate database instances, 
each with their own password. Additionally, multiple production, stage, and 
training platforms may also need to be supported. Files may reside in one 
directory on Windows, another directory on Unix.

At the same time, many configuration parameters are unchanged across 
multiple environments.

Many projects use one or more property files to define configuration 
parameters. There are many problems inherent in this approach. If you want 
to have different values depending on your environment you have several 
choices, none of them good.
1.      Modify your configuration files every time you deploy in a new 
environment.
This is error prone and time consuming, especially if you are building often.
2.      Maintain separate copies of configuration files in different 
locations with the same file name.
This means you can’t have your configuration files in your source control 
repository since they have the same name. It also means that when you 
change one, you need to change every version of that file in every 
location. That is a more difficult task since the files may be in different 
locations and are not tracked by version control.

3.      Maintain separate copies but with different names.
You’ve solved the version control problem, but you still need to change 
every version when you change one.

When clients install applications remotely, the client is often required to 
make modifications to the configuration file. Every time they get a new 
version of the application, they need to refit their local updates. This 
too is error prone and time consuming.

Often, configuration parameters are stored in property files. But property 
files are java specific. If the same configuration parameters are needed in 
a non-java environment, or even scripts used to control a java environment, 
even more configuration files may be necessary.


THE SOLUTION

What is needed is a hierarchical approach to configuration files. The 
ninety percent of configuration values that do not change can be maintained 
in a base file. The other ten percent (or less) may be maintained in their 
own distinct configuration file. At run time, the files are layered on top 
of each other to provide a flexible, manageable configuration. For example, 
in a development environment myhost.config.xml combines with dev.config.xml 
and base.config.xml to form my unique configuration.

Each configuration file may then be maintained in version control as they 
have unique names. Only the base files need to be modified when base values 
change, and it is easy to see the difference between versions. Another 
major benefit is that changes to the base configuration file will be 
exhaustively tested before deployment.

Often part of a value may change, but not the entire value. Many values may 
follow this pattern. For instance, in application X, development files are 
written to …/dev/files. Support emails go to devsupport@xcompany.com, with 
a subject line of “dev error”.  In production, files are written to 
…/prod/files. Support emails go to prodsupport@xcompany.com, with a subject 
line of “prod error in application X.” The only thing that has really 
changed is the words “dev” and “prod” There is really only one change here, 
not three. The use of substitution variables enables us to accomplish this 
with only one entry.

In the case of applications installed by clients, if the client were to 
make their modifications to a client configuration that inherited from the 
base configuration, it would eliminate the need to refit the client 
modifications when the product is upgraded. It would also be easier to see 
exactly what changes the client made, making it easier to debug 
configuration problems.

This configuration object provides many of these solutions. It allows for a 
combination of a hierarchy of configuration files, substitution variables 
and property variables. Methods are provided to get values stored in a 
configuration dictionary with a variety of types (String, array, integer, 
float, boolean, etc) and default values.

After some initial setup, the end result will be a cleaner, more easily 
maintainable application.



USAGE

The class Config implements the public interface for the configuration 
object. To get a singleton instance of this object, use 
Config.getInstance(). To get values from the config object, use 
Config.getInstance().getValue(sectionName, key) Or Config. 
getInstance().getValue(sectionName, key, defaultValue) If you use a default 
value and the section/key is not found, the default value will return. If 
you don't use a default value and the section/key is not found, 
ConfigException is thrown.

To set Config values, use Config.setConfigurationValue(sectionName, key, 
newValue);
For the most part, however, config values are set during the initial 
parsing of one or more ini files.

To show a complete listing of Config values, use 
Config.getInstance().printConfigurationDictionary()


  SETTING UP THE CONFIGURATION FILE(S):

There are two formats for specifying configurations. One is XML based, the 
other is standard INI file format.


XML Format

<configuration>

     <section name="locos">
         <entry key="instance" value="development" />
     </section>

    <section name="Paths">
       <entry key="locosHome" value="d:/[locos{instance}/project/" />
       <entry key="locosExternal" value="d:/external/[locos{instance}/" />
    </section>

    <section name="attachments">
       <entry key="attachmentDirectory" 
value="[paths{locosExternal}attachments/"/>
    </section>

</configuration>
INI File Format

  Section names are in brackets, followed by key/value pairs.

  Following is a sample file:

  [project]
  instance=development
  timeout=5

  [Paths]
  locosHome=d:/project/[locos{instance}/
  locosExternal=d:/[locos{instance}/

  [attachments]
  attachmentDirectory=[paths{locosHome}attachments/


  The first section is very straightforward. A call to
  getValue("project", "instance") returns value "development".


SUBSTITUTION WITHIN ONE CONFIGURATION FILE

The second value, section Paths, key locosHome, uses the value of  project, 
instance to build its value. So, locosHome resolves to 
d:/project/development . The next entry, attachmentDirectory, uses the 
previous entry to build its value. Thus, attachmentDirectory resolves to 
d:/project/development/attachment.

This comes in very handy when there are many values that have common 
relative paths or some other value in common. The root value can be changed 
once without having to change every value.


SUBSTITUTION VARIABLES

As shown in the above sample file you can "build" values using substition 
variables.  Substitution variables are in the format: [section{key}

PROPERTY VARIABLES

You may also substitute property variables in the format: $xxx$ . You can 
use this to dynamically build values such as classpaths or file paths based 
on the settings of your environment.


MULTIPLE CONFIGURATION FILES

One of the most useful aspects of this configuration object is the ability 
to layer multiple configuration files. This can be very helpful as you can 
maintain one stable base configuration file and make modifications to other 
config files. The “include” directive instructs the Configuration parser to 
find the included configuration file. Files are parsed in reverse order. 
Thus, if a prod config file includes a base config file, the base config is 
parsed first. The prod config will be parsed next and any duplicate 
section/key values will overlay those in the base config file.
Note: The ini format has not been upgraded to recognize the include directive.

You can maintain a base configuration file for an application while 
allowing users to modify a local or site copy. That way, the base 
configuration file is unchanged and it is easy to see what values are 
overridden. If you upgrade the base configuration file, local overrides do 
not need to be re-implemented. Developers could take advantage of this by 
making their changes to a test configuration file, leaving the base 
unchanged and making it very obvious as to what is being overridden.


COMMENTS AND CONTINUATION LINES

Other syntax rules are "*" for comments and "," for continuation lines 
(This pertains to ini file format only).


HELPER METHODS

The most used method, getValue, returns its value as a String. 
getIntegerValue, getFloatValue, getLongValue, and getBooleanValue will 
return values in the corresponding primitive types.

Additional helper methods are getArrayValue which parses values separated 
by commas, semicolons, or spaces into a String array.

getValuesStartingWith returns a List of values starting with the specified 
Key/Value.

See the javadoc for more information.


CASE INSENSITIVITY

This configuration package is case insensitive. While values retain their 
case, the keys to access them will work regardless of case. So, if, the 
section is "aSection" and  the key is aKey", the value may be retrieved 
using AsEcTiOn, AkEy or any other case pattern. There is a slight 
performance benefit if you do use the correct case.


STARTUP PARAMETERS

Two parameters tell the Configuration package what configuration file to 
look for and where to find it. These are:
·       config.filename
o       Specifies the name of the starting configuration file to use.
Default value is <hostname>.config. xml.
- where hostname is the name of the current machine. Please note that the 
hostname is converted to lower case. This has no effect on Windows but it 
could on another OS.

·       config.location
o       The two valid values are “classpath” and “file”. The default value 
is classpath.
config..location=classpath tells Configuration to look in the classpath for 
the configuration file.

config..location=file tells Configuration to look in the file system for 
the configuration file.

Startup Examples:
1.      Process Config file prod.config.xml in the file system.

·       java Dconfig.file=c:/project/xyz/prod.config.xml 
Dconfig.location=file com.domain.StartServer





2.      Process Config  file prod.config.xml in the classpath. Note that 
these two are the same as classpath is the defaulf config.location.

·       java Dconfig.file=prod.config.xml 
Dconfig.location=classpath  com.domain.StartServer
·       java Dconfig.file=prod.config.xml com.domain.StartServer


3.      Process Config  file hostname.config.xml in the classpath, assuming 
we are running on host named prodserver. Note that these three are the same 
as classpath is the defaulf config.location and prodserver..config.xml is 
the default config.file on host prodserver



·       java Dconfig.file=prodserver..config.xml Dconfig.location=classpath 
com.domain.StartServer
·       java Dconfig.file=prodserver.config.xml com.domain.StartServer
·       java com.domain.StartServer



INITIALIZATION

The first time the Configuration object is invoked it lazily initializes 
the configuration files and builds the configuration dictionary. Subsequent 
calls retrieve values from the dictionary. It would be wise to make a call 
in your startup for better control. Simply add the line:
Config.getInstance()
to invoke initialization.


REPROCESSING and CONFIGURATION LISTENERS

To reprocess Configuration on a running system, call 
Config.getInstance().reprocessConfig().

If you have made changes to your configuration files, these changes will be 
reflected in the current running configuration. You may want to use a jsp 
to invoke this call. Examples are available.

Classes implement the ConfigListener interface to detect when configuration 
changes have been made via reprocessConfig(). For example, your code may 
have Scheduler object that initializes its schedule parameters using 
Configuration. However, if you wish to have the flexibility to change these 
parameters, you may have your Scheduler implement ConfigListener. Then when 
you change the parameters in your configuration file and run 
reprocessConfig(), the Scheduler will know to reprocess its schedule 
parameters.

In another case, you may have a Thread that makes a call to Configuration 
to determine how long to sleep. If it calls once at startup and caches that 
value, it will need to implement ConfigListener to know if the value has 
changed. If it calls Configuration on each iteration, there would be no 
need to implement ConfigListener as it will get the changed value on the 
subsequent invocation.

PERL IMPLEMENTATION

There is a perl implementation of this configuration functionality. This 
allows us to write perl scripts to start, stop, or otherwise control a java 
environment using many of the same configuration values.