[jira] Updated: (DIRSERVER-760) reading .schema files at server start-up

["Norval Hope (JIRA)" <ji...@apache.org>]

     [ http://issues.apache.org/jira/browse/DIRSERVER-760?page=all ]

Norval Hope updated DIRSERVER-760:
----------------------------------

    Attachment: schema_branch.patch

Here is an initial patch of a working server (at least given the testing I've done) against https://svn.apache.org/repos/asf/directory/branches/apacheds-schema , where no output from the Maven schema plugin is used in the server.xml file (I ran ./apache/server_main/apacheds.bat).

This is obviously a work in progress to show the job can be done, and one possible implementational approach. There are a number of significant questions that still need to be addressed by some developers more knowledgeable about ApacheDS then myself:
  1. Maven support: currently I ran by looking up the .schema files in ./apache/server_main/conf/*.schema (configured via ./apache/server_main/server.xml, which I also copied into the installers dir). I copied these files manually to test, but presume they should be copied from apache/core/src/main/schema to somewhere else for packaging in all supported release packages. Even if the standard .schema files are packaged in .jars, I think it is important that a dir outside of jars can be easily configured to accept extra .schema files not included in the standard build.
    a. Currently extra content (not attribute types or object classes) that is associated with a .schema file, but which must be manually maintained is found by looking for a matching ???SchemaAdditions class (eg SystemSchemaAdditions associated with "00_system.schema") using reflection. A more explicit approach would be better but simply using Spring is problematic as the major variable in the manual configurations is the code (normalizer/comparator etc) for each item, and having a separate class for each them is most probably not a viable approach.
  2. Tidy up: I haven't removed any code made defunct by this patch, but there is some (eg BootstrapSchemaLoader). There was some ThreadLocal support in this class which I don't understand the motivation for, so I may have missed something.
    a. I don't understand the reasoning behind the BootstrapRegistries versus the GlobalRegistries, and it seems to me that ultimately a single Registries flavour should be supported, and it should support schema being contributed to the server dynamically after it has started up (for instance on a custom partition being dynamically activated).
     b. Some refactoring of project modules will most probably be required if my implementation approach is accepted. In particular the apache/core-plugin module should perhaps be deleted and the Antlr support in it moved to apache/core.
  3. Once more knowledgeable peeps then I have weighed in so that the details are clearer, I'm happy to document the details.

> reading .schema files at server start-up
> ----------------------------------------
>
>                 Key: DIRSERVER-760
>                 URL: http://issues.apache.org/jira/browse/DIRSERVER-760
>             Project: Directory ApacheDS
>          Issue Type: New Feature
>          Components: core
>    Affects Versions: 1.5.1
>         Environment: N/A
>            Reporter: Norval Hope
>         Attachments: schema_branch.patch, schema_loader.patch
>
>
> I am submitting a patch for a feature I required, and which I've seen a number of queries about on the DEV list. Currently the only way to get new .schema information into ApacheDS is to use the Maven2 plugin, requiring a rebuild of the server (not to mention working around problems like methods being generated which are too long for Java to handle).
> Instead this patch adds support for reading of  specified .schema files from server.xml at server startup, via a new interface SchemaLoader and a default implementation FileSystemSchemaLoader. The latter supports reading all files matching a specified regex in a specified directory (in which case users must be careful to ensure that files appear lexicographically before their dependant files, ala init.rc in Linux) or reading an ordered list of file names under a specified directory. An example server.xml snippet is as follows:
>     <property name="bootstrapSchemas">
>       <set>
>         <bean class="org.apache.directory.server.core.schema.bootstrap.AutofsSchema"/>
>         <bean class="org.apache.directory.server.core.schema.bootstrap.CorbaSchema"/>
>         <bean class="org.apache.directory.server.core.schema.bootstrap.CoreSchema"/>
>         <bean class="org.apache.directory.server.core.schema.bootstrap.CosineSchema"/>
>         <bean class="org.apache.directory.server.core.schema.bootstrap.ApacheSchema"/>
>         <bean class="org.apache.directory.server.core.schema.bootstrap.CollectiveSchema"/>
>         <bean class="org.apache.directory.server.core.schema.bootstrap.InetorgpersonSchema"/>
>         <bean class="org.apache.directory.server.core.schema.bootstrap.JavaSchema"/>
>         <bean class="org.apache.directory.server.core.schema.bootstrap.Krb5kdcSchema"/>
>         <bean class="org.apache.directory.server.core.schema.bootstrap.NisSchema"/>
>         <bean class="org.apache.directory.server.core.schema.bootstrap.SystemSchema"/>
>         <bean class="org.apache.directory.server.core.schema.bootstrap.ApachednsSchema"/>
>       </set>
>     </property>
>     <property name="schemaLoaders">
>         <list>
>             <bean class="org.apache.directory.server.core.schema.FileSystemSchemaLoader">
>                 <constructor-arg><value>./</value></constructor-arg>
>                 <!--<constructor-arg><value>.*_openldap.schema</value></constructor-arg>-->
>                 <constructor-arg>
>                     <list>
>                         <value>my1_openldap.schema</value>
>                         <value>my2_openldap.schema</value>
>                     </list>
>                 </constructor-arg>
>             </bean>
>         </list>
>     </property>
> noting that the Maven2 style approach is of course still supported where desired. A list is used so that multiple SchemaLoader implementations can be appended if required (which I do). 
> I have also included SchemaFromSearchConverter which can read a schema from another directory via conventional query, which may come in useful for people using ApacheDS as a virtual directory container of sorts. It is not required by the core patch and can be excluded if no generic use is seen for it.
> A few issues are still outstanding:
>    1. Is the patch seen as worthwhile by the DEV list and/or are there any issues with my implementation of it?
>    2. At it's core the patch uses the OpenLdapSchemaParser to read the .schema files, which is currently only available in the Maven2 plugin artifact     
> <groupId>org.apache.directory.server</groupId>
> <artifactId>apacheds-core-plugin</artifactId>
>    
>     which is a bit clunky as it means people need to include this jar in their classpath (or in the UberJar). Perhaps this parsing component should be separated from the rest of the maven plugin to tidy this up (I don't know maven well enough to do this myself).
>    
>    3. If the feature and implementation are ok, what further steps should I take re preparing the patch for inclusion?
> Thanks (by the way I was previously known as Norbert Reilly)

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira