You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Alex Karasulu <ak...@apache.org> on 2009/04/13 03:56:46 UTC
[ApacheDS] Enabling any partition implementation for system
Hi all,
I've been rearranging and cleaning up some of the modules and packages
(minimally for now) in my LDIF branch. There are multiple aims behind this
action but one of them is to enable users to be able to specify any
partition implementation to be used as the schema/system partition. I can
now see clearly how this can be done. This email describes how and the
tradeoffs.
Change #1: Merge Schema & System Partitions
----------------------------------------------------------------------
First we have to merge the system and schema partitions into one. There
should only be one partition required to start the server. The ou=schema
area should be relocated to:
ou=schem,ou=system
A single partition containing all critical startup information helps replace
this partition with any implementation. Later this will also help when
implementing our configuration in DIT solution.
Change #2: Alter Initialization and Startup
------------------------------------------------------------
I have modified (and renamed) the init method (now initialize()) on the
Partition interface to only require schema Registries. This was done to
allow Partitions to be initialized and made usable before the
DirectoryService has been started or even created. Before, the Partition
interface required a DirectoryService instance to initialize. There was no
easy way we could provide a hot Partition to the DirectoryService to
startup() itself if Partitions required a live DirectoryService. Chicken
and egg should come to mind.
Let's require the DirectoryService be provided a hot partition, specifically
the system partition. Initialization of the DirectoryService will be
delayed until this Partition is made available to provide the configuration
and schema data needed for bootstrapping the service.
Programmatic Configuration vs. DIT Configuration
-----------------------------------------------------------------------
Zoerner really stumped me with this one at ApacheCon and I did not have an
immediate answer (must have been the beers). Just for the record I will
repost it here:
"When embedding with CiDIT, which configuration is the authoritative
configuration: the programmatic configuration or the configuration in the
DIT?"
Hope this is right Stefan. Please correct me if I'm wrong.
We will take a simple common sense approach. Any application embedding the
DirectoryService will simply delegate configuration to it. Meaning the
configuration in the *system* Partition will drive the wiring and
configuration process. There will be no need to programmatically configure
the DirectoryService. So how can embedding applications alter the stock
configuration that ships with ApacheDS?
(1) With dynamic reconfiguration the stock partition containing a basic
configuration can be altered after the DS is started up by an embedding
application and these changes will persist.
(2) LDIFs can be loaded right after startup to alter the configuration which
will dynamically alter the server.
(3) A custom 'system' partition with the preferred configuration can be
supplied.
Thoughts? Comments?
--
Alex Karasulu
My Blog :: http://www.jroller.com/akarasulu/
Apache Directory Server :: http://directory.apache.org
Apache MINA :: http://mina.apache.org
Re: [ApacheDS] Enabling any partition implementation for system
Posted by Alex Karasulu <ak...@gmail.com>.
Hi Andrea,
On Tue, Apr 14, 2009 at 8:02 AM, Andrea Gariboldi <
andrea.gariboldi@gmail.com> wrote:
> Hi alex,
> back from my vacation :(,
> there are some comments inline.
>
> 2009/4/13, Alex Karasulu <ak...@apache.org>:
>>
>> Hi all,
>>
>>
snip ...
>
> Change #2: Alter Initialization and Startup
>> ------------------------------------------------------------
>>
>> I have modified (and renamed) the init method (now initialize()) on the
>> Partition interface to only require schema Registries. This was done to
>> allow Partitions to be initialized and made usable before the
>> DirectoryService has been started or even created. Before, the Partition
>> interface required a DirectoryService instance to initialize. There was no
>> easy way we could provide a hot Partition to the DirectoryService to
>> startup() itself if Partitions required a live DirectoryService. Chicken
>> and egg should come to mind.
>
>
> Which registries instance are you passing into the partition
> initialization? I mean in the DefaultDirectoryService in the initialize()
> method the registries instance is changed after the schema is loaded:
>
> //
> --------------------------------------------------------------------
> // Initialize schema subsystem and reset registries
> //
> --------------------------------------------------------------------
>
> PartitionSchemaLoader schemaLoader = new PartitionSchemaLoader(
> schemaPartition, registries );
> Registries globalRegistries = new DefaultRegistries( "global",
> schemaLoader, registries.getOidRegistry() );
> schemaLoader.loadEnabled( globalRegistries );
> registries = globalRegistries; //
> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>
> Looks like the first registries instance is used only to check
> JdbmPartition indexed attributes are "known" in the schema, and than
> the schema is repopulated with some kind of filter (but i haven't read the
> code schemaLoader.loadEnabled() yet).
>
Basically you need some sort of minimal bootstrap schema in order to
normalize and search in the schema partition. This is why an initial
Registries object is created and set immediately. This is not don't for all
partitions but for the schema partition which uses a limited set of
attributes we control.
Once the real schema is loaded from the schema partition it is then used to
reset the registries on the schema partition. So hence the registries
should be resettable.
>
> This is why for example i have to keep a directoryService instance in the
> OraclePartition (for ou=system) instead of having a "broken" registries
> instance (caching the registries in the initialization fase).
>
Just expose which we have to now ... a setter to reset the registries.
> So that when i getRegistries() on the directoryService i get the right
> instance.
>
You should be given the right instance after initialization if of course
your partition is used for the schema partition which we will now merge with
system.
> I need registries in the partition only to normalize DN and attributes.
> Specifically the first thing that is asked is to normalize the suffixDN of
> the partition before the DefaultPartitionNexus is bootstrapped or the
> initialization will fail because it will not find the ou=system partition
> when searching for the uid=admin entry existence in the
> DefaultDirectoryService.createBootstrapEntries() method. So sounds like you
> have to initilize the schema before you have a valid registries instance...
> Sounds like this is the reason why we have a separate ou=schema partition,
> so that you can:
>
Right.
>
> create the schema partition -> initialize the registries from schema data
> -> use the registries to initialize other partitions (even ou=system).
>
> that make sense. So may be you have to keep separate the ou=schema and the
> ou=system. So that you have a smaller, simpler, and "normailzed" schema
> partition (using predefined meta-objectclasses and meta-attributeTypes, to
> describe objectclasses and attributeTypes), than a more complex system
> partition (with objectclasses and attributeTypes you loaded from the
> ou=schema partition data).
>
I was worrying about this considerring the schema extensions that will be
added to the server with CiDIT. No longer will we have a controlled set of
schema elements used by entries under the configuration area.
--
Alex Karasulu
My Blog :: http://www.jroller.com/akarasulu/
Apache Directory Server :: http://directory.apache.org
Apache MINA :: http://mina.apache.org
Re: [ApacheDS] Enabling any partition implementation for system
Posted by Andrea Gariboldi <an...@gmail.com>.
Hi alex,
back from my vacation :(,
there are some comments inline.
2009/4/13, Alex Karasulu <ak...@apache.org>:
>
> Hi all,
>
> I've been rearranging and cleaning up some of the modules and packages
> (minimally for now) in my LDIF branch. There are multiple aims behind this
> action but one of them is to enable users to be able to specify any
> partition implementation to be used as the schema/system partition. I can
> now see clearly how this can be done. This email describes how and the
> tradeoffs.
>
>
> Change #1: Merge Schema & System Partitions
> ----------------------------------------------------------------------
>
> First we have to merge the system and schema partitions into one. There
> should only be one partition required to start the server. The ou=schema
> area should be relocated to:
>
> ou=schem,ou=system
>
> A single partition containing all critical startup information helps
> replace this partition with any implementation. Later this will also help
> when implementing our configuration in DIT solution.
>
Change #2: Alter Initialization and Startup
> ------------------------------------------------------------
>
> I have modified (and renamed) the init method (now initialize()) on the
> Partition interface to only require schema Registries. This was done to
> allow Partitions to be initialized and made usable before the
> DirectoryService has been started or even created. Before, the Partition
> interface required a DirectoryService instance to initialize. There was no
> easy way we could provide a hot Partition to the DirectoryService to
> startup() itself if Partitions required a live DirectoryService. Chicken
> and egg should come to mind.
Which registries instance are you passing into the partition initialization?
I mean in the DefaultDirectoryService in the initialize() method the
registries instance is changed after the schema is loaded:
//
--------------------------------------------------------------------
// Initialize schema subsystem and reset registries
//
--------------------------------------------------------------------
PartitionSchemaLoader schemaLoader = new PartitionSchemaLoader(
schemaPartition, registries );
Registries globalRegistries = new DefaultRegistries( "global",
schemaLoader, registries.getOidRegistry() );
schemaLoader.loadEnabled( globalRegistries );
registries = globalRegistries; //
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Looks like the first registries instance is used only to check JdbmPartition
indexed attributes are "known" in the schema, and than
the schema is repopulated with some kind of filter (but i haven't read the
code schemaLoader.loadEnabled() yet).
This is why for example i have to keep a directoryService instance in the
OraclePartition (for ou=system) instead of having a "broken" registries
instance (caching the registries in the initialization fase). So that when i
getRegistries() on the directoryService i get the right instance. I need
registries in the partition only to normalize DN and attributes.
Specifically the first thing that is asked is to normalize the suffixDN of
the partition before the DefaultPartitionNexus is bootstrapped or the
initialization will fail because it will not find the ou=system partition
when searching for the uid=admin entry existence in the
DefaultDirectoryService.createBootstrapEntries() method. So sounds like you
have to initilize the schema before you have a valid registries instance...
Sounds like this is the reason why we have a separate ou=schema partition,
so that you can:
create the schema partition -> initialize the registries from schema data ->
use the registries to initialize other partitions (even ou=system).
that make sense. So may be you have to keep separate the ou=schema and the
ou=system. So that you have a smaller, simpler, and "normailzed" schema
partition (using predefined meta-objectclasses and meta-attributeTypes, to
describe objectclasses and attributeTypes), than a more complex system
partition (with objectclasses and attributeTypes you loaded from the
ou=schema partition data).
Let's require the DirectoryService be provided a hot partition, specifically
> the system partition. Initialization of the DirectoryService will be
> delayed until this Partition is made available to provide the configuration
> and schema data needed for bootstrapping the service.
Programmatic Configuration vs. DIT Configuration
> -----------------------------------------------------------------------
>
> Zoerner really stumped me with this one at ApacheCon and I did not have an
> immediate answer (must have been the beers). Just for the record I will
> repost it here:
>
> "When embedding with CiDIT, which configuration is the authoritative
> configuration: the programmatic configuration or the configuration in the
> DIT?"
>
> Hope this is right Stefan. Please correct me if I'm wrong.
>
> We will take a simple common sense approach. Any application embedding the
> DirectoryService will simply delegate configuration to it. Meaning the
> configuration in the *system* Partition will drive the wiring and
> configuration process. There will be no need to programmatically configure
> the DirectoryService. So how can embedding applications alter the stock
> configuration that ships with ApacheDS?
>
> (1) With dynamic reconfiguration the stock partition containing a basic
> configuration can be altered after the DS is started up by an embedding
> application and these changes will persist.
> (2) LDIFs can be loaded right after startup to alter the configuration
> which will dynamically alter the server.
> (3) A custom 'system' partition with the preferred configuration can be
> supplied.
>
>
> Thoughts? Comments?
>
> --
> Alex Karasulu
> My Blog :: http://www.jroller.com/akarasulu/
> Apache Directory Server :: http://directory.apache.org
> Apache MINA :: http://mina.apache.org
>
>