[Schema Refactoring] Where are we now ?

[Emmanuel Lecharny <el...@apache.org>]

Some heads up about the Schema Refactoring : The Grand Vision

we need to be able to manipulate SchemaObjects and register them in the 
Registries without keeping the Registries consistent : this is typically 
the case when :
- we are using this part of the code in Studio, allowing people to 
define their schema without being blocked by missing elements
- we are loading the initial schemas and we don't want to have them 
loaded in a specific order
- we are enabling a schema which may not be ordered.

In the last two cases, however, we should not allow the final Registries 
to be broken.

The necessary sequence to guarantee that we have a consistent Registries 
is the following :
 
1) read the schemas and load each SchemaObjects into a temporary 
Registries without check. The SchemaObjects are not yet related to each 
other
(we keep the OID of the referenced SchemaObjects)
2) If we want to check the Registries, we first have to apply the 
Registries to each SchemaObject : if there are some mossing references,
then we store the errors in a list of errors (usefull for debugging or 
for Studio to mark the wrong elements). We also have to check that
other relations are valid : typically, if an AT does not have a Syntax 
nor a SUP, then it's an error.

In order to protect the Registries from concurrent modification, I 
suggest that when we load or remove a SchemaObject from the Registries,
then the Registries is protected for the duration of the operation. It 
can be done either by synchronizing the Reistries instance, but the
best would be to store the reference into another data structure 
(SchemaManager ?) which hide the Registries : nowhere in the code should we
access the Registries directly. Then if this inner Registries is 
declared volatile, we can swap it easily. The Registries modification will
then just be the mater of :
1) Clone the Registries
2) Apply the modifications to the cloned Registries
3) Swap the new Registries with the old one which will be removed (we 
have to be carefull about memory leaks at this point)

As no one should modify a SchemaObject taken from the Registriesn but 
the SchemaManager, we must encapsulate those ShcemaObject into some 
immutable
wrapper. Clonning the Registries will lead to do a complete clonig of 
all the current SchemaObject, as we may change many parts of it.

In order to avoid such a costly operation, we might also allow atomic 
modification on the Registries (SO addition/removal/renaming) as we can 
perfectly
control that the added/deleted/renamed SO will let the Registries 
consistent before changing anything in it. However, we cannot guarantee the
concurrent access protection in this case with a volatile.

We have pretty much all what we need to complete this last phase of the 
SchemaRefactoring. It's just a matter of how long it will take now :)




-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org