You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Emmanuel Lecharny <el...@gmail.com> on 2012/02/02 21:10:35 UTC
Some thoughts about the SchemaObjects
Hi guys,
today, we had a long discussion with Pierre-Arnaud about AT, schema and
pathetic state of the planet. The last problem, we can't solve...
For the former issues, which has been raised when we started to try to
extend the API to allow a user to add new Schema elements locally, we
think that we must modify the current data structure for schema objects.
Here are a few brain dump and some examples :
First, the SchemaManager will only manage immutable SchemaObjects (ie,
here, AttributeType), as there is no reason to allow someone to pull a
SchemaObject from the SchemaManager and to modify it on the fly. That
would be destructive for the sc hemaManager user, as it may impact other
users.
Now, for Studio, being able to pull an AT from teh SM, modify this AT
and inject it back to the SM is useful.
We then discussed about Mutable and Immutable schema objects, and how
they can help us solving this issue.
If a user want to modify an existing SchemaObject pulled from the
SchemaManager must first make it mutable :
AttributeType attributeType = schemaManager.getAttributeType( "2.5.4.11" );
MutableAttributeType mat = new MutableAttributeType( attributeType );
In this case, the resulting instance is a copy of the initial immutable
object.
In order to be able to implement such a proposal, the following
hierarchy could be enough :
(SchemaObject)<-------------------(MutableSchemaObject)
o ^
| |
{AbstractSchemaObject} |
^ |
| |
[AttributeType]<------------------[MutableAttributeType]
where (III) are interfaces, {AAA} are abstract classes and [CCC] are
normal classes.
The base implementation is :
o (SchemaObject) expose all the SO getters.
o (MutableSchemaObject) interface expose the SO setters.
o {AbstractSchemaObject} implements the the SO getters
o [AttributeType] implements the AttributeType getters
o [MutableAttributeType] implements the AttributeType setters
(see an exemple at the end of this mail)
With those classes and interface, it's possible to hide the setters for
a user manipulating an AT he got from the SchemaManager, but this user
has the possibility to modify this AT by wrapping it into a new MutableAT.
In order to create new SchemaObject, a user can :
1) create a MutableSchemaObject, and get its immutable copy :
MutableAttributeType mutableAT = new MutableAttributeType();
mutableAT.setXXX( yyy );
...
AttributeType attributeType = new AttributeType( mutableAT );
2) create a new AttributeType using the RFC notation :
AttributeType attributeType = new AttributeType( "( 2.5.4.58 NAME
'attributeCertificateAttribute' DESC 'attribute certificate use ;binary'
SYNTAX 1.3.6.1.4.1.1466.115.121.1.8 )" );
In any case, everything stored in the SchemaManager must be immutable.
public interface SchemaObject
{
int getValue();
}
public abstract class AbstractSchemaObject implements SchemaObject
{
// Protected to hide it from the user but modifiable by a MutableAT
instance
protected int value;
protected AbstractSchemaObject( int value )
{
this.value = value;
}
public int getValue()
{
return value;
}
}
public class AttributeType extends AbstractSchemaObject
{
// This is protected to be modifable by a MutableAT instance
protected String descr;
public AttributeType()
{
super( 1 );
descr = "Test";
}
// A constructor creating an immutable AT from a MutableAT
public AttributeType( MutableAttributeType attributeType )
{
super( attributeType.getValue() );
descr = attributeType.getDescr();
}
public String getDescr()
{
return descr;
}
}
public interface MutableSchemaObject
{
void setValue( int value );
}
public class MutableAttributeType extends AttributeType implements
MutableSchemaObject
{
public MutableAttributeType()
{
}
// A constructor creating a MutableAT from an AT
public MutableAttributeType( AttributeType attributeType )
{
super();
value = attributeType.getValue();
descr = attributeType.getDescr();
}
public void setValue( int value )
{
this.value = value;
}
public void setDescr( String descr )
{
this.descr = descr;
}
}
Thoughts ?
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Some thoughts about the SchemaObjects
Posted by Emmanuel Lécharny <el...@apache.org>.
On 2/4/12 12:21 AM, Alex Karasulu wrote:
> On Thu, Feb 2, 2012 at 10:10 PM, Emmanuel Lecharny<el...@gmail.com>wrote:
>
>> For the former issues, which has been raised when we started to try to
>> extend the API to allow a user to add new Schema elements locally, we think
>> that we must modify the current data structure for schema objects.
>>
>> Here are a few brain dump and some examples :
>>
>> First, the SchemaManager will only manage immutable SchemaObjects (ie,
>> here, AttributeType), as there is no reason to allow someone to pull a
>> SchemaObject from the SchemaManager and to modify it on the fly. That would
>> be destructive for the sc hemaManager user, as it may impact other users.
>>
>>
> Right you don't want to mess with the in memory structure (graph of schema
> objects) that is managed by the schema manager directly.
Only when we expose those schemaObjects to the public. Something that
will become real if we expose the SchemaManager to the public in the API.
On the server, it's a non issue, if we are careful as developpers.
>
>> We then discussed about Mutable and Immutable schema objects, and how they
>> can help us solving this issue.
>>
>> If a user want to modify an existing SchemaObject pulled from the
>> SchemaManager must first make it mutable :
>>
>> AttributeType attributeType = schemaManager.**getAttributeType(
>> "2.5.4.11" );
>> MutableAttributeType mat = new MutableAttributeType( attributeType );
>>
>> In this case, the resulting instance is a copy of the initial immutable
>> object.
>>
>>
> Will the mutable will track the differences (the deltas) in the mutable
> from the original schema object being wrapped?
As the SchemaManager holds only immutable objects, mutable objects are
only copies of those immutable objects. We don't track change, namely.
We can diff them, but I'm not sure it's relevant.
> Thoughts ?
>
>
> I would like to share a view I have in my head about all the in memory
> schema data structures we have. Just a quick review as some points/facts
> first:
>
> (1) We have schema objects that directly reference other schema objects
> resulting a graph of schema objects.
Yes. I have a graph I draw years ago about all the relations between all
the objects. I can share it if needed (not now, I'm in a train but I can
add it on a wiki page).
>
> (2) The designed model of schema objects let's the containment hierarchy
> naturally walk the graph. Like for example looking at the MAY list of an
> ObjectClass will reference actual AttributeType objects in the graph
> connected to the ObjectClass. Further walking the AT object to see it's
> Syntax and MatchingRules does the same.
>
> (3) Registry objects serve as map structures for rapidly indexing into
> pools of schema objects by type based on alias names and their OID.
I would add a very important fact :
4) The only interface to manipulate SchemaObject is the SchemaManager.
Registries are nit anymore exposed to the public, they are internal to
the SchemaManager
>
> NOTE:
> Contained objects like a Syntax referenced by an AttributeType should not
> be directly referenced. Instead the Syntax's OID should be kept in the
> AttributeType and an accessor like getSyntax() should use a lookup via the
> Syntax Registry. This is important from both an OSGi standpoint and to
> easily implement a change mechanism to this grand data structure atomic,
> consistent and isolated.
Hmmm. That would be killing, IMO. Doing a lookup is a costly operation,
and if we can avoid doing it, that would be good. I mean, the
SchemaManager contains only immutable objects (or should contain only
immutable objects), so once those objects are loaded into teh SM, they
is no reason to change anything.
Updating the schema is an operation which is done globally : we create a
new SM, load it with the modified SO, and swap it with the old one if
it's checked and validated. I'm pretty sure that using OSGi does not
create a pb in this context.
>
> ----
>
> I like the Mutator wrappers introduced in this mail thread above: and I
> think it's key to implement a proper change algorithm. I also like the idea
> of them serving to just store deltas and track changes from the original
> immutable objects that they directly reference. This probably will make the
> schema editor code a lot easier to implement.
Well, mutable objects are *not* tracking changes made in immutable
objects. It's just that they use the Immutable objects as a base storage
for fields. It offers an extended 'vision' on the inner data, allowing
the user to modify them.
But you can't take a immutable object, wrap them into a mutable object,
and modify them. To do this implies a copy of the whole immutable object
under the hood.
>
> I see a set of mutators being collected/tracked as a group, then applied in
> an atomic batch to the main data structure after a validation test to
> determine if the resultant graph is consistent.
This is already what we do.
> Then the entire structure
> can be read and write locked, and changes to it from the mutators applied
> throughout, then the structure unlocked.
It's even easier. The SM instance is a volatile, and you simply swap the
newly created SM with the old one. All the threads using the old SM will
of course be impacted, but at some point, we can live with that :
modifications on a schema shloud not be considered as a light operation,
done in production...
> The full structure locking for
> readers and writers is acceptable since we rarely perform schema change
> operations and it maintains a consistent view.
Yes. See above.
>
> We should also fire some events to inform those that need to listen to
> schema changes that something was altered. Sending those listeners the
> deltas or mutator objects might be valuable for them to refresh themselves.
That is a bit more complex to implement, and with the way it is done
currently, I don't see a simple path to implement listeners.
We can discuss this further, but that would require a full rework on the
way the schema is implemented...
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Some thoughts about the SchemaObjects
Posted by Alex Karasulu <ak...@apache.org>.
On Thu, Feb 2, 2012 at 10:10 PM, Emmanuel Lecharny <el...@gmail.com>wrote:
> For the former issues, which has been raised when we started to try to
> extend the API to allow a user to add new Schema elements locally, we think
> that we must modify the current data structure for schema objects.
>
> Here are a few brain dump and some examples :
>
> First, the SchemaManager will only manage immutable SchemaObjects (ie,
> here, AttributeType), as there is no reason to allow someone to pull a
> SchemaObject from the SchemaManager and to modify it on the fly. That would
> be destructive for the sc hemaManager user, as it may impact other users.
>
>
Right you don't want to mess with the in memory structure (graph of schema
objects) that is managed by the schema manager directly.
> Now, for Studio, being able to pull an AT from teh SM, modify this AT and
> inject it back to the SM is useful.
>
Yes for the schema editor we've discussed this a couple times.
>
> We then discussed about Mutable and Immutable schema objects, and how they
> can help us solving this issue.
>
> If a user want to modify an existing SchemaObject pulled from the
> SchemaManager must first make it mutable :
>
> AttributeType attributeType = schemaManager.**getAttributeType(
> "2.5.4.11" );
> MutableAttributeType mat = new MutableAttributeType( attributeType );
>
> In this case, the resulting instance is a copy of the initial immutable
> object.
>
>
Will the mutable will track the differences (the deltas) in the mutable
from the original schema object being wrapped?
> In order to be able to implement such a proposal, the following hierarchy
> could be enough :
>
>
> (SchemaObject)<---------------**----(MutableSchemaObject)
> o ^
> | |
> {AbstractSchemaObject} |
> ^ |
> | |
> [AttributeType]<--------------**----[MutableAttributeType]
>
>
>
> where (III) are interfaces, {AAA} are abstract classes and [CCC] are
> normal classes.
>
> The base implementation is :
>
> o (SchemaObject) expose all the SO getters.
> o (MutableSchemaObject) interface expose the SO setters.
> o {AbstractSchemaObject} implements the the SO getters
> o [AttributeType] implements the AttributeType getters
> o [MutableAttributeType] implements the AttributeType setters
>
>
With you here.
> (see an exemple at the end of this mail)
>
> With those classes and interface, it's possible to hide the setters for a
> user manipulating an AT he got from the SchemaManager, but this user has
> the possibility to modify this AT by wrapping it into a new MutableAT.
>
> In order to create new SchemaObject, a user can :
>
> 1) create a MutableSchemaObject, and get its immutable copy :
>
> MutableAttributeType mutableAT = new MutableAttributeType();
> mutableAT.setXXX( yyy );
> ...
> AttributeType attributeType = new AttributeType( mutableAT );
>
> 2) create a new AttributeType using the RFC notation :
>
> AttributeType attributeType = new AttributeType( "( 2.5.4.58 NAME '**attributeCertificateAttribute'
> DESC 'attribute certificate use ;binary' SYNTAX
> 1.3.6.1.4.1.1466.115.121.1.8 )" );
>
> In any case, everything stored in the SchemaManager must be immutable.
>
>
>
SNIP
> Thoughts ?
>
>
I would like to share a view I have in my head about all the in memory
schema data structures we have. Just a quick review as some points/facts
first:
(1) We have schema objects that directly reference other schema objects
resulting a graph of schema objects.
(2) The designed model of schema objects let's the containment hierarchy
naturally walk the graph. Like for example looking at the MAY list of an
ObjectClass will reference actual AttributeType objects in the graph
connected to the ObjectClass. Further walking the AT object to see it's
Syntax and MatchingRules does the same.
(3) Registry objects serve as map structures for rapidly indexing into
pools of schema objects by type based on alias names and their OID.
NOTE:
Contained objects like a Syntax referenced by an AttributeType should not
be directly referenced. Instead the Syntax's OID should be kept in the
AttributeType and an accessor like getSyntax() should use a lookup via the
Syntax Registry. This is important from both an OSGi standpoint and to
easily implement a change mechanism to this grand data structure atomic,
consistent and isolated.
----
I like the Mutator wrappers introduced in this mail thread above: and I
think it's key to implement a proper change algorithm. I also like the idea
of them serving to just store deltas and track changes from the original
immutable objects that they directly reference. This probably will make the
schema editor code a lot easier to implement.
I see a set of mutators being collected/tracked as a group, then applied in
an atomic batch to the main data structure after a validation test to
determine if the resultant graph is consistent. Then the entire structure
can be read and write locked, and changes to it from the mutators applied
throughout, then the structure unlocked. The full structure locking for
readers and writers is acceptable since we rarely perform schema change
operations and it maintains a consistent view.
We should also fire some events to inform those that need to listen to
schema changes that something was altered. Sending those listeners the
deltas or mutator objects might be valuable for them to refresh themselves.
Thoughts?
--
Best Regards,
-- Alex
Re: Some thoughts about the SchemaObjects
Posted by Alex Karasulu <ak...@apache.org>.
OK I will respond to this ... just haivng access and time issues for past 2
days ... not ignored.
Cheers,
Alex
On Thu, Feb 2, 2012 at 10:10 PM, Emmanuel Lecharny <el...@gmail.com>wrote:
> Hi guys,
>
> today, we had a long discussion with Pierre-Arnaud about AT, schema and
> pathetic state of the planet. The last problem, we can't solve...
>
> For the former issues, which has been raised when we started to try to
> extend the API to allow a user to add new Schema elements locally, we think
> that we must modify the current data structure for schema objects.
>
> Here are a few brain dump and some examples :
>
> First, the SchemaManager will only manage immutable SchemaObjects (ie,
> here, AttributeType), as there is no reason to allow someone to pull a
> SchemaObject from the SchemaManager and to modify it on the fly. That would
> be destructive for the sc hemaManager user, as it may impact other users.
>
> Now, for Studio, being able to pull an AT from teh SM, modify this AT and
> inject it back to the SM is useful.
>
> We then discussed about Mutable and Immutable schema objects, and how they
> can help us solving this issue.
>
> If a user want to modify an existing SchemaObject pulled from the
> SchemaManager must first make it mutable :
>
> AttributeType attributeType = schemaManager.**getAttributeType(
> "2.5.4.11" );
> MutableAttributeType mat = new MutableAttributeType( attributeType );
>
> In this case, the resulting instance is a copy of the initial immutable
> object.
>
> In order to be able to implement such a proposal, the following hierarchy
> could be enough :
>
>
> (SchemaObject)<---------------**----(MutableSchemaObject)
> o ^
> | |
> {AbstractSchemaObject} |
> ^ |
> | |
> [AttributeType]<--------------**----[MutableAttributeType]
>
>
>
> where (III) are interfaces, {AAA} are abstract classes and [CCC] are
> normal classes.
>
> The base implementation is :
>
> o (SchemaObject) expose all the SO getters.
> o (MutableSchemaObject) interface expose the SO setters.
> o {AbstractSchemaObject} implements the the SO getters
> o [AttributeType] implements the AttributeType getters
> o [MutableAttributeType] implements the AttributeType setters
>
> (see an exemple at the end of this mail)
>
> With those classes and interface, it's possible to hide the setters for a
> user manipulating an AT he got from the SchemaManager, but this user has
> the possibility to modify this AT by wrapping it into a new MutableAT.
>
> In order to create new SchemaObject, a user can :
>
> 1) create a MutableSchemaObject, and get its immutable copy :
>
> MutableAttributeType mutableAT = new MutableAttributeType();
> mutableAT.setXXX( yyy );
> ...
> AttributeType attributeType = new AttributeType( mutableAT );
>
> 2) create a new AttributeType using the RFC notation :
>
> AttributeType attributeType = new AttributeType( "( 2.5.4.58 NAME '**attributeCertificateAttribute'
> DESC 'attribute certificate use ;binary' SYNTAX
> 1.3.6.1.4.1.1466.115.121.1.8 )" );
>
> In any case, everything stored in the SchemaManager must be immutable.
>
>
> public interface SchemaObject
> {
> int getValue();
> }
>
> public abstract class AbstractSchemaObject implements SchemaObject
> {
> // Protected to hide it from the user but modifiable by a MutableAT
> instance
> protected int value;
>
> protected AbstractSchemaObject( int value )
> {
> this.value = value;
> }
>
> public int getValue()
> {
> return value;
> }
> }
>
> public class AttributeType extends AbstractSchemaObject
> {
> // This is protected to be modifable by a MutableAT instance
> protected String descr;
>
> public AttributeType()
> {
> super( 1 );
> descr = "Test";
> }
>
> // A constructor creating an immutable AT from a MutableAT
> public AttributeType( MutableAttributeType attributeType )
> {
> super( attributeType.getValue() );
>
> descr = attributeType.getDescr();
> }
>
> public String getDescr()
> {
> return descr;
> }
> }
>
>
> public interface MutableSchemaObject
> {
> void setValue( int value );
> }
>
>
> public class MutableAttributeType extends AttributeType implements
> MutableSchemaObject
> {
> public MutableAttributeType()
> {
> }
>
> // A constructor creating a MutableAT from an AT
> public MutableAttributeType( AttributeType attributeType )
> {
> super();
>
> value = attributeType.getValue();
> descr = attributeType.getDescr();
> }
>
> public void setValue( int value )
> {
> this.value = value;
> }
>
> public void setDescr( String descr )
> {
> this.descr = descr;
> }
> }
>
>
> Thoughts ?
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
>
--
Best Regards,
-- Alex