You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Alex Karasulu <ak...@apache.org> on 2006/12/06 19:35:06 UTC
Convo on IRC about schema
Hi all,
This was a convo about schema info on IRC. My buffer does however cut
some of the conversation.
Alex
<akarasulu> :)
<elecharny> not a big deal
<elecharny> ok
<akarasulu> even with OID you cannot have two different OID with same names
<elecharny> so each partition has its own subset of the common schema
<akarasulu> you know this list of names each entity can have
<akarasulu> like cn and commonName for example
<elecharny> yeah
<elecharny> you can't reuse cn for something else
<elecharny> you should have only one cn in the world
<elecharny> like you have only one 2.5.4.5
<ersiner> Let me do X.500++ in my new thesis :)
<elecharny> yeah, good idea :)
<akarasulu> exactly
<elecharny> ok, so what were we talking about at the beginning ?
<ersiner> we came here from need of parsers
<elecharny> let say we have only one partition, cn=example,dc=com
<ersiner> alex was explaining that we would use them in two places
<elecharny> ahhh, yes, the parsers
<akarasulu> so we need those parsers
<elecharny> so having parsers will be used if a partition admin (or
somebody allowed to modify the partition schema)
<elecharny> want to add a new AT or OC
<akarasulu> for not only the description syntaxes defined by the
protocol but we're going to add new description syntaxes
<akarasulu> we're going to add one for each of the following :
<akarasulu> comparators
<akarasulu> normalizers
<akarasulu> syntaxCheckers
<akarasulu> these descriptions will have either a FQCN value in it (for
built in entities bundled with ADS) or a BYTECODE value (base 64
encoded) in it
<elecharny> the funny thing is that ifyou have to create a new SC,
<akarasulu> these will be used to store user added schema constructs
that need bytecode to be loaded from them
<elecharny> you will have to write a java class for it
<akarasulu> yeah this is for new SC
<akarasulu> yeah
<elecharny> and calssload it into ADS
<akarasulu> exactly
<akarasulu> well this mechanism can be used
<akarasulu> you could use this for extended OpenLDAP file format too btw
<elecharny> and then, you may generate the schema data out of the class :)
<elecharny> for instance, for a SC, all you need is :
<elecharny> - the SC OID,
<akarasulu> this way though we completely describe all the elements
needed for describing a new schema
<elecharny> and the SC description
<akarasulu> well not really
<akarasulu> elecharny listening?
<ersiner> guys i need to go offline for ph.d bullshit
<elecharny> yep
<akarasulu> np
<akarasulu> bye ersiner
<ersiner> cul8r
<akarasulu> elecharny let me clarify ...
<elecharny> we will keep this convo and post it to you ersiner
<ersiner> oh thanks
<ersiner> really.
<ersiner> bye.
<-- ersiner has quit ("Ex-Chat")
<akarasulu> We want to be able to dynamically load a newly added SC
without having to restart the server right?
<elecharny> yes
<akarasulu> someone adds a new schema
<akarasulu> they add normalizers and comparators and SCs with it
<elecharny> yes
<akarasulu> so we want it so this shit is loaded immediately into memory
and a SC is instantiated and added to it's registry
<akarasulu> so it is made available
<elecharny> yes
<akarasulu> and can be seen via cn=schema at least
<elecharny> yes
<akarasulu> if not by other partition specific subentries
<akarasulu> so we need to invent a new description syntax for these
entities (SC, N, and C)
<akarasulu> for example ...
<akarasulu> ( 1.2.34.32.2.21 BYTECODE '12341ab8923c09efa932' )
<akarasulu> this can be an SC for example for checking for correct
social security numbers
<akarasulu> for example
<akarasulu> make sense?
<akarasulu> another representation may be ...
<elecharny> you already have these descriptions in RFC 4517
<akarasulu> ( 1.2.34.32.2.21 FQCN
org.apache.directory.server.schema.SocialSecuritySyntaxChecker )
<elecharny> ah, sorry, no
<elecharny> ok
<akarasulu> no no this is specific to ADS
<elecharny> yeah
<elecharny> my bad
<akarasulu> we're extending schema this way
<akarasulu> to allow dynamic changes
<akarasulu> no LDAP server on the market can do this
<elecharny> yeah yeah yeah
<akarasulu> i.e. to make SUN DS do this you have to write your own
plugins for it and restart it
<elecharny> ok
<elecharny> but then, what I said is that
<akarasulu> so we will have 100% extendible and dymamic schema
<akarasulu> shoot
<elecharny> as soon as you inject this ( 1.2.34.32.2.21 BYTECODE
'12341ab8923c09efa932' ) object
<akarasulu> ahha
<elecharny> where the bytecodes represent the compiled java class
<akarasulu> ahha
<elecharny> then you can extract out of it all the information regarding
the SC
<akarasulu> exactly
<elecharny> (if these bytcodes is a SC compiled class)
<elecharny> so no more need to inject the SC for that
<akarasulu> yeah we will do verification at the point this new
syntaxCheckers attribute value is added
<elecharny> because we can generate them on the fly too
<akarasulu> yeah exactly
<elecharny> let me explain
<elecharny> to see if we are on the same line
<akarasulu> ok listening
<elecharny> for instance, we want to inject the BooleanSyntaxChecker object
<elecharny> (ok, it's already existing, but suppose we are creating it)
<akarasulu> well this is a built in
<akarasulu> ok ok
<akarasulu> np
<elecharny> we will compile the BooleanSyntaxChecker.java
<elecharny> producting some bytecode
<akarasulu> aha
<akarasulu> yep
<elecharny> and we will inject it into ADS
<akarasulu> yep
<elecharny> now, you have :
<elecharny> ( 1.3.6.1.4.1.1466.115.121.1.7 DESC 'Boolean' )
<elecharny> which is the definition of the Boolean SC in RFC 4517
<akarasulu> yeah this is the syntax not the SC ;)
<elecharny> (yes, but let me ficnish ;)
<akarasulu> this a value in the syntaxes attribute
<akarasulu> ok
<elecharny> what I mean is that now, you can read the compiled class
using introspection,
<elecharny> extract it's OID (which is stored,
<elecharny> and suppose we also added a desc member
<elecharny> we can extract the desc field,
<akarasulu> ahha
<elecharny> and inject into cn=schema this Boolean Syntax
<akarasulu> well I see what you're thinking
<elecharny> we have just built from the bytecode
<akarasulu> take a step back for a second
<elecharny> so no need to have a Syntax Checker Parser, because we have
injected the binary object
<akarasulu> hmmm
<elecharny> from which we will be able to regenrate this SC syntax
<elecharny> ok, stepping back
<akarasulu> well you're mixing a few things together
<akarasulu> let me clarify this picture
<akarasulu> first a SC is not a Syntax
<akarasulu> looking at the protocol way to add schema elements
<elecharny> sorry, I was talking about syntax definitions
<akarasulu> you cannot add a new syntax to the syntaxes attribute unless
you have a SC for that syntax
<akarasulu> but technically you could just generate the syntax once you
have the SC
<elecharny> yes
<akarasulu> but this is mixing up two steps
<elecharny> this is what I had in mind
<akarasulu> hmmm let me think
<akarasulu> so you add the SC and that automatically generates the
Syntax for that SC
<elecharny> let's go back to basic
<akarasulu> is this what you mean?
<elecharny> you want to inject an entry
<akarasulu> wait don't go forward
<akarasulu> let's not mix up concepts
<elecharny> ok
<elecharny> sorry
<akarasulu> you're mechanism is a good idea that can basically reduce a
2 step process into a 1 step process
<akarasulu> so instead of adding a SC then adding a S you just add an SC
<akarasulu> this is what you mean?
<elecharny> what I wanted to say is that when you describe a new
attributeType, you have to describe the syntax it must respect
<elecharny> for instance, the attributeType :
<akarasulu> and use reflection to determine all the info for the Syntax
like it's DESC value etc ?
<elecharny> yeah
<akarasulu> yeah now you are mixing into AT stuff
<akarasulu> ok this will confuse us more
<akarasulu> let's finish SC verses S concept first
<akarasulu> IMHO this shit is very complex
<elecharny> I just wanted to say that you NEED to know the Syntax OID
before using it in an AttributeType
<akarasulu> yeah which means ...\
<akarasulu> you must add the syntax before you can add the AT
<elecharny> yeah
<akarasulu> but this is another subject
<elecharny> correct
<akarasulu> unreated to what we are currently discussing
<elecharny> true
<akarasulu> now SC vs. S
<akarasulu> I think this shit is complex and confusing
<elecharny> just wanted to be clear, but I'm just moving the mud :)
<elecharny> ok.
<akarasulu> let's not take shortcuts
<akarasulu> or overload meanings
<akarasulu> let's be mechanical about it
<elecharny> so in a standard approach, you first declare the S, and then
load the SC
<akarasulu> hmmm no
<akarasulu> you first do SC
<akarasulu> then do S
<akarasulu> SC btw use OID of the S
<akarasulu> SC does not have it's own OID
<akarasulu> this is how the association takes place
<elecharny> so you must declare S before SC if Sc use S :)
<akarasulu> not really
<elecharny> SC is just the implementation
<akarasulu> remember S depends on SC in the object relationship sense
<akarasulu> yeah a Syntax object has a handle on the SC
<elecharny> so S comes first :)
<akarasulu> Syntax is a valid schema object with DESC and all the other
stuff
<akarasulu> well here's how we simplify it
<akarasulu> you add SC first
<akarasulu> then S
<akarasulu> you cannot add an S to the server before the SC
<akarasulu> here is why
<akarasulu> let me explain it has to do with the protocol
<elecharny> ahhh
<akarasulu> we cannot change the S syntax because this is mandated by
the protocol
<elecharny> I get it
<akarasulu> right?
<elecharny> I'm talking about specs, you are talking about loading those
guys into ADS
<elecharny> in this way, I agree
<akarasulu> well I'm talking both
<elecharny> you first load SC,
<elecharny> and then you declare S
<akarasulu> I am constrained by many things you're not thinking about at
this point
<akarasulu> let me explain its important
<akarasulu> I want you to have this in mind fully
<akarasulu> basically
<akarasulu> ...
<akarasulu> (1) protocol demans a certain syntax for syntaxDescriptions
and we cannot change this
<akarasulu> (2) We cannot just add a new syntax using the protocol's
representation with a syntaxDescrition because it's not a functional
representation of a syntax. Meaning we cannot do anything with (
1.23.4.2 DESC 'blah blah blah' ). The server cannot check values with
just this string. It needs a syntaxChecker object associated with this
syntax to functionally enforce this new syntax.
<elecharny> yes, I know that.
<akarasulu> So SC must be added before an S can be added.
<akarasulu> If someone adds an S and there is no SC for the OID of that
S then the server must reject that add.
<akarasulu> because the server cannot enforce that syntax
<elecharny> yes
<akarasulu> this is why we have this break up of S and SC
<elecharny> ok.
<elecharny> but the strange thing about the protocol,
<elecharny> is taht it defines a syntax for S,
<akarasulu> because of both protocol limitations and the need to allow
for protocol based syntax updates that extend the server
<elecharny> but does bnot explain how to add a new S :)
<akarasulu> yeah the protocol sucks
<akarasulu> exactly
<elecharny> not a big deal, thoough
<akarasulu> protocol was not even remotely aware of how to dynamically
change the code of the server
<akarasulu> the guys who wrote it did not know about Java
<akarasulu> :D
<elecharny> let say it will be meaningfull for exposing the S when you
request the schema
<akarasulu> yeah exactly
<akarasulu> people will see that a syntax is defined for it
<elecharny> ok. At this point, we agree that if we can introspect SC, we
can generate S
<akarasulu> we can but IMHO this is bad
<akarasulu> I recommend absolutely no shortcuts or overloading of
actions or meanings for simplicity
<akarasulu> this stuff is too complex
<akarasulu> leave it so user must load SC then load S
<akarasulu> or load both at the same time
<akarasulu> do you agree ?
<elecharny> ok
<elecharny> not a big deal, though
<akarasulu> you may not in which case we can discuss it
<elecharny> we can ask the user to load SC then S
<elecharny> it's not really a PITA
<akarasulu> well hopefull we let LDAP studio do this
<elecharny> I just tried to see if we can avoid one of those two steps
<akarasulu> if it works with elcipse you can design and write your java SC
<elecharny> yes
<akarasulu> and then LS will load it for u into live server
<elecharny> sure
<akarasulu> LS can make it seem like one step
<akarasulu> really for me I am looking to keep all functionality simple
<elecharny> ok
<akarasulu> not overloaded with meaning
<elecharny> I agree
<elecharny> the point is that then you will need thos efucking parsers :)
<akarasulu> yeah it's ok
<akarasulu> the string is simple
<elecharny> (and I wanted to avoid writting them ;)
<elecharny> ok.
<elecharny> we just have to implement those parser in syntaxCheckers,
<akarasulu> well we need either an SC or S parser
<elecharny> where they are described in RFC 4517
<akarasulu> if we overload
<akarasulu> if not then we can write parsers for both S and SC
<elecharny> we don't have to write a SC parser
<akarasulu> I recommend doing it for both the syntax of these
descriptions are really simple
<akarasulu> yeah we do
<akarasulu> we need to write S and SC
<elecharny> the SC is the implementation
<akarasulu> no we need it because user can add a SC via the subentry
<elecharny> but what will be the SC syntax then ???
<akarasulu> ( 1.12.3.21.3 BYTECODE '23aae23a543bc343' ) for example
<elecharny> ahhh, it's OUR syntax
<akarasulu> yeah
<elecharny> ok ok ok
<akarasulu> there will be two forms
<elecharny> I thought you were talking about one existing RFC syntax
<akarasulu> one for existing classes in the server that come packaged
with ADS like so ...
<elecharny> ok, I agree.
<elecharny> man, I have to go.
<akarasulu> ( 1.12.3.21.3 FQCN
org.apache.directory.server.XYZSyntaxChecker ) for example
<elecharny> can you copy/paste all this interesting convo and send it to
the list ?
<akarasulu> so one uses FQCN and other uses BYTECODE
Re: Convo on IRC about schema
Posted by Alex Karasulu <ak...@apache.org>.
Emmanuel Lecharny wrote:
> Hi Ersin,
>
> sorry, but my buffer was also limited to 300 lines... I increased it to
> 3000, but I'm sorry to tell you that your interesting conribution has
> been lost forever, except for us :(
We need an IRC bot to collect this stuff and publish it online for
reference.
Alex
Re: Convo on IRC about schema
Posted by Emmanuel Lecharny <el...@gmail.com>.
Hi Ersin,
sorry, but my buffer was also limited to 300 lines... I increased it to
3000, but I'm sorry to tell you that your interesting conribution has been
lost forever, except for us :(
Thanks anyway, it was quite an interesting convo, and it has helped me to
better understad some parts of the schema system.
And sorry if I was quite confusing, I had some strange misconception about
what we have to do, because of my lack of knolewdge in huge part of X500 :(
Emmanuel
On 12/6/06, Ersin Er <er...@gmail.com> wrote:
>
> It would be better if your buffer did not cut the portion of the convo
> which I talked much more and told about much more useful stuff :-)
>
> On 12/6/06, Alex Karasulu <ak...@apache.org> wrote:
> >
> > Hi all,
> >
> > This was a convo about schema info on IRC. My buffer does however cut
> > some of the conversation.
> >
> > Alex
> >
> >
> > <akarasulu> :)
> > <elecharny> not a big deal
> > <elecharny> ok
> > <akarasulu> even with OID you cannot have two different OID with same
> > names
> > <elecharny> so each partition has its own subset of the common schema
> > <akarasulu> you know this list of names each entity can have
> > <akarasulu> like cn and commonName for example
> > <elecharny> yeah
> > <elecharny> you can't reuse cn for something else
> > <elecharny> you should have only one cn in the world
> > <elecharny> like you have only one 2.5.4.5
> > <ersiner> Let me do X.500++ in my new thesis :)
> > <elecharny> yeah, good idea :)
> > <akarasulu> exactly
> > <elecharny> ok, so what were we talking about at the beginning ?
> > <ersiner> we came here from need of parsers
> > <elecharny> let say we have only one partition, cn=example,dc=com
> > <ersiner> alex was explaining that we would use them in two places
> > <elecharny> ahhh, yes, the parsers
> > <akarasulu> so we need those parsers
> > <elecharny> so having parsers will be used if a partition admin (or
> > somebody allowed to modify the partition schema)
> > <elecharny> want to add a new AT or OC
> > <akarasulu> for not only the description syntaxes defined by the
> > protocol but we're going to add new description syntaxes
> > <akarasulu> we're going to add one for each of the following :
> > <akarasulu> comparators
> > <akarasulu> normalizers
> > <akarasulu> syntaxCheckers
> > <akarasulu> these descriptions will have either a FQCN value in it (for
> > built in entities bundled with ADS) or a BYTECODE value (base 64
> > encoded) in it
> > <elecharny> the funny thing is that ifyou have to create a new SC,
> > <akarasulu> these will be used to store user added schema constructs
> > that need bytecode to be loaded from them
> > <elecharny> you will have to write a java class for it
> > <akarasulu> yeah this is for new SC
> > <akarasulu> yeah
> > <elecharny> and calssload it into ADS
> > <akarasulu> exactly
> > <akarasulu> well this mechanism can be used
> > <akarasulu> you could use this for extended OpenLDAP file format too btw
> >
> > <elecharny> and then, you may generate the schema data out of the class
> > :)
> > <elecharny> for instance, for a SC, all you need is :
> > <elecharny> - the SC OID,
> > <akarasulu> this way though we completely describe all the elements
> > needed for describing a new schema
> > <elecharny> and the SC description
> > <akarasulu> well not really
> > <akarasulu> elecharny listening?
> > <ersiner> guys i need to go offline for ph.d bullshit
> > <elecharny> yep
> > <akarasulu> np
> > <akarasulu> bye ersiner
> > <ersiner> cul8r
> > <akarasulu> elecharny let me clarify ...
> > <elecharny> we will keep this convo and post it to you ersiner
> > <ersiner> oh thanks
> > <ersiner> really.
> > <ersiner> bye.
> > <-- ersiner has quit ("Ex-Chat")
> > <akarasulu> We want to be able to dynamically load a newly added SC
> > without having to restart the server right?
> > <elecharny> yes
> > <akarasulu> someone adds a new schema
> > <akarasulu> they add normalizers and comparators and SCs with it
> > <elecharny> yes
> > <akarasulu> so we want it so this shit is loaded immediately into memory
> >
> > and a SC is instantiated and added to it's registry
> > <akarasulu> so it is made available
> > <elecharny> yes
> > <akarasulu> and can be seen via cn=schema at least
> > <elecharny> yes
> > <akarasulu> if not by other partition specific subentries
> > <akarasulu> so we need to invent a new description syntax for these
> > entities (SC, N, and C)
> > <akarasulu> for example ...
> > <akarasulu> ( 1.2.34.32.2.21 BYTECODE '12341ab8923c09efa932' )
> > <akarasulu> this can be an SC for example for checking for correct
> > social security numbers
> > <akarasulu> for example
> > <akarasulu> make sense?
> > <akarasulu> another representation may be ...
> > <elecharny> you already have these descriptions in RFC 4517
> > <akarasulu> ( 1.2.34.32.2.21 FQCN
> > org.apache.directory.server.schema.SocialSecuritySyntaxChecker )
> > <elecharny> ah, sorry, no
> > <elecharny> ok
> > <akarasulu> no no this is specific to ADS
> > <elecharny> yeah
> > <elecharny> my bad
> > <akarasulu> we're extending schema this way
> > <akarasulu> to allow dynamic changes
> > <akarasulu> no LDAP server on the market can do this
> > <elecharny> yeah yeah yeah
> > <akarasulu> i.e. to make SUN DS do this you have to write your own
> > plugins for it and restart it
> > <elecharny> ok
> > <elecharny> but then, what I said is that
> > <akarasulu> so we will have 100% extendible and dymamic schema
> > <akarasulu> shoot
> > <elecharny> as soon as you inject this ( 1.2.34.32.2.21 BYTECODE
> > '12341ab8923c09efa932' ) object
> > <akarasulu> ahha
> > <elecharny> where the bytecodes represent the compiled java class
> > <akarasulu> ahha
> > <elecharny> then you can extract out of it all the information regarding
> > the SC
> > <akarasulu> exactly
> > <elecharny> (if these bytcodes is a SC compiled class)
> > <elecharny> so no more need to inject the SC for that
> > <akarasulu> yeah we will do verification at the point this new
> > syntaxCheckers attribute value is added
> > <elecharny> because we can generate them on the fly too
> > <akarasulu> yeah exactly
> > <elecharny> let me explain
> > <elecharny> to see if we are on the same line
> > <akarasulu> ok listening
> > <elecharny> for instance, we want to inject the BooleanSyntaxChecker
> > object
> > <elecharny> (ok, it's already existing, but suppose we are creating it)
> > <akarasulu> well this is a built in
> > <akarasulu> ok ok
> > <akarasulu> np
> > <elecharny> we will compile the BooleanSyntaxChecker.java
> > <elecharny> producting some bytecode
> > <akarasulu> aha
> > <akarasulu> yep
> > <elecharny> and we will inject it into ADS
> > <akarasulu> yep
> > <elecharny> now, you have :
> > <elecharny> ( 1.3.6.1.4.1.1466.115.121.1.7 DESC 'Boolean' )
> > <elecharny> which is the definition of the Boolean SC in RFC 4517
> > <akarasulu> yeah this is the syntax not the SC ;)
> > <elecharny> (yes, but let me ficnish ;)
> > <akarasulu> this a value in the syntaxes attribute
> > <akarasulu> ok
> > <elecharny> what I mean is that now, you can read the compiled class
> > using introspection,
> > <elecharny> extract it's OID (which is stored,
> > <elecharny> and suppose we also added a desc member
> > <elecharny> we can extract the desc field,
> > <akarasulu> ahha
> > <elecharny> and inject into cn=schema this Boolean Syntax
> > <akarasulu> well I see what you're thinking
> > <elecharny> we have just built from the bytecode
> > <akarasulu> take a step back for a second
> > <elecharny> so no need to have a Syntax Checker Parser, because we have
> > injected the binary object
> > <akarasulu> hmmm
> > <elecharny> from which we will be able to regenrate this SC syntax
> > <elecharny> ok, stepping back
> > <akarasulu> well you're mixing a few things together
> > <akarasulu> let me clarify this picture
> > <akarasulu> first a SC is not a Syntax
> > <akarasulu> looking at the protocol way to add schema elements
> > <elecharny> sorry, I was talking about syntax definitions
> > <akarasulu> you cannot add a new syntax to the syntaxes attribute unless
> > you have a SC for that syntax
> > <akarasulu> but technically you could just generate the syntax once you
> > have the SC
> > <elecharny> yes
> > <akarasulu> but this is mixing up two steps
> > <elecharny> this is what I had in mind
> > <akarasulu> hmmm let me think
> > <akarasulu> so you add the SC and that automatically generates the
> > Syntax for that SC
> > <elecharny> let's go back to basic
> > <akarasulu> is this what you mean?
> > <elecharny> you want to inject an entry
> > <akarasulu> wait don't go forward
> > <akarasulu> let's not mix up concepts
> > <elecharny> ok
> > <elecharny> sorry
> > <akarasulu> you're mechanism is a good idea that can basically reduce a
> > 2 step process into a 1 step process
> > <akarasulu> so instead of adding a SC then adding a S you just add an SC
> >
> > <akarasulu> this is what you mean?
> > <elecharny> what I wanted to say is that when you describe a new
> > attributeType, you have to describe the syntax it must respect
> > <elecharny> for instance, the attributeType :
> > <akarasulu> and use reflection to determine all the info for the Syntax
> > like it's DESC value etc ?
> > <elecharny> yeah
> > <akarasulu> yeah now you are mixing into AT stuff
> > <akarasulu> ok this will confuse us more
> > <akarasulu> let's finish SC verses S concept first
> > <akarasulu> IMHO this shit is very complex
> > <elecharny> I just wanted to say that you NEED to know the Syntax OID
> > before using it in an AttributeType
> > <akarasulu> yeah which means ...\
> > <akarasulu> you must add the syntax before you can add the AT
> > <elecharny> yeah
> > <akarasulu> but this is another subject
> > <elecharny> correct
> > <akarasulu> unreated to what we are currently discussing
> > <elecharny> true
> > <akarasulu> now SC vs. S
> > <akarasulu> I think this shit is complex and confusing
> > <elecharny> just wanted to be clear, but I'm just moving the mud :)
> > <elecharny> ok.
> > <akarasulu> let's not take shortcuts
> > <akarasulu> or overload meanings
> > <akarasulu> let's be mechanical about it
> > <elecharny> so in a standard approach, you first declare the S, and then
> >
> > load the SC
> > <akarasulu> hmmm no
> > <akarasulu> you first do SC
> > <akarasulu> then do S
> > <akarasulu> SC btw use OID of the S
> > <akarasulu> SC does not have it's own OID
> > <akarasulu> this is how the association takes place
> > <elecharny> so you must declare S before SC if Sc use S :)
> > <akarasulu> not really
> > <elecharny> SC is just the implementation
> > <akarasulu> remember S depends on SC in the object relationship sense
> > <akarasulu> yeah a Syntax object has a handle on the SC
> > <elecharny> so S comes first :)
> > <akarasulu> Syntax is a valid schema object with DESC and all the other
> > stuff
> > <akarasulu> well here's how we simplify it
> > <akarasulu> you add SC first
> > <akarasulu> then S
> > <akarasulu> you cannot add an S to the server before the SC
> > <akarasulu> here is why
> > <akarasulu> let me explain it has to do with the protocol
> > <elecharny> ahhh
> > <akarasulu> we cannot change the S syntax because this is mandated by
> > the protocol
> > <elecharny> I get it
> > <akarasulu> right?
> > <elecharny> I'm talking about specs, you are talking about loading those
> >
> > guys into ADS
> > <elecharny> in this way, I agree
> > <akarasulu> well I'm talking both
> > <elecharny> you first load SC,
> > <elecharny> and then you declare S
> > <akarasulu> I am constrained by many things you're not thinking about at
> >
> > this point
> > <akarasulu> let me explain its important
> > <akarasulu> I want you to have this in mind fully
> > <akarasulu> basically
> > <akarasulu> ...
> > <akarasulu> (1) protocol demans a certain syntax for syntaxDescriptions
> > and we cannot change this
> > <akarasulu> (2) We cannot just add a new syntax using the protocol's
> > representation with a syntaxDescrition because it's not a functional
> > representation of a syntax. Meaning we cannot do anything with (
> > 1.23.4.2 DESC 'blah blah blah' ). The server cannot check values with
> > just this string. It needs a syntaxChecker object associated with this
> > syntax to functionally enforce this new syntax.
> > <elecharny> yes, I know that.
> > <akarasulu> So SC must be added before an S can be added.
> > <akarasulu> If someone adds an S and there is no SC for the OID of that
> > S then the server must reject that add.
> > <akarasulu> because the server cannot enforce that syntax
> > <elecharny> yes
> > <akarasulu> this is why we have this break up of S and SC
> > <elecharny> ok.
> > <elecharny> but the strange thing about the protocol,
> > <elecharny> is taht it defines a syntax for S,
> > <akarasulu> because of both protocol limitations and the need to allow
> > for protocol based syntax updates that extend the server
> > <elecharny> but does bnot explain how to add a new S :)
> > <akarasulu> yeah the protocol sucks
> > <akarasulu> exactly
> > <elecharny> not a big deal, thoough
> > <akarasulu> protocol was not even remotely aware of how to dynamically
> > change the code of the server
> > <akarasulu> the guys who wrote it did not know about Java
> > <akarasulu> :D
> > <elecharny> let say it will be meaningfull for exposing the S when you
> > request the schema
> > <akarasulu> yeah exactly
> > <akarasulu> people will see that a syntax is defined for it
> > <elecharny> ok. At this point, we agree that if we can introspect SC, we
> > can generate S
> > <akarasulu> we can but IMHO this is bad
> > <akarasulu> I recommend absolutely no shortcuts or overloading of
> > actions or meanings for simplicity
> > <akarasulu> this stuff is too complex
> > <akarasulu> leave it so user must load SC then load S
> > <akarasulu> or load both at the same time
> > <akarasulu> do you agree ?
> > <elecharny> ok
> > <elecharny> not a big deal, though
> > <akarasulu> you may not in which case we can discuss it
> > <elecharny> we can ask the user to load SC then S
> > <elecharny> it's not really a PITA
> > <akarasulu> well hopefull we let LDAP studio do this
> > <elecharny> I just tried to see if we can avoid one of those two steps
> > <akarasulu> if it works with elcipse you can design and write your java
> > SC
> > <elecharny> yes
> > <akarasulu> and then LS will load it for u into live server
> > <elecharny> sure
> > <akarasulu> LS can make it seem like one step
> > <akarasulu> really for me I am looking to keep all functionality simple
> > <elecharny> ok
> > <akarasulu> not overloaded with meaning
> > <elecharny> I agree
> > <elecharny> the point is that then you will need thos efucking parsers
> > :)
> > <akarasulu> yeah it's ok
> > <akarasulu> the string is simple
> > <elecharny> (and I wanted to avoid writting them ;)
> > <elecharny> ok.
> > <elecharny> we just have to implement those parser in syntaxCheckers,
> > <akarasulu> well we need either an SC or S parser
> > <elecharny> where they are described in RFC 4517
> > <akarasulu> if we overload
> > <akarasulu> if not then we can write parsers for both S and SC
> > <elecharny> we don't have to write a SC parser
> > <akarasulu> I recommend doing it for both the syntax of these
> > descriptions are really simple
> > <akarasulu> yeah we do
> > <akarasulu> we need to write S and SC
> > <elecharny> the SC is the implementation
> > <akarasulu> no we need it because user can add a SC via the subentry
> > <elecharny> but what will be the SC syntax then ???
> > <akarasulu> ( 1.12.3.21.3 BYTECODE '23aae23a543bc343' ) for example
> > <elecharny> ahhh, it's OUR syntax
> > <akarasulu> yeah
> > <elecharny> ok ok ok
> > <akarasulu> there will be two forms
> > <elecharny> I thought you were talking about one existing RFC syntax
> > <akarasulu> one for existing classes in the server that come packaged
> > with ADS like so ...
> > <elecharny> ok, I agree.
> > <elecharny> man, I have to go.
> > <akarasulu> ( 1.12.3.21.3 FQCN
> > org.apache.directory.server.XYZSyntaxChecker ) for example
> > <elecharny> can you copy/paste all this interesting convo and send it to
> >
> > the list ?
> > <akarasulu> so one uses FQCN and other uses BYTECODE
> >
> >
> >
>
>
> --
> Ersin
--
Cordialement,
Emmanuel Lécharny
Re: Convo on IRC about schema
Posted by Ersin Er <er...@gmail.com>.
It would be better if your buffer did not cut the portion of the convo which
I talked much more and told about much more useful stuff :-)
On 12/6/06, Alex Karasulu <ak...@apache.org> wrote:
>
> Hi all,
>
> This was a convo about schema info on IRC. My buffer does however cut
> some of the conversation.
>
> Alex
>
>
> <akarasulu> :)
> <elecharny> not a big deal
> <elecharny> ok
> <akarasulu> even with OID you cannot have two different OID with same
> names
> <elecharny> so each partition has its own subset of the common schema
> <akarasulu> you know this list of names each entity can have
> <akarasulu> like cn and commonName for example
> <elecharny> yeah
> <elecharny> you can't reuse cn for something else
> <elecharny> you should have only one cn in the world
> <elecharny> like you have only one 2.5.4.5
> <ersiner> Let me do X.500++ in my new thesis :)
> <elecharny> yeah, good idea :)
> <akarasulu> exactly
> <elecharny> ok, so what were we talking about at the beginning ?
> <ersiner> we came here from need of parsers
> <elecharny> let say we have only one partition, cn=example,dc=com
> <ersiner> alex was explaining that we would use them in two places
> <elecharny> ahhh, yes, the parsers
> <akarasulu> so we need those parsers
> <elecharny> so having parsers will be used if a partition admin (or
> somebody allowed to modify the partition schema)
> <elecharny> want to add a new AT or OC
> <akarasulu> for not only the description syntaxes defined by the
> protocol but we're going to add new description syntaxes
> <akarasulu> we're going to add one for each of the following :
> <akarasulu> comparators
> <akarasulu> normalizers
> <akarasulu> syntaxCheckers
> <akarasulu> these descriptions will have either a FQCN value in it (for
> built in entities bundled with ADS) or a BYTECODE value (base 64
> encoded) in it
> <elecharny> the funny thing is that ifyou have to create a new SC,
> <akarasulu> these will be used to store user added schema constructs
> that need bytecode to be loaded from them
> <elecharny> you will have to write a java class for it
> <akarasulu> yeah this is for new SC
> <akarasulu> yeah
> <elecharny> and calssload it into ADS
> <akarasulu> exactly
> <akarasulu> well this mechanism can be used
> <akarasulu> you could use this for extended OpenLDAP file format too btw
> <elecharny> and then, you may generate the schema data out of the class :)
> <elecharny> for instance, for a SC, all you need is :
> <elecharny> - the SC OID,
> <akarasulu> this way though we completely describe all the elements
> needed for describing a new schema
> <elecharny> and the SC description
> <akarasulu> well not really
> <akarasulu> elecharny listening?
> <ersiner> guys i need to go offline for ph.d bullshit
> <elecharny> yep
> <akarasulu> np
> <akarasulu> bye ersiner
> <ersiner> cul8r
> <akarasulu> elecharny let me clarify ...
> <elecharny> we will keep this convo and post it to you ersiner
> <ersiner> oh thanks
> <ersiner> really.
> <ersiner> bye.
> <-- ersiner has quit ("Ex-Chat")
> <akarasulu> We want to be able to dynamically load a newly added SC
> without having to restart the server right?
> <elecharny> yes
> <akarasulu> someone adds a new schema
> <akarasulu> they add normalizers and comparators and SCs with it
> <elecharny> yes
> <akarasulu> so we want it so this shit is loaded immediately into memory
> and a SC is instantiated and added to it's registry
> <akarasulu> so it is made available
> <elecharny> yes
> <akarasulu> and can be seen via cn=schema at least
> <elecharny> yes
> <akarasulu> if not by other partition specific subentries
> <akarasulu> so we need to invent a new description syntax for these
> entities (SC, N, and C)
> <akarasulu> for example ...
> <akarasulu> ( 1.2.34.32.2.21 BYTECODE '12341ab8923c09efa932' )
> <akarasulu> this can be an SC for example for checking for correct
> social security numbers
> <akarasulu> for example
> <akarasulu> make sense?
> <akarasulu> another representation may be ...
> <elecharny> you already have these descriptions in RFC 4517
> <akarasulu> ( 1.2.34.32.2.21 FQCN
> org.apache.directory.server.schema.SocialSecuritySyntaxChecker )
> <elecharny> ah, sorry, no
> <elecharny> ok
> <akarasulu> no no this is specific to ADS
> <elecharny> yeah
> <elecharny> my bad
> <akarasulu> we're extending schema this way
> <akarasulu> to allow dynamic changes
> <akarasulu> no LDAP server on the market can do this
> <elecharny> yeah yeah yeah
> <akarasulu> i.e. to make SUN DS do this you have to write your own
> plugins for it and restart it
> <elecharny> ok
> <elecharny> but then, what I said is that
> <akarasulu> so we will have 100% extendible and dymamic schema
> <akarasulu> shoot
> <elecharny> as soon as you inject this ( 1.2.34.32.2.21 BYTECODE
> '12341ab8923c09efa932' ) object
> <akarasulu> ahha
> <elecharny> where the bytecodes represent the compiled java class
> <akarasulu> ahha
> <elecharny> then you can extract out of it all the information regarding
> the SC
> <akarasulu> exactly
> <elecharny> (if these bytcodes is a SC compiled class)
> <elecharny> so no more need to inject the SC for that
> <akarasulu> yeah we will do verification at the point this new
> syntaxCheckers attribute value is added
> <elecharny> because we can generate them on the fly too
> <akarasulu> yeah exactly
> <elecharny> let me explain
> <elecharny> to see if we are on the same line
> <akarasulu> ok listening
> <elecharny> for instance, we want to inject the BooleanSyntaxChecker
> object
> <elecharny> (ok, it's already existing, but suppose we are creating it)
> <akarasulu> well this is a built in
> <akarasulu> ok ok
> <akarasulu> np
> <elecharny> we will compile the BooleanSyntaxChecker.java
> <elecharny> producting some bytecode
> <akarasulu> aha
> <akarasulu> yep
> <elecharny> and we will inject it into ADS
> <akarasulu> yep
> <elecharny> now, you have :
> <elecharny> ( 1.3.6.1.4.1.1466.115.121.1.7 DESC 'Boolean' )
> <elecharny> which is the definition of the Boolean SC in RFC 4517
> <akarasulu> yeah this is the syntax not the SC ;)
> <elecharny> (yes, but let me ficnish ;)
> <akarasulu> this a value in the syntaxes attribute
> <akarasulu> ok
> <elecharny> what I mean is that now, you can read the compiled class
> using introspection,
> <elecharny> extract it's OID (which is stored,
> <elecharny> and suppose we also added a desc member
> <elecharny> we can extract the desc field,
> <akarasulu> ahha
> <elecharny> and inject into cn=schema this Boolean Syntax
> <akarasulu> well I see what you're thinking
> <elecharny> we have just built from the bytecode
> <akarasulu> take a step back for a second
> <elecharny> so no need to have a Syntax Checker Parser, because we have
> injected the binary object
> <akarasulu> hmmm
> <elecharny> from which we will be able to regenrate this SC syntax
> <elecharny> ok, stepping back
> <akarasulu> well you're mixing a few things together
> <akarasulu> let me clarify this picture
> <akarasulu> first a SC is not a Syntax
> <akarasulu> looking at the protocol way to add schema elements
> <elecharny> sorry, I was talking about syntax definitions
> <akarasulu> you cannot add a new syntax to the syntaxes attribute unless
> you have a SC for that syntax
> <akarasulu> but technically you could just generate the syntax once you
> have the SC
> <elecharny> yes
> <akarasulu> but this is mixing up two steps
> <elecharny> this is what I had in mind
> <akarasulu> hmmm let me think
> <akarasulu> so you add the SC and that automatically generates the
> Syntax for that SC
> <elecharny> let's go back to basic
> <akarasulu> is this what you mean?
> <elecharny> you want to inject an entry
> <akarasulu> wait don't go forward
> <akarasulu> let's not mix up concepts
> <elecharny> ok
> <elecharny> sorry
> <akarasulu> you're mechanism is a good idea that can basically reduce a
> 2 step process into a 1 step process
> <akarasulu> so instead of adding a SC then adding a S you just add an SC
> <akarasulu> this is what you mean?
> <elecharny> what I wanted to say is that when you describe a new
> attributeType, you have to describe the syntax it must respect
> <elecharny> for instance, the attributeType :
> <akarasulu> and use reflection to determine all the info for the Syntax
> like it's DESC value etc ?
> <elecharny> yeah
> <akarasulu> yeah now you are mixing into AT stuff
> <akarasulu> ok this will confuse us more
> <akarasulu> let's finish SC verses S concept first
> <akarasulu> IMHO this shit is very complex
> <elecharny> I just wanted to say that you NEED to know the Syntax OID
> before using it in an AttributeType
> <akarasulu> yeah which means ...\
> <akarasulu> you must add the syntax before you can add the AT
> <elecharny> yeah
> <akarasulu> but this is another subject
> <elecharny> correct
> <akarasulu> unreated to what we are currently discussing
> <elecharny> true
> <akarasulu> now SC vs. S
> <akarasulu> I think this shit is complex and confusing
> <elecharny> just wanted to be clear, but I'm just moving the mud :)
> <elecharny> ok.
> <akarasulu> let's not take shortcuts
> <akarasulu> or overload meanings
> <akarasulu> let's be mechanical about it
> <elecharny> so in a standard approach, you first declare the S, and then
> load the SC
> <akarasulu> hmmm no
> <akarasulu> you first do SC
> <akarasulu> then do S
> <akarasulu> SC btw use OID of the S
> <akarasulu> SC does not have it's own OID
> <akarasulu> this is how the association takes place
> <elecharny> so you must declare S before SC if Sc use S :)
> <akarasulu> not really
> <elecharny> SC is just the implementation
> <akarasulu> remember S depends on SC in the object relationship sense
> <akarasulu> yeah a Syntax object has a handle on the SC
> <elecharny> so S comes first :)
> <akarasulu> Syntax is a valid schema object with DESC and all the other
> stuff
> <akarasulu> well here's how we simplify it
> <akarasulu> you add SC first
> <akarasulu> then S
> <akarasulu> you cannot add an S to the server before the SC
> <akarasulu> here is why
> <akarasulu> let me explain it has to do with the protocol
> <elecharny> ahhh
> <akarasulu> we cannot change the S syntax because this is mandated by
> the protocol
> <elecharny> I get it
> <akarasulu> right?
> <elecharny> I'm talking about specs, you are talking about loading those
> guys into ADS
> <elecharny> in this way, I agree
> <akarasulu> well I'm talking both
> <elecharny> you first load SC,
> <elecharny> and then you declare S
> <akarasulu> I am constrained by many things you're not thinking about at
> this point
> <akarasulu> let me explain its important
> <akarasulu> I want you to have this in mind fully
> <akarasulu> basically
> <akarasulu> ...
> <akarasulu> (1) protocol demans a certain syntax for syntaxDescriptions
> and we cannot change this
> <akarasulu> (2) We cannot just add a new syntax using the protocol's
> representation with a syntaxDescrition because it's not a functional
> representation of a syntax. Meaning we cannot do anything with (
> 1.23.4.2 DESC 'blah blah blah' ). The server cannot check values with
> just this string. It needs a syntaxChecker object associated with this
> syntax to functionally enforce this new syntax.
> <elecharny> yes, I know that.
> <akarasulu> So SC must be added before an S can be added.
> <akarasulu> If someone adds an S and there is no SC for the OID of that
> S then the server must reject that add.
> <akarasulu> because the server cannot enforce that syntax
> <elecharny> yes
> <akarasulu> this is why we have this break up of S and SC
> <elecharny> ok.
> <elecharny> but the strange thing about the protocol,
> <elecharny> is taht it defines a syntax for S,
> <akarasulu> because of both protocol limitations and the need to allow
> for protocol based syntax updates that extend the server
> <elecharny> but does bnot explain how to add a new S :)
> <akarasulu> yeah the protocol sucks
> <akarasulu> exactly
> <elecharny> not a big deal, thoough
> <akarasulu> protocol was not even remotely aware of how to dynamically
> change the code of the server
> <akarasulu> the guys who wrote it did not know about Java
> <akarasulu> :D
> <elecharny> let say it will be meaningfull for exposing the S when you
> request the schema
> <akarasulu> yeah exactly
> <akarasulu> people will see that a syntax is defined for it
> <elecharny> ok. At this point, we agree that if we can introspect SC, we
> can generate S
> <akarasulu> we can but IMHO this is bad
> <akarasulu> I recommend absolutely no shortcuts or overloading of
> actions or meanings for simplicity
> <akarasulu> this stuff is too complex
> <akarasulu> leave it so user must load SC then load S
> <akarasulu> or load both at the same time
> <akarasulu> do you agree ?
> <elecharny> ok
> <elecharny> not a big deal, though
> <akarasulu> you may not in which case we can discuss it
> <elecharny> we can ask the user to load SC then S
> <elecharny> it's not really a PITA
> <akarasulu> well hopefull we let LDAP studio do this
> <elecharny> I just tried to see if we can avoid one of those two steps
> <akarasulu> if it works with elcipse you can design and write your java SC
> <elecharny> yes
> <akarasulu> and then LS will load it for u into live server
> <elecharny> sure
> <akarasulu> LS can make it seem like one step
> <akarasulu> really for me I am looking to keep all functionality simple
> <elecharny> ok
> <akarasulu> not overloaded with meaning
> <elecharny> I agree
> <elecharny> the point is that then you will need thos efucking parsers :)
> <akarasulu> yeah it's ok
> <akarasulu> the string is simple
> <elecharny> (and I wanted to avoid writting them ;)
> <elecharny> ok.
> <elecharny> we just have to implement those parser in syntaxCheckers,
> <akarasulu> well we need either an SC or S parser
> <elecharny> where they are described in RFC 4517
> <akarasulu> if we overload
> <akarasulu> if not then we can write parsers for both S and SC
> <elecharny> we don't have to write a SC parser
> <akarasulu> I recommend doing it for both the syntax of these
> descriptions are really simple
> <akarasulu> yeah we do
> <akarasulu> we need to write S and SC
> <elecharny> the SC is the implementation
> <akarasulu> no we need it because user can add a SC via the subentry
> <elecharny> but what will be the SC syntax then ???
> <akarasulu> ( 1.12.3.21.3 BYTECODE '23aae23a543bc343' ) for example
> <elecharny> ahhh, it's OUR syntax
> <akarasulu> yeah
> <elecharny> ok ok ok
> <akarasulu> there will be two forms
> <elecharny> I thought you were talking about one existing RFC syntax
> <akarasulu> one for existing classes in the server that come packaged
> with ADS like so ...
> <elecharny> ok, I agree.
> <elecharny> man, I have to go.
> <akarasulu> ( 1.12.3.21.3 FQCN
> org.apache.directory.server.XYZSyntaxChecker ) for example
> <elecharny> can you copy/paste all this interesting convo and send it to
> the list ?
> <akarasulu> so one uses FQCN and other uses BYTECODE
>
>
>
--
Ersin