You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Emmanuel Lecharny <el...@gmail.com> on 2010/10/31 17:42:22 UTC
Moving the configuration beans to core-api
Hi guys,
now that the configuration branch has been merged back to trunk (with
some bump in the road, I must admit : I just fixed some badly merged
file, which caused some errors), I'm now checking some classes that may
need a bit of refactoring : the Index hierarchy.
There is a base Index class, with two main children : AvlIndex and
JdbmIndex. There is a third child, named genericIndex, which is only
used to deal with junit annotations used to create a new server instance
(it's a holder class). I was wondering if it would not be better to use
the created IndexBean we have in server-config, but obviously, we can't
if the beans are in server-config.
I suggest that all those beans are moved to core-api, then used by the
core/server-annotations projects to store the server configuration.
That would help simplifying the Index hierarchy.
thougts ?
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Moving the configuration beans to core-api
Posted by Emmanuel Lecharny <el...@gmail.com>.
On 10/31/10 6:51 PM, Kiran Ayyagari wrote:
> hi Emmanuel,
>
> On 10/31/10, Emmanuel Lecharny<el...@gmail.com> wrote:
>> Hi guys,
>>
>> now that the configuration branch has been merged back to trunk (with
>> some bump in the road, I must admit : I just fixed some badly merged
> IMO np, what you have done was huge in the branch
>> file, which caused some errors), I'm now checking some classes that may
>> need a bit of refactoring : the Index hierarchy.
>>
>> There is a base Index class, with two main children : AvlIndex and
>> JdbmIndex. There is a third child, named genericIndex, which is only
>> used to deal with junit annotations used to create a new server instance
>> (it's a holder class). I was wondering if it would not be better to use
>> the created IndexBean we have in server-config, but obviously, we can't
>> if the beans are in server-config.
>>
>> I suggest that all those beans are moved to core-api, then used by the
>> core/server-annotations projects to store the server configuration.
>>
>> That would help simplifying the Index hierarchy.
>>
> I initially agreed for move on IRC but on a second though after
> reading this mail I think we
> shouldn't do this:
>
> 1. Moving all the beans to core-api *might* make other developers to use them
> everywhere in the code (be it in our server or in their apps) which IMHO not
> desirable for us.
Hmmm, that's right.
> 2. Having GenericIndex makes creating a index implementation easier and we can
> introduce a type parameter in the @CreateIndex annotation to
> support various
> index implementations. e.x @CreateIndex( type=AvlIndex.class, ..)
Well, I still think we should get rid of the GenericIndex, bt if we can
default to using JdbmIndex if we don't have any type parameter, then we
can remove the GenericIndex and not use the IndexBean either.
Seems fine. Will try to do that this way.
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Moving the configuration beans to core-api
Posted by Kiran Ayyagari <ka...@apache.org>.
hi Emmanuel,
On 10/31/10, Emmanuel Lecharny <el...@gmail.com> wrote:
> Hi guys,
>
> now that the configuration branch has been merged back to trunk (with
> some bump in the road, I must admit : I just fixed some badly merged
IMO np, what you have done was huge in the branch
> file, which caused some errors), I'm now checking some classes that may
> need a bit of refactoring : the Index hierarchy.
>
> There is a base Index class, with two main children : AvlIndex and
> JdbmIndex. There is a third child, named genericIndex, which is only
> used to deal with junit annotations used to create a new server instance
> (it's a holder class). I was wondering if it would not be better to use
> the created IndexBean we have in server-config, but obviously, we can't
> if the beans are in server-config.
>
> I suggest that all those beans are moved to core-api, then used by the
> core/server-annotations projects to store the server configuration.
>
> That would help simplifying the Index hierarchy.
>
I initially agreed for move on IRC but on a second though after
reading this mail I think we
shouldn't do this:
1. Moving all the beans to core-api *might* make other developers to use them
everywhere in the code (be it in our server or in their apps) which IMHO not
desirable for us.
2. Having GenericIndex makes creating a index implementation easier and we can
introduce a type parameter in the @CreateIndex annotation to
support various
index implementations. e.x @CreateIndex( type=AvlIndex.class, ..)
thanks for the great work on the configuration setup
--
Kiran Ayyagari
Re: Moving the configuration beans to core-api
Posted by Emmanuel Lecharny <el...@gmail.com>.
On 11/1/10 8:51 AM, Kiran Ayyagari wrote:
> On 11/1/10, Emmanuel Lecharny<el...@gmail.com> wrote:
>> One thing I forgot to say : the current configReader is not able to
>> process anything but JdbmIndex atm. It has to evolve to be able to do so
>> (probably using reflection to do so)
> in order to support all those 'dynamic extension' things talked about
> in this thread
> we have to depend extensively on reflection,
> e.x instead of having a separate OC for each index implementation type
> we need to
> have a single OC like 'ads-index' with a MUST AT 'ads-indexClass' so
> that we can
> instantiate the class using default constructor and then inject all
> the configured attribute
> values into that object.
>
This is feasable. We have to do the same for the Partition, too.
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Moving the configuration beans to core-api
Posted by Emmanuel Lécharny <el...@apache.org>.
On 11/1/10 11:36 AM, Alex Karasulu wrote:
> On Mon, Nov 1, 2010 at 12:29 PM, Emmanuel Lecharny<el...@gmail.com>wrote:
>
>> On 11/1/10 11:04 AM, Alex Karasulu wrote:
>>
>>> On Mon, Nov 1, 2010 at 9:51 AM, Kiran Ayyagari<ka...@apache.org>
>>> wrote:
>>>
>>> On 11/1/10, Emmanuel Lecharny<el...@gmail.com> wrote:
>>>>> One thing I forgot to say : the current configReader is not able to
>>>>> process anything but JdbmIndex atm. It has to evolve to be able to do so
>>>>> (probably using reflection to do so)
>>>>>
>>>> in order to support all those 'dynamic extension' things talked about
>>>> in this thread
>>>> we have to depend extensively on reflection,
>>>> e.x instead of having a separate OC for each index implementation type
>>>> we need to
>>>> have a single OC like 'ads-index' with a MUST AT 'ads-indexClass' so
>>>> that we can
>>>> instantiate the class using default constructor and then inject all
>>>> the configured attribute
>>>> values into that object.
>>>>
>>>> Yes exactly! This has to be done for all extension points in the server.
>>> What are those extension points?
>>>
>>> - Partitions and their associated classes.
>>> - Interceptors and their associated classes.
>>> - Authenticators and their associated classes.
>>> - LDAP Protocol Handlers and their associated classes.
>>>
>>> I guess there might be more extension points but this is what I see off
>>> the
>>> top of my head. And these things need to be using reflection with the same
>>> MUST AT to class load the respective implementation that users might
>>> provide.
>>>
>> There are more extensions point : ExtendedOperations, Schema
>> normalizers/SyntaxCheckers/Compartors.
>>
>>
> ExtendedOperations are part of the LDAP Protocol Handlers.
>
> And we don't need an extension point for Schema since the Schema Subsystem
> already has this capability built in with updates via the protocol to extend
> Schema. The schema extensions are directly loaded into the DIT and a
> reflection mechanism is already in place to load them appropriately.
Tight.
Anyway, we have to focus on Partitions/Store/Index atm, this is the key
element at stake atm.
Note : this is *not* urgent. I opened the discussion, but I'm not sure
we want to do that before 2.0-RC1 is out.
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Moving the configuration beans to core-api
Posted by Alex Karasulu <ak...@apache.org>.
On Mon, Nov 1, 2010 at 12:29 PM, Emmanuel Lecharny <el...@gmail.com>wrote:
> On 11/1/10 11:04 AM, Alex Karasulu wrote:
>
>> On Mon, Nov 1, 2010 at 9:51 AM, Kiran Ayyagari<ka...@apache.org>
>> wrote:
>>
>> On 11/1/10, Emmanuel Lecharny<el...@gmail.com> wrote:
>>>
>>>> One thing I forgot to say : the current configReader is not able to
>>>> process anything but JdbmIndex atm. It has to evolve to be able to do so
>>>> (probably using reflection to do so)
>>>>
>>> in order to support all those 'dynamic extension' things talked about
>>> in this thread
>>> we have to depend extensively on reflection,
>>> e.x instead of having a separate OC for each index implementation type
>>> we need to
>>> have a single OC like 'ads-index' with a MUST AT 'ads-indexClass' so
>>> that we can
>>> instantiate the class using default constructor and then inject all
>>> the configured attribute
>>> values into that object.
>>>
>>> Yes exactly! This has to be done for all extension points in the server.
>> What are those extension points?
>>
>> - Partitions and their associated classes.
>> - Interceptors and their associated classes.
>> - Authenticators and their associated classes.
>> - LDAP Protocol Handlers and their associated classes.
>>
>> I guess there might be more extension points but this is what I see off
>> the
>> top of my head. And these things need to be using reflection with the same
>> MUST AT to class load the respective implementation that users might
>> provide.
>>
> There are more extensions point : ExtendedOperations, Schema
> normalizers/SyntaxCheckers/Compartors.
>
>
ExtendedOperations are part of the LDAP Protocol Handlers.
And we don't need an extension point for Schema since the Schema Subsystem
already has this capability built in with updates via the protocol to extend
Schema. The schema extensions are directly loaded into the DIT and a
reflection mechanism is already in place to load them appropriately.
--
Alex Karasulu
My Blog :: http://www.jroller.com/akarasulu/
Apache Directory Server :: http://directory.apache.org
Apache MINA :: http://mina.apache.org
To set up a meeting with me: http://tungle.me/AlexKarasulu
Re: Moving the configuration beans to core-api
Posted by Emmanuel Lecharny <el...@gmail.com>.
On 11/1/10 11:04 AM, Alex Karasulu wrote:
> On Mon, Nov 1, 2010 at 9:51 AM, Kiran Ayyagari<ka...@apache.org> wrote:
>
>> On 11/1/10, Emmanuel Lecharny<el...@gmail.com> wrote:
>>> One thing I forgot to say : the current configReader is not able to
>>> process anything but JdbmIndex atm. It has to evolve to be able to do so
>>> (probably using reflection to do so)
>> in order to support all those 'dynamic extension' things talked about
>> in this thread
>> we have to depend extensively on reflection,
>> e.x instead of having a separate OC for each index implementation type
>> we need to
>> have a single OC like 'ads-index' with a MUST AT 'ads-indexClass' so
>> that we can
>> instantiate the class using default constructor and then inject all
>> the configured attribute
>> values into that object.
>>
> Yes exactly! This has to be done for all extension points in the server.
> What are those extension points?
>
> - Partitions and their associated classes.
> - Interceptors and their associated classes.
> - Authenticators and their associated classes.
> - LDAP Protocol Handlers and their associated classes.
>
> I guess there might be more extension points but this is what I see off the
> top of my head. And these things need to be using reflection with the same
> MUST AT to class load the respective implementation that users might
> provide.
There are more extensions point : ExtendedOperations, Schema
normalizers/SyntaxCheckers/Compartors.
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Moving the configuration beans to core-api
Posted by Alex Karasulu <ak...@apache.org>.
On Mon, Nov 1, 2010 at 9:51 AM, Kiran Ayyagari <ka...@apache.org> wrote:
> On 11/1/10, Emmanuel Lecharny <el...@gmail.com> wrote:
> > One thing I forgot to say : the current configReader is not able to
> > process anything but JdbmIndex atm. It has to evolve to be able to do so
> > (probably using reflection to do so)
> in order to support all those 'dynamic extension' things talked about
> in this thread
> we have to depend extensively on reflection,
> e.x instead of having a separate OC for each index implementation type
> we need to
> have a single OC like 'ads-index' with a MUST AT 'ads-indexClass' so
> that we can
> instantiate the class using default constructor and then inject all
> the configured attribute
> values into that object.
>
Yes exactly! This has to be done for all extension points in the server.
What are those extension points?
- Partitions and their associated classes.
- Interceptors and their associated classes.
- Authenticators and their associated classes.
- LDAP Protocol Handlers and their associated classes.
I guess there might be more extension points but this is what I see off the
top of my head. And these things need to be using reflection with the same
MUST AT to class load the respective implementation that users might
provide.
--
Alex Karasulu
My Blog :: http://www.jroller.com/akarasulu/
Apache Directory Server :: http://directory.apache.org
Apache MINA :: http://mina.apache.org
To set up a meeting with me: http://tungle.me/AlexKarasulu
Re: Moving the configuration beans to core-api
Posted by Kiran Ayyagari <ka...@apache.org>.
On 11/1/10, Emmanuel Lecharny <el...@gmail.com> wrote:
> One thing I forgot to say : the current configReader is not able to
> process anything but JdbmIndex atm. It has to evolve to be able to do so
> (probably using reflection to do so)
in order to support all those 'dynamic extension' things talked about
in this thread
we have to depend extensively on reflection,
e.x instead of having a separate OC for each index implementation type
we need to
have a single OC like 'ads-index' with a MUST AT 'ads-indexClass' so
that we can
instantiate the class using default constructor and then inject all
the configured attribute
values into that object.
--
Kiran Ayyagari
Re: Moving the configuration beans to core-api
Posted by Emmanuel Lecharny <el...@gmail.com>.
One thing I forgot to say : the current configReader is not able to
process anything but JdbmIndex atm. It has to evolve to be able to do so
(probably using reflection to do so)
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Moving the configuration beans to core-api
Posted by Emmanuel Lécharny <el...@apache.org>.
On 10/31/10 10:38 PM, Alex Karasulu wrote:
> On Sun, Oct 31, 2010 at 11:11 PM, Emmanuel Lécharny<el...@apache.org>wrote:
>
>> On 10/31/10 9:54 PM, Alex Karasulu wrote:
>>
>>> On Sun, Oct 31, 2010 at 6:42 PM, Emmanuel Lecharny<elecharny@gmail.com
>>>> wrote:
>>> Hi guys,
>>>> There is a base Index class, with two main children : AvlIndex and
>>>> JdbmIndex. There is a third child, named genericIndex, which is only used
>>>> to
>>>> deal with junit annotations used to create a new server instance (it's a
>>>> holder class).
>>>>
>>> What's a holder class?
>>>
>> A POJO, if you prefer. It holds some value, but does not have any logic
>> inside. A Bean. A container.
>>
>> As I'm going a but further in the code analysis, I see that Index are
>> handled in two steps in the server :
>> - first we create them
>> - then we initialize them.
>>
>> When adding a partition with indices, the Index will have a different type
>> depending on the Partition's store type. We can create a JdbmIndex, an
>> AvlIndex, etc. That means we need to have an object containing the Index
>> description which will be used to instantiate the correct index (and this is
>> why we have this GenericIndex, plus some convertAndInit() method in the
>> different kinds of stores).
>>
>> This does not seems to be the correct approach to me. What I would prefer
>> do is to use either the IndexBean (which has all the needed parameters
>> necessary to create an index), and let the Store creates the correct index
>> directly, instead of calling this convertAndInit method.
>>
>>
> There will be subtypes that will need to carry more information associated
> with the specific partition implementation index. How will you account for
> all that. You cannot now foresee all the different kinds of wacky
> implementations our users will come up with down the line.
Nor can the GenericIndex. I see no difference here.
> I think you're making a big mistake and not thinking of dynamic extension
> scenarios here where someone adds FooPartition that has FooIndex with bar
> property specific to it.
Again, using what we have does not allow such extended system to be
initialized in any better way. And, yes, I have in mind the dynamic
extension of the system (more specifically thinking about HBase index
and Oracle Index here)
> If you take the brute force hardcoded approach to
> having just an IndexBean with what you at this point see as all foreseen
> values we're going to have serious problems with extension points on
> Partition.
What brute force approach are you talking about ?
My point is that defining a class called GenericIndex inheriting the
Index interface, but having *all* the methods throwing a
UnsupportedOperationException is just bad design.
I'm also saying that having a convertAndInit is just useless : it takes
the base informations we have put into the GenericIndex object (which
does not hold anything more than what we have in the IndexBean, and in
fact, it contains less parameter), and initialize the index the way it
should be considering the Index type. How is it different with calling
the Index constructor passing the IndexBean containing the configurtion
for this index, and then the init() method ?
In any case, if someone want to define a different kind of Partition,
which a specific kind of Index, having some specific configuration which
are not supported currently, then it's not an issue :
- the implementer will have to define a new OC describing the parameters
as AT.
- he will also have to extend the IndexBean class to fit this new
configurtion
- the configReader will feed this newly created configuration class
based on the newly defined OC
- then this bean will be passed to his new Store which will be able to
initialize the index as needed.
Base line : the core server code won't have to be changed, as all is
already written to support such extensions.
If we don't do that, then we are forced to modify the GenericIndex to
accept the new parameters, which in a way is a brute force approach.
Plus it's a duplicated effort, because you still have to write your own
version of the Bean.
There is only one remaining issue atm : the @ used to write unit tests
have to benefit from this approach (should not be an issue, as soon as
the @ tells which is the index type).
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Moving the configuration beans to core-api
Posted by Alex Karasulu <ak...@apache.org>.
On Sun, Oct 31, 2010 at 11:11 PM, Emmanuel Lécharny <el...@apache.org>wrote:
> On 10/31/10 9:54 PM, Alex Karasulu wrote:
>
>> On Sun, Oct 31, 2010 at 6:42 PM, Emmanuel Lecharny<elecharny@gmail.com
>> >wrote:
>>
>> Hi guys,
>>>
>>> There is a base Index class, with two main children : AvlIndex and
>>> JdbmIndex. There is a third child, named genericIndex, which is only used
>>> to
>>> deal with junit annotations used to create a new server instance (it's a
>>> holder class).
>>>
>>
>> What's a holder class?
>>
> A POJO, if you prefer. It holds some value, but does not have any logic
> inside. A Bean. A container.
>
> As I'm going a but further in the code analysis, I see that Index are
> handled in two steps in the server :
> - first we create them
> - then we initialize them.
>
> When adding a partition with indices, the Index will have a different type
> depending on the Partition's store type. We can create a JdbmIndex, an
> AvlIndex, etc. That means we need to have an object containing the Index
> description which will be used to instantiate the correct index (and this is
> why we have this GenericIndex, plus some convertAndInit() method in the
> different kinds of stores).
>
> This does not seems to be the correct approach to me. What I would prefer
> do is to use either the IndexBean (which has all the needed parameters
> necessary to create an index), and let the Store creates the correct index
> directly, instead of calling this convertAndInit method.
>
>
There will be subtypes that will need to carry more information associated
with the specific partition implementation index. How will you account for
all that. You cannot now foresee all the different kinds of wacky
implementations our users will come up with down the line.
I think you're making a big mistake and not thinking of dynamic extension
scenarios here where someone adds FooPartition that has FooIndex with bar
property specific to it. If you take the brute force hardcoded approach to
having just an IndexBean with what you at this point see as all foreseen
values we're going to have serious problems with extension points on
Partition.
--
Alex Karasulu
My Blog :: http://www.jroller.com/akarasulu/
Apache Directory Server :: http://directory.apache.org
Apache MINA :: http://mina.apache.org
To set up a meeting with me: http://tungle.me/AlexKarasulu
Re: Moving the configuration beans to core-api
Posted by Emmanuel Lécharny <el...@apache.org>.
On 10/31/10 9:54 PM, Alex Karasulu wrote:
> On Sun, Oct 31, 2010 at 6:42 PM, Emmanuel Lecharny<el...@gmail.com>wrote:
>
>> Hi guys,
>>
>> There is a base Index class, with two main children : AvlIndex and
>> JdbmIndex. There is a third child, named genericIndex, which is only used to
>> deal with junit annotations used to create a new server instance (it's a
>> holder class).
>
> What's a holder class?
A POJO, if you prefer. It holds some value, but does not have any logic
inside. A Bean. A container.
As I'm going a but further in the code analysis, I see that Index are
handled in two steps in the server :
- first we create them
- then we initialize them.
When adding a partition with indices, the Index will have a different
type depending on the Partition's store type. We can create a JdbmIndex,
an AvlIndex, etc. That means we need to have an object containing the
Index description which will be used to instantiate the correct index
(and this is why we have this GenericIndex, plus some convertAndInit()
method in the different kinds of stores).
This does not seems to be the correct approach to me. What I would
prefer do is to use either the IndexBean (which has all the needed
parameters necessary to create an index), and let the Store creates the
correct index directly, instead of calling this convertAndInit method.
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Moving the configuration beans to core-api
Posted by Alex Karasulu <ak...@apache.org>.
On Sun, Oct 31, 2010 at 6:42 PM, Emmanuel Lecharny <el...@gmail.com>wrote:
> Hi guys,
>
> There is a base Index class, with two main children : AvlIndex and
> JdbmIndex. There is a third child, named genericIndex, which is only used to
> deal with junit annotations used to create a new server instance (it's a
> holder class).
What's a holder class?
--
Alex Karasulu
My Blog :: http://www.jroller.com/akarasulu/
Apache Directory Server :: http://directory.apache.org
Apache MINA :: http://mina.apache.org
To set up a meeting with me: http://tungle.me/AlexKarasulu