You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Felix Knecht <fe...@apache.org> on 2008/11/15 20:50:24 UTC

Partition configurations [was Re: [CONF] Apache Directory SandBox: Draft - ... ]

Hi all

Following the ML I recently saw multiple questions about writing a custom partition so there seems to be an interest in
this topic.

Now I'm just following Stefans (big thanks about this howto!) and hit on the needs to change the server.xml when adding
a custom partition.

I'd like to raise a discussion about this.
Is it really needed to do this?!
Isn't a new partition just a module to plug into an existing server? Wouldn't it be nice just to drop in the jar
containing the custom partition and restart the server?
No need to change any configurations?

Why not let spring detecting all the existing partition modules for us?

It could be nicely done as all partitions implements the same interface. So get all beans implementing this interaface
and load them as partition. This way any partition module (=jar) contains its own definition. Adding the jar to the ds
servers lib directory and restarting the server we be enough to get a new partition.

See also springs ListableBeanFactory [1] or [2]

[1] http://static.springframework.org/spring/docs/2.5.x/api/org/springframework/beans/factory/ListableBeanFactory.html
[2] http://forum.springframework.org/showthread.php?t=48066

Switching to such a module plug in way for partitions would it also make easy to create a maven archetype for creating a
custom partition.

WDOT?

Regards
Felix


confluence@apache.org schrieb:
> 
> Page Edited : DIRxSBOX
> <http://cwiki.apache.org/confluence/display/DIRxSBOX> : Draft - How to
> write a simple custom partition for ApacheDS
> <http://cwiki.apache.org/confluence/display/DIRxSBOX/Draft+-+How+to+write+a+simple+custom+partition+for+ApacheDS>

<snip/>
>         Adding it to a server.xml file
> 
> In order to use the partition in a standard installation of ApacheDS, we
> simply add it to the /server.xml/ configuration. We have not used
> annotations (xbean) to ease XML editing, so we have to provide a
> "native" Spring bean.
> 
> *server.xml*
> 
> <spring:beans xmlns:spring="http://xbean.apache.org/schemas/spring/1.0" 
> 			  xmlns:s="http://www.springframework.org/schema/beans"
> 			  xmlns="http://apacheds.org/config/1.0">
> 
>   ...
>    <defaultDirectoryService ...>
>      ...
>      <partitions>
>      ...
>       <s:bean 
>           id="helloPartition" 
>           class="org.apache.directory.samples.partition.hello.HelloWorldPartition">
>         <s:property name="suffix" value="ou=helloWorld" />
>       </s:bean>
>     </partitions>
>    ...
>   </defaultDirectoryService>
> ...
> 


Re: Partition configurations [was Re: [CONF] Apache Directory SandBox: Draft - ... ]

Posted by Felix Knecht <fe...@apache.org>.
Kiran Ayyagari schrieb:
> hi Felix,
> 
>    I have a question(am not a spring expert)

And I'm not a ApacheDS expert at all :-)

 if the partition bean gets
> loaded
>    dynamically then where can we change its configuration e.x adding a new
>    index to the partition.

This would then probably be in the bean configuration within the partition jar - you would need to redeploy the jar with
the new bean configuration containing the index now and then restart the ds server. Probably instead of changing the
server.xml and restart the ds server (or does it reloads the configuration automagically when it has changed - I don't
know).


> 
>    P.S :- I liked the way Stefan configured the partition for this HOW-TO,
>           IMO using XBean will add more complexity and confusion for
>           configuring the partition than implementing it :).
> 
> Kiran Ayyagari
> 
> Felix Knecht wrote:
>> Hi all
>>
>> Following the ML I recently saw multiple questions about writing a
>> custom partition so there seems to be an interest in
>> this topic.
>>
>> Now I'm just following Stefans (big thanks about this howto!) and hit
>> on the needs to change the server.xml when adding
>> a custom partition.
>>
>> I'd like to raise a discussion about this.
>> Is it really needed to do this?!
>> Isn't a new partition just a module to plug into an existing server?
>> Wouldn't it be nice just to drop in the jar
>> containing the custom partition and restart the server?
>> No need to change any configurations?
>>
>> Why not let spring detecting all the existing partition modules for us?
>>
>> It could be nicely done as all partitions implements the same
>> interface. So get all beans implementing this interaface
>> and load them as partition. This way any partition module (=jar)
>> contains its own definition. Adding the jar to the ds
>> servers lib directory and restarting the server we be enough to get a
>> new partition.
>>
>> See also springs ListableBeanFactory [1] or [2]
>>
>> [1]
>> http://static.springframework.org/spring/docs/2.5.x/api/org/springframework/beans/factory/ListableBeanFactory.html
>>
>> [2] http://forum.springframework.org/showthread.php?t=48066
>>
>> Switching to such a module plug in way for partitions would it also
>> make easy to create a maven archetype for creating a
>> custom partition.
>>
>> WDOT?
>>
>> Regards
>> Felix
>>
>>
>> confluence@apache.org schrieb:
>>> Page Edited : DIRxSBOX
>>> <http://cwiki.apache.org/confluence/display/DIRxSBOX> : Draft - How to
>>> write a simple custom partition for ApacheDS
>>> <http://cwiki.apache.org/confluence/display/DIRxSBOX/Draft+-+How+to+write+a+simple+custom+partition+for+ApacheDS>
>>>
>>
>> <snip/>
>>>         Adding it to a server.xml file
>>>
>>> In order to use the partition in a standard installation of ApacheDS, we
>>> simply add it to the /server.xml/ configuration. We have not used
>>> annotations (xbean) to ease XML editing, so we have to provide a
>>> "native" Spring bean.
>>>
>>> *server.xml*
>>>
>>> <spring:beans
>>> xmlns:spring="http://xbean.apache.org/schemas/spring/1.0"
>>>               xmlns:s="http://www.springframework.org/schema/beans"
>>>               xmlns="http://apacheds.org/config/1.0">
>>>
>>>   ...
>>>    <defaultDirectoryService ...>
>>>      ...
>>>      <partitions>
>>>      ...
>>>       <s:bean           id="helloPartition"          
>>> class="org.apache.directory.samples.partition.hello.HelloWorldPartition">
>>>
>>>         <s:property name="suffix" value="ou=helloWorld" />
>>>       </s:bean>
>>>     </partitions>
>>>    ...
>>>   </defaultDirectoryService>
>>> ...
>>>
>>
>>
> 


Re: Partition configurations [was Re: [CONF] Apache Directory SandBox: Draft - ... ]

Posted by Kiran Ayyagari <ay...@gmail.com>.
hi Felix,

    I have a question(am not a spring expert) if the partition bean gets loaded
    dynamically then where can we change its configuration e.x adding a new
    index to the partition.

    P.S :- I liked the way Stefan configured the partition for this HOW-TO,
           IMO using XBean will add more complexity and confusion for
           configuring the partition than implementing it :).

Kiran Ayyagari

Felix Knecht wrote:
> Hi all
> 
> Following the ML I recently saw multiple questions about writing a custom partition so there seems to be an interest in
> this topic.
> 
> Now I'm just following Stefans (big thanks about this howto!) and hit on the needs to change the server.xml when adding
> a custom partition.
> 
> I'd like to raise a discussion about this.
> Is it really needed to do this?!
> Isn't a new partition just a module to plug into an existing server? Wouldn't it be nice just to drop in the jar
> containing the custom partition and restart the server?
> No need to change any configurations?
> 
> Why not let spring detecting all the existing partition modules for us?
> 
> It could be nicely done as all partitions implements the same interface. So get all beans implementing this interaface
> and load them as partition. This way any partition module (=jar) contains its own definition. Adding the jar to the ds
> servers lib directory and restarting the server we be enough to get a new partition.
> 
> See also springs ListableBeanFactory [1] or [2]
> 
> [1] http://static.springframework.org/spring/docs/2.5.x/api/org/springframework/beans/factory/ListableBeanFactory.html
> [2] http://forum.springframework.org/showthread.php?t=48066
> 
> Switching to such a module plug in way for partitions would it also make easy to create a maven archetype for creating a
> custom partition.
> 
> WDOT?
> 
> Regards
> Felix
> 
> 
> confluence@apache.org schrieb:
>> Page Edited : DIRxSBOX
>> <http://cwiki.apache.org/confluence/display/DIRxSBOX> : Draft - How to
>> write a simple custom partition for ApacheDS
>> <http://cwiki.apache.org/confluence/display/DIRxSBOX/Draft+-+How+to+write+a+simple+custom+partition+for+ApacheDS>
> 
> <snip/>
>>         Adding it to a server.xml file
>>
>> In order to use the partition in a standard installation of ApacheDS, we
>> simply add it to the /server.xml/ configuration. We have not used
>> annotations (xbean) to ease XML editing, so we have to provide a
>> "native" Spring bean.
>>
>> *server.xml*
>>
>> <spring:beans xmlns:spring="http://xbean.apache.org/schemas/spring/1.0" 
>> 			  xmlns:s="http://www.springframework.org/schema/beans"
>> 			  xmlns="http://apacheds.org/config/1.0">
>>
>>   ...
>>    <defaultDirectoryService ...>
>>      ...
>>      <partitions>
>>      ...
>>       <s:bean 
>>           id="helloPartition" 
>>           class="org.apache.directory.samples.partition.hello.HelloWorldPartition">
>>         <s:property name="suffix" value="ou=helloWorld" />
>>       </s:bean>
>>     </partitions>
>>    ...
>>   </defaultDirectoryService>
>> ...
>>
> 
> 

Re: Partition configurations [was Re: [CONF] Apache Directory SandBox: Draft - ... ]

Posted by Stefan Zoerner <st...@labeo.de>.
Emmanuel Lecharny wrote:
> I'm afraid that I introduced some confusion with my previous mail. I was 
> talking about the partition from the user point of view, not from the 
> developper point of view. In this case, yes, you have to implement the 
> Partition interface.
> 
> What users want, mainly, is to define their own partition like, say, 
> "dc=my-company,dc=com", and inject data into it, not defining a new 
> partition with a SGBD storage (I guess).
> 
> I agree this is confusing, as we are mixing the partition we have in the 
> conf and the Partition interface, with all the magic needed to store 
> data and search into it.
> 
> Am I right about tis possible confusion ?

Yes, we talked about the way an advanced developer would implement his 
own backend storage, not configuration of a JDBM partition in server.xml

There are several mails on the lists about this topic. So I started to 
write something to have a starting point. In it's disclaimer, I have 
written the following:

--8<--
If you simply plan to add another suffix to ApacheDS (besides 
dc=example,dc=com, for instance) in order to store data, it is not 
necessary to write any code. You can simply add some lines to the 
configuration. The following is for developers who plan to implement 
another storage mechanism than the provided default.
-->8--

The discussion about configuration in general, although interesting, is 
a another thing.

Greetings, Stefan


Re: Partition configurations [was Re: [CONF] Apache Directory SandBox: Draft - ... ]

Posted by Emmanuel Lecharny <el...@gmail.com>.
Stefan Zoerner wrote:
> <snip/>
> As far as I understand it, the alias in the configuration file is not 
> necessaryeven a configuration file is not necessary (if you run 
> ApacheDS in embedded mode).
True, as we will then bootstrap the server from hardwired values.
>
> The entry representing the name/suffix has not to be injected into the 
> server, if the underlying partition implementation already has this 
> entry.
You inject this entry only once, when creating the partition (and even 
then, it's not necessary, but you won't be able to read the partition 
itself)
>
> Emmanuel, would you please take a look at my minimal partition 
> implementation,
>
> http://cwiki.apache.org/confluence/display/DIRxSBOX/Draft+-+How+to+write+a+simple+custom+partition+for+ApacheDS 
>
>
> which simply carries one entry
>
> dn: ou=helloWorld
> objectClass: top
> objectClass: organizationalUnit
> description: hello, world
> description: a minimal partition
> ou: helloWorld
>
> It has not been injected, because it is the only one it has, and it is 
> created during the init-method.
>
> Any feedback would be helpful. Is this a valid partition (and yes, I 
> have created something completely useless from a storage point of view 
> -- it is just a starting point for all the guys trying to write their 
> own partition).
>
> I already have some questions. But I plan to stabilize my minimal 
> example first in order to have something for the docs. In have a 
> second example, which publishes the System.properties and their values 
> as a tree in order to show more methods from the partition interface.
I'm afraid that I introduced some confusion with my previous mail. I was 
talking about the partition from the user point of view, not from the 
developper point of view. In this case, yes, you have to implement the 
Partition interface.

What users want, mainly, is to define their own partition like, say, 
"dc=my-company,dc=com", and inject data into it, not defining a new 
partition with a SGBD storage (I guess).

I agree this is confusing, as we are mixing the partition we have in the 
conf and the Partition interface, with all the magic needed to store 
data and search into it.

Am I right about tis possible confusion ?


-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org



Re: Partition configurations [was Re: [CONF] Apache Directory SandBox: Draft - ... ]

Posted by Emmanuel Lecharny <el...@gmail.com>.
David Jencks wrote:

Hi David,

I will try to coo my yesturday temper : I was a bit tired... :)
>
> On Nov 16, 2008, at 5:14 PM, Emmanuel Lecharny wrote:
>
>> David Jencks wrote:
>>>
>>> I can't say I regard xbean-spring as an ideal solution, but I don't 
>>> really understand why people think its any harder to use than plain 
>>> spring.
>> Because it is !!! Instead of having a single file where you have all 
>> you need to get connected to the source, you have to permanently go 
>> back and forth from conf to source, guessing what could be the class 
>> associated with a name in the conf file, and what can be the 
>> attributes you can configure.
>
> I understand that figuring out what class relates to what element is 
> more difficult at the moment with xbean-spring.
It's always a balance : either you have a lightweight XML configuration 
file, and I must admit that Spring carries a lot of dead weight with all 
those packages before the class name, so having a simple name instead is 
an improvement, or you have a heavy config file, but it's easier to 
understand what is initialized when you start the server, as you have 
all the information needed.

The problem is that ADS is about 2500 classes, and it's pretty hard to 
know where you can find a class by its name (grep is not an option ;)


> IIRC the default is to use the java package in the namespace and the 
> class name as the element name.  I thought lots of namespaces would be 
> harder to use than a single namespace, maybe this was not a good 
> decision.  Quite possibly it would be better to have a few packages 
> with the configurable objects and namespaces for each.
>
> I'm confused about your claim that it's harder to figure out which 
> attributes you can configure with xbean-spring than with plain spring.  
Let me explain : as we have a pretty deep inheritance tree, you have to 
check for all the setXXX() in all the parent's classes, which is a bit 
painfull, IMHO. What would be _way_ better is to describe the attributes 
with annotation, instead of letting xbean extrapolate them using 
introspection. At least, if you don't have such an annotation present, 
then it will default to introspection, but otherwise, if a 
@xbean.arguments( "attr1, attr2, ..." ) is present (or whatever other 
solution), it will help a lot the developpers.
>  Without looking at the xsd or the class I don't see you have any way 
> of telling in either case.  If you look at the class its pretty 
> obvious in either case.  With xbean spring you also get a schema that 
> most editors will use to provide context-sensitive suggestions about 
> what elements or attributes are allowed: this is not possible with 
> plain spring AFAIK.  If you can explain further I'd like to understand 
> what the problem is.
I agree that plain Spring does not help you to do that. I hope that I 
explained what I had in mind in my previous comment.
> <snip/>
>> well, I don't have any favorite XSD editor, and I don't have time to 
>> absorb the messy XSD syntax. That's one of the reason I find it 
>> overkilling.
>
> Xsds are a pretty horrible language, but I admit I don't find them 
> that hard to understand.  
I have spent a hell of time on compilers, trully, BNF is a beauty. XSD, 
not so much (this is a direct reference to Borat ;)

To recall a reference to some comment you can find in Gnu preprocessor :

  /* Pre-C-Preprocessor to translate ANSI trigraph idiocy in BUF
     before main CCCP processing.  Name `pcp' is also in honor of the
     drugs the trigraph designers must have been on.

XSD must be one of the latest cool drug the guys who invented XSD must 
have been on when they drafted it... I must admit that DTD was much 
worst, but at least, it was so disgusting that you had no chance to 
become an addict :)

Seriously, XSD is crap... Even if it's not _that_ complex :)
> But with most editors (idea and I'm sure eclipse) 
I didn't found a free and easy to use XSD editor on eclipse so far... 
And trust me, I tried a bunch of those crappy SF drops...
> if you tell the editor about the schema it will provide validation as 
> you type and context sensitive hints about what is allowed.  I find 
> this a lot easier to use than having to consult the class (or the 
> schema) to find out what is allowed.
Well, if you can read an XSD file, sure. IMHO, XSD is only valid when 
you want to validate a XML file, and only when it's handled by a 
computer, not by human being. Eh, I was used to write code using direct 
hexadecimal on my first Z80 or 6502 computers, but I don't think it fits 
well in term of productivity :) XSD is a bit like assembly to me...
>
>>
>>
>> May be it's just me, but from discussions I had with other projects 
>> as well, XBeans may be a cool techno, but certainly not something 
>> which makes developpers life easier. IMHO.
>
> I hope I'm not being too much of a nuisance here.... 
Nah, don't worry ! I'm just whining ;)
> I don't think xbean is the last word in configuration, but I am 
> leaning towards jaxb based xml <--> configuration object solutions 
> which have the same basic features as xbean spring so I'd really like 
> to understand if there are fundamental problems with this kind of 
> approach of having an xml domain specific language for component 
> configuration.
I understand you approach, David. It makes a hell of sense, as 
manipulating XML files to make them closer to the code is now necessary 
because of IoC. And I also agree that those damn <package 
name>+.<className> are just a PITA.

Here is what I think : Spring, and all those IoC/DI stuff are just 
totally over rated. People tend to think that by defining their system 
using beans is the ultimate way to go, as we weren't able to send a man 
to the moon without all those craps... This is just a nonsense. Either 
you have a plugable architecture (ie, JDBC drivers), or a Container 
based system, or you don't. Usually, it's a mix. When it comes to ADS, 
we have many plugable modules, like authentication systems,  
SyntaxCheckers, etc, but I find this IoC thing far too complicated for 
what we need. Either from the user POV (because it forces the user to 
manipulate a pretty verbose file), but also from the developper POV, as 
you simply can't anymore step into the initialization phase, unless you 
put a number of BP into your code.

IMHO, Spring is one of the worst system ever, because not only it's 
useless in 99,9% of the time, but also because it spreads myths like "it 
makes your life easy". I may be wrong, of course, but I'm following the 
Spring buzz since 2001 (or 2002, I don't remember), and I can't stop 
thinking that it will die sooner or later. The simple fact that you need 
to design XBean to ease the pain Spring is introducing is just another 
nail in the coffin...

Thanks David !

-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org



Re: Partition configurations [was Re: [CONF] Apache Directory SandBox: Draft - ... ]

Posted by David Jencks <da...@yahoo.com>.
On Nov 16, 2008, at 5:14 PM, Emmanuel Lecharny wrote:

> David Jencks wrote:
>>
>> I can't say I regard xbean-spring as an ideal solution, but I don't  
>> really understand why people think its any harder to use than plain  
>> spring.
> Because it is !!! Instead of having a single file where you have all  
> you need to get connected to the source, you have to permanently go  
> back and forth from conf to source, guessing what could be the class  
> associated with a name in the conf file, and what can be the  
> attributes you can configure.

I understand that figuring out what class relates to what element is  
more difficult at the moment with xbean-spring.  IIRC the default is  
to use the java package in the namespace and the class name as the  
element name.  I thought lots of namespaces would be harder to use  
than a single namespace, maybe this was not a good decision.  Quite  
possibly it would be better to have a few packages with the  
configurable objects and namespaces for each.

I'm confused about your claim that it's harder to figure out which  
attributes you can configure with xbean-spring than with plain  
spring.  Without looking at the xsd or the class I don't see you have  
any way of telling in either case.  If you look at the class its  
pretty obvious in either case.  With xbean spring you also get a  
schema that most editors will use to provide context-sensitive  
suggestions about what elements or attributes are allowed: this is not  
possible with plain spring AFAIK.  If you can explain further I'd like  
to understand what the problem is.

>
>> Unless you are changing the tree structure of the configuration  
>> javabeans you don't even need to change the "annotations" to get  
>> the updated schema, just build....
> We don't change the annotation. Never. However, that does not make  
> things easier.
>> at which point you can use your favorite xsd-aware editor to check  
>> on the validity of your xml configuration before you start up the  
>> server and discover you misspelled an attribute name.  What am I  
>> missing?
> well, I don't have any favorite XSD editor, and I don't have time to  
> absorb the messy XSD syntax. That's one of the reason I find it  
> overkilling.

Xsds are a pretty horrible language, but I admit I don't find them  
that hard to understand.  But with most editors (idea and I'm sure  
eclipse) if you tell the editor about the schema it will provide  
validation as you type and context sensitive hints about what is  
allowed.  I find this a lot easier to use than having to consult the  
class (or the schema) to find out what is allowed.

>
>
> May be it's just me, but from discussions I had with other projects  
> as well, XBeans may be a cool techno, but certainly not something  
> which makes developpers life easier. IMHO.

I hope I'm not being too much of a nuisance here.... I don't think  
xbean is the last word in configuration, but I am leaning towards jaxb  
based xml <--> configuration object solutions which have the same  
basic features as xbean spring so I'd really like to understand if  
there are fundamental problems with this kind of approach of having an  
xml domain specific language for component configuration.

thanks
david jencks

>
>
> Sorry, David, I just dislike xbeans, it makes my life miserable when  
> it comes to manipulate ADS configuration...
>
> -- 
> --
> cordialement, regards,
> Emmanuel Lécharny
> www.iktek.com
> directory.apache.org
>
>


Re: Partition configurations [was Re: [CONF] Apache Directory SandBox: Draft - ... ]

Posted by Emmanuel Lecharny <el...@gmail.com>.
David Jencks wrote:
>
> I can't say I regard xbean-spring as an ideal solution, but I don't 
> really understand why people think its any harder to use than plain 
> spring.  
Because it is !!! Instead of having a single file where you have all you 
need to get connected to the source, you have to permanently go back and 
forth from conf to source, guessing what could be the class associated 
with a name in the conf file, and what can be the attributes you can 
configure.
> Unless you are changing the tree structure of the configuration 
> javabeans you don't even need to change the "annotations" to get the 
> updated schema, just build.... 
We don't change the annotation. Never. However, that does not make 
things easier.
> at which point you can use your favorite xsd-aware editor to check on 
> the validity of your xml configuration before you start up the server 
> and discover you misspelled an attribute name.  What am I missing?
well, I don't have any favorite XSD editor, and I don't have time to 
absorb the messy XSD syntax. That's one of the reason I find it 
overkilling.

May be it's just me, but from discussions I had with other projects as 
well, XBeans may be a cool techno, but certainly not something which 
makes developpers life easier. IMHO.

Sorry, David, I just dislike xbeans, it makes my life miserable when it 
comes to manipulate ADS configuration...

-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org



Re: Partition configurations [was Re: [CONF] Apache Directory SandBox: Draft - ... ]

Posted by David Jencks <da...@yahoo.com>.
On Nov 16, 2008, at 4:06 PM, Norval Hope wrote:

> Hi All,
>
> Just wanted to throw in some comments/observations from the
> prespective of a Virtual Directory implementation which sits on top of
> AD. In this kind of use-case it is completely necessary to discover
> connector implementations (aka custom partitions) dynamically at
> start-up (like a poor-man's OSGi implementation which requires a
> restart - as I too have had proper OSGi connector bundles on my list
> of TODOs for ages). For this purpose I have used Spring's
> FileSystemXmlApplicationContext's ability to discover files in jars on
> the the classpath, using a command line arguments like:
> "server_jcs.xml "classpath*:conf/connector.xml" (where each
> connector's .jar file has a specialized conf/connector.xml in it which
> configures only settings for the connector itself). Hence this
> approach is definitely possible, and desireable for cases like mine
> where connectors come and go regularly.
>
> Secondly I get extremely nervous when I hear "use the DIT and ditch
> Spring" as has been mooted a number of times on the ML before, unless
> the DIT is merely a used as a storage place for XML orginally taken
> from "live" connector.xml files. This is because the content of my
> connector.xml files are custom POJOs, many of which are even specific
> to a single connector implementation, and hence I have no interest in
> reworking all this config into a heavy-weight custom LDAP schema when
> Spring is doing exactly the right job for me currently. When it comes
> to custom partition implementations I don't think my use-case is
> atypical, i.e. I would expect it to be the norm that they will
> generally require custom configuration too, and therefore think the
> combination of Spring and custom POJOs is a perfect match. As I
> mentioned earlier, having an option where the config stays as XML but
> can optionally appear as an attribute value in the DIT (presumably for
> the parition itself) may allow the best of both worlds, keeping easy
> Spring configuration but still allowing replication etc. For the same
> reason (and the fact I have to remain backward compatible) I'm also
> not keen on the extra work required to tidy up the config files using
> XBeans, while the Spring syntax may not be as compact it is easy
> enough to understand (and comes with a 0% maintenance load - this
> additional load seems to me to be the price you pay for the more
> compact XBeans synatx).
>
> So in summary - in my world-view fluid configuration is a must. I want
> to be able to change just configuration JavaBeans and a few associated
> settings in XML. I don't want to have additionally update a
> heavy-weight custom configuration schema and/or XBeans wrappers.

I can't say I regard xbean-spring as an ideal solution, but I don't  
really understand why people think its any harder to use than plain  
spring.  Unless you are changing the tree structure of the  
configuration javabeans you don't even need to change the  
"annotations" to get the updated schema, just build.... at which point  
you can use your favorite xsd-aware editor to check on the validity of  
your xml configuration before you start up the server and discover you  
misspelled an attribute name.  What am I missing?

But, if you don't want to use it for your extensions, you can  
certainly mix normal spring and xbean-spring xml in the same xml  
configuration file.

thanks
david jencks


>
>
> Cheers
>
>
> On Mon, Nov 17, 2008 at 4:29 AM, Stefan Zoerner <st...@labeo.de>  
> wrote:
>> Hi Emmanuel,
>>
>> Thanks for snapping in the discussion! I will add some of your  
>> valuable
>> information to the documentation.
>>
>> Emmanuel Lecharny wrote:
>>>
>>> I will try to add some clarification about those partitions,  
>>> because it's
>>> not really something trivial.
>>>
>>> First, let's assume we are talking about 1.5 here.
>>
>> Yes we do.
>>
>>> A partition is nothing more than, to take an analogy, a mounting  
>>> point on
>>> a Unix filesystem. It has a root name (a DN in our case, and the  
>>> underlying
>>> storage can be whatever we want, byt from now on, a JDBM based  
>>> storage (we
>>> can perfectly set a partition to use BDB JE, as Kiran implemented  
>>> it, or
>>> Oracle, or MySql, as some of our users did).
>>>
>>> The main difference is that a partition _must_ be associated with an
>>> entry. It's npt different from what you have on OpenLdap anyway.
>>>
>>> So let's sum up. You need :
>>> - a name ("dc=example,dc=com", for instance)
>>> - an alias in the configuration file ("example", here)
>>> - an underlying storage ( <jdbmPartition id="example"...)
>>> - and an entry which has to be injected in the server :
>>> dn: dc=example,dc=com
>>> ObjectClass: top
>>> ObjectClass: domain
>>> dc: example
>>>
>>> and that's pretty much it.
>>
>> As far as I understand it, the alias in the configuration file is not
>> necessaryeven a configuration file is not necessary (if you run  
>> ApacheDS in
>> embedded mode).
>>
>> The entry representing the name/suffix has not to be injected into  
>> the
>> server, if the underlying partition implementation already has this  
>> entry.
>>
>> Emmanuel, would you please take a look at my minimal partition
>> implementation,
>>
>> http://cwiki.apache.org/confluence/display/DIRxSBOX/Draft+-+How+to+write+a+simple+custom+partition+for+ApacheDS
>>
>> which simply carries one entry
>>
>> dn: ou=helloWorld
>> objectClass: top
>> objectClass: organizationalUnit
>> description: hello, world
>> description: a minimal partition
>> ou: helloWorld
>>
>> It has not been injected, because it is the only one it has, and it  
>> is
>> created during the init-method.
>>
>> Any feedback would be helpful. Is this a valid partition (and yes,  
>> I have
>> created something completely useless from a storage point of view  
>> -- it is
>> just a starting point for all the guys trying to write their own  
>> partition).
>>
>> I already have some questions. But I plan to stabilize my minimal  
>> example
>> first in order to have something for the docs. In have a second  
>> example,
>> which publishes the System.properties and their values as a tree in  
>> order to
>> show more methods from the partition interface.
>>
>> Greetings from Hamburg,
>>   Stefan
>>
>>
>>


Re: Partition configurations [was Re: [CONF] Apache Directory SandBox: Draft - ... ]

Posted by Norval Hope <nr...@gmail.com>.
Hi Emmanuel, comments inline ...

On Mon, Nov 17, 2008 at 11:40 AM, Emmanuel Lecharny <el...@gmail.com> wrote:
>> Just wanted to throw in some comments/observations from the
>> prespective of a Virtual Directory implementation which sits on top of
>> AD. In this kind of use-case it is completely necessary to discover
>> connector implementations (aka custom partitions) dynamically at
>> start-up (like a poor-man's OSGi implementation which requires a
>> restart - as I too have had proper OSGi connector bundles on my list
>> of TODOs for ages). For this purpose I have used Spring's
>> FileSystemXmlApplicationContext's ability to discover files in jars on
>> the the classpath, using a command line arguments like:
>> "server_jcs.xml "classpath*:conf/connector.xml" (where each
>> connector's .jar file has a specialized conf/connector.xml in it which
>> configures only settings for the connector itself). Hence this
>> approach is definitely possible, and desireable for cases like mine
>> where connectors come and go regularly.
>>
>
> Not sure I get a clear picture, but if it's just about separating
> partition's configuration in many different files to avoid having a big fat
> config file, then I agree : it's necessary. Tell me if I understood
> correctly.

Yes, it is about separating partition's config files out from
server.xml, but also bundling inside the jar for each connector (aka
custom partition). Every connector's .jar includes a
"conf/connector.xml" and Spring's FileSystemXmlApplicationContext
magically loads the config for all the custom jars found on the
classpath via the "classpath*:conf/connector.xml" syntax. Hence
removing support for a custom connector requires exactly a) removeing
it's .jar and b) restarting AD.

>>
>> Secondly I get extremely nervous when I hear "use the DIT and ditch
>> Spring" as has been mooted a number of times on the ML before, unless
>> the DIT is merely a used as a storage place for XML orginally taken
>> from "live" connector.xml files.
>
> Let's go back a bit backward. The very first version of ADS was using a
> typical java property file : key=value, that's it. Worked well, simple, and
> pretty verbose. But adding comment into it was just so easy ...
>
> The someone suggested to use Spring 3 years ago, when it started to be more
> than over-hyped : it was over-used. (Ok, I'm biased). Later in the process
> of migrating to 2.0, XBeans was introduced in an attempt to "simplify" the
> configuration file.

I'm personally quite impressed with Spring and its ability to
configure any JavaBean, even custom ones used by extensions to the AD
core (or even my VD core) code. When the config settings are
completely fixed there may be no need to use Spring's IoC power, but
it brings very valuable flexibility in a plug-in type environment.

>>
>> This is because the content of my
>> connector.xml files are custom POJOs, many of which are even specific
>> to a single connector implementation, and hence I have no interest in
>> reworking all this config into a heavy-weight custom LDAP schema when
>> Spring is doing exactly the right job for me currently.
>
> Well, I have to say that as far as I sympathize with your concern, i must
> tell you that it's also a bit peripherical to the project. Not to say that
> we don't care, because we _do_, but because no decision has been made atm,
> and I don't think that we will make a choice in the next few months, neither
> in the next 6 months. And I'm pretty positive when I say it won't change
> before 2.0.
>
> In fact, things are pretty clear : we don't have time for ditching this
> server.xml file now.
>

I understand my usecase maybe seem a little left-field for AD at the
moment, but I think flexible configuration is always likely to be
useful when writing custom partition implementations. It may be
preferrable to split config into some standard settings (like the
injected DN etc) and a separate more flexible text fragment found
elsewhere, which I have no problem with as long as the latter fragment
can be supported in some way.

At any rate, I'll have to maintain my current config approach no
matter what direction the AD core heads off in. As such, I'm not
trying to force AD in one direction just to suit my purposes. My
config / connector loading functionality is currently a custom
extension I've made to AD and I can maintain it as such. Just wanted
to chime in with another usecase for consideration...

> However, many of us are considering that the current configuration is too
> complex, and has many drawbacks. Sadly enough, there is no magic bullet.
> What some of us like with the 'config in DIT' things is that we will have a
> coherent system (LDAP from floor to the roof) . But this is obviously not
> that simple...
>
> <my own opinion>
> Let's be clear : when debugging the server, Spring is a PITA. You have to
> *know* what is initialized by looking at the server.xml file to be able to
> follow the initialization path. When something goes wrong - and trust me, it
> happens all the time - you are FU big time. I spent more than one hour last
> friday with a server blowing chunks just because some part of the server.xml
> was doomed, due to some classpath issue.
>
> When you combine it with xbeans, it's even worse. You have no f*ckinck
> direct clue about which class is associated with an alias, as all of that is
> hidden in a 1.0 file (yes, the file's name is 1.0 !!!), deeply buried into a
> directory somwhere (just dig...). And as it's absed on annotations, you also
> have to drill the inheritence to actually _know_ which values are
> initialized. Of course, if you are in love with XSD, you can have fun with a
> XSD file... I'm not _that_ deviant ;)
>
> So bottom line : I deeply regret the old property file... Conf in DIT is to
> me a way to come back to something I can handle : text file without one or
> two level of indirections (assuming that the conf is stored into a text
> based partition, be it a LDIF file or whatever). FYI, OpenLdap configuration
> can now mainly be stored this way.
>
> But this may be just me ...
> </my own opinion>

I understand what you mean, I have had some problems debugging Spring
too. Also XBeans aims to solve a problem (concise more readable XML
representation) which I'm not much interested in - we will probably
end up with 30 or so connector implementations eventually and there is
no way I want to maintain either a) 30 custom config schemas or b) 30
XBean views of the config requirements for each.

>>
>> When it comes
>> to custom partition implementations I don't think my use-case is
>> atypical, i.e. I would expect it to be the norm that they will
>> generally require custom configuration too, and therefore think the
>> combination of Spring and custom POJOs is a perfect match.
>
> Let's call a cat a cat : what we want is a plugin approach : a new custom
> partition should be a plugin, and you should be able to separate this custom
> partition configuration from the existing server configuration. Am I on the
> same page ?

You are on the same page. I go further and say a custom partition is a
single .jar file including implementation and default configuration,
although I allow the default configuration to also be overridden by an
XML file on the filesystem too. Long term I want a custom connector to
be an OSGi bundle so that they can be hot-deployed without restarting
AD too. If the configuration is read from the .jar / bundle and
pusblished into the DIT somehow, then that's fine by me.

>>
>> As I
>> mentioned earlier, having an option where the config stays as XML but
>> can optionally appear as an attribute value in the DIT (presumably for
>> the parition itself) may allow the best of both worlds, keeping easy
>> Spring configuration but still allowing replication etc. For the same
>> reason (and the fact I have to remain backward compatible) I'm also
>> not keen on the extra work required to tidy up the config files using
>> XBeans, while the Spring syntax may not be as compact it is easy
>> enough to understand (and comes with a 0% maintenance load - this
>> additional load seems to me to be the price you pay for the more
>> compact XBeans synatx).
>>
>
> I think we can reach a kind of concensus here, if we consider that it's just
> a matter of starting the server. Considering that when you use an embedded
> server, you don't have to deal with all this Spring+Xbeans crazyness, it's
> obvioulsy possible to define a somution where :
> - you can use Spring if necessary
> - OR use a simple property file if you like it
> - OR store the configuration as LDIF file
> and still having one simple system where the conf is stored into the DIT,
> once the server has been launched (ie, the conf is loaded in memory, and
> addressed through the LDAP API).

Sure - just trying to register a +1 for solutions which are still
compatible with your first option (Spring config), even if it is not
directly implemented.

>
> I see no reason why we couldn't reach this decoupling between configuration
> system and the running server. But in any case, I agree with Norval that
> XBeans introducing some more coupling, adding some annotations in the code.
>>
>> So in summary - in my world-view fluid configuration is a must. I want
>> to be able to change just configuration JavaBeans and a few associated
>> settings in XML. I don't want to have additionally update a
>> heavy-weight custom configuration schema and/or XBeans wrappers.
>>

Re: Partition configurations [was Re: [CONF] Apache Directory SandBox: Draft - ... ]

Posted by Emmanuel Lecharny <el...@gmail.com>.
Norval Hope wrote:
> Hi All,
>   
Hi Norval,

comment inline
> Just wanted to throw in some comments/observations from the
> prespective of a Virtual Directory implementation which sits on top of
> AD. In this kind of use-case it is completely necessary to discover
> connector implementations (aka custom partitions) dynamically at
> start-up (like a poor-man's OSGi implementation which requires a
> restart - as I too have had proper OSGi connector bundles on my list
> of TODOs for ages). For this purpose I have used Spring's
> FileSystemXmlApplicationContext's ability to discover files in jars on
> the the classpath, using a command line arguments like:
> "server_jcs.xml "classpath*:conf/connector.xml" (where each
> connector's .jar file has a specialized conf/connector.xml in it which
> configures only settings for the connector itself). Hence this
> approach is definitely possible, and desireable for cases like mine
> where connectors come and go regularly.
>   
Not sure I get a clear picture, but if it's just about separating 
partition's configuration in many different files to avoid having a big 
fat config file, then I agree : it's necessary. Tell me if I understood 
correctly.
> Secondly I get extremely nervous when I hear "use the DIT and ditch
> Spring" as has been mooted a number of times on the ML before, unless
> the DIT is merely a used as a storage place for XML orginally taken
> from "live" connector.xml files. 
Let's go back a bit backward. The very first version of ADS was using a 
typical java property file : key=value, that's it. Worked well, simple, 
and pretty verbose. But adding comment into it was just so easy ...

The someone suggested to use Spring 3 years ago, when it started to be 
more than over-hyped : it was over-used. (Ok, I'm biased). Later in the 
process of migrating to 2.0, XBeans was introduced in an attempt to 
"simplify" the configuration file.
> This is because the content of my
> connector.xml files are custom POJOs, many of which are even specific
> to a single connector implementation, and hence I have no interest in
> reworking all this config into a heavy-weight custom LDAP schema when
> Spring is doing exactly the right job for me currently. 
Well, I have to say that as far as I sympathize with your concern, i 
must tell you that it's also a bit peripherical to the project. Not to 
say that we don't care, because we _do_, but because no decision has 
been made atm, and I don't think that we will make a choice in the next 
few months, neither in the next 6 months. And I'm pretty positive when I 
say it won't change before 2.0.

In fact, things are pretty clear : we don't have time for ditching this 
server.xml file now.

However, many of us are considering that the current configuration is 
too complex, and has many drawbacks. Sadly enough, there is no magic 
bullet. What some of us like with the 'config in DIT' things is that we 
will have a coherent system (LDAP from floor to the roof) . But this is 
obviously not that simple...

<my own opinion>
Let's be clear : when debugging the server, Spring is a PITA. You have 
to *know* what is initialized by looking at the server.xml file to be 
able to follow the initialization path. When something goes wrong - and 
trust me, it happens all the time - you are FU big time. I spent more 
than one hour last friday with a server blowing chunks just because some 
part of the server.xml was doomed, due to some classpath issue.

When you combine it with xbeans, it's even worse. You have no f*ckinck 
direct clue about which class is associated with an alias, as all of 
that is hidden in a 1.0 file (yes, the file's name is 1.0 !!!), deeply 
buried into a directory somwhere (just dig...). And as it's absed on 
annotations, you also have to drill the inheritence to actually _know_ 
which values are initialized. Of course, if you are in love with XSD, 
you can have fun with a XSD file... I'm not _that_ deviant ;)

So bottom line : I deeply regret the old property file... Conf in DIT is 
to me a way to come back to something I can handle : text file without 
one or two level of indirections (assuming that the conf is stored into 
a text based partition, be it a LDIF file or whatever). FYI, OpenLdap 
configuration can now mainly be stored this way.

But this may be just me ...
</my own opinion>
> When it comes
> to custom partition implementations I don't think my use-case is
> atypical, i.e. I would expect it to be the norm that they will
> generally require custom configuration too, and therefore think the
> combination of Spring and custom POJOs is a perfect match. 
Let's call a cat a cat : what we want is a plugin approach : a new 
custom partition should be a plugin, and you should be able to separate 
this custom partition configuration from the existing server 
configuration. Am I on the same page ?
> As I
> mentioned earlier, having an option where the config stays as XML but
> can optionally appear as an attribute value in the DIT (presumably for
> the parition itself) may allow the best of both worlds, keeping easy
> Spring configuration but still allowing replication etc. For the same
> reason (and the fact I have to remain backward compatible) I'm also
> not keen on the extra work required to tidy up the config files using
> XBeans, while the Spring syntax may not be as compact it is easy
> enough to understand (and comes with a 0% maintenance load - this
> additional load seems to me to be the price you pay for the more
> compact XBeans synatx).
>   
I think we can reach a kind of concensus here, if we consider that it's 
just a matter of starting the server. Considering that when you use an 
embedded server, you don't have to deal with all this Spring+Xbeans 
crazyness, it's obvioulsy possible to define a somution where :
- you can use Spring if necessary
- OR use a simple property file if you like it
- OR store the configuration as LDIF file
and still having one simple system where the conf is stored into the 
DIT, once the server has been launched (ie, the conf is loaded in 
memory, and addressed through the LDAP API).

I see no reason why we couldn't reach this decoupling between 
configuration system and the running server. But in any case, I agree 
with Norval that XBeans introducing some more coupling, adding some 
annotations in the code.
> So in summary - in my world-view fluid configuration is a must. I want
> to be able to change just configuration JavaBeans and a few associated
> settings in XML. I don't want to have additionally update a
> heavy-weight custom configuration schema and/or XBeans wrappers.
>   
Same conclusion here...

thanks Norval !

-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org



Re: Partition configurations [was Re: [CONF] Apache Directory SandBox: Draft - ... ]

Posted by Norval Hope <nr...@gmail.com>.
Hi All,

Just wanted to throw in some comments/observations from the
prespective of a Virtual Directory implementation which sits on top of
AD. In this kind of use-case it is completely necessary to discover
connector implementations (aka custom partitions) dynamically at
start-up (like a poor-man's OSGi implementation which requires a
restart - as I too have had proper OSGi connector bundles on my list
of TODOs for ages). For this purpose I have used Spring's
FileSystemXmlApplicationContext's ability to discover files in jars on
the the classpath, using a command line arguments like:
"server_jcs.xml "classpath*:conf/connector.xml" (where each
connector's .jar file has a specialized conf/connector.xml in it which
configures only settings for the connector itself). Hence this
approach is definitely possible, and desireable for cases like mine
where connectors come and go regularly.

Secondly I get extremely nervous when I hear "use the DIT and ditch
Spring" as has been mooted a number of times on the ML before, unless
the DIT is merely a used as a storage place for XML orginally taken
from "live" connector.xml files. This is because the content of my
connector.xml files are custom POJOs, many of which are even specific
to a single connector implementation, and hence I have no interest in
reworking all this config into a heavy-weight custom LDAP schema when
Spring is doing exactly the right job for me currently. When it comes
to custom partition implementations I don't think my use-case is
atypical, i.e. I would expect it to be the norm that they will
generally require custom configuration too, and therefore think the
combination of Spring and custom POJOs is a perfect match. As I
mentioned earlier, having an option where the config stays as XML but
can optionally appear as an attribute value in the DIT (presumably for
the parition itself) may allow the best of both worlds, keeping easy
Spring configuration but still allowing replication etc. For the same
reason (and the fact I have to remain backward compatible) I'm also
not keen on the extra work required to tidy up the config files using
XBeans, while the Spring syntax may not be as compact it is easy
enough to understand (and comes with a 0% maintenance load - this
additional load seems to me to be the price you pay for the more
compact XBeans synatx).

So in summary - in my world-view fluid configuration is a must. I want
to be able to change just configuration JavaBeans and a few associated
settings in XML. I don't want to have additionally update a
heavy-weight custom configuration schema and/or XBeans wrappers.

Cheers


On Mon, Nov 17, 2008 at 4:29 AM, Stefan Zoerner <st...@labeo.de> wrote:
> Hi Emmanuel,
>
> Thanks for snapping in the discussion! I will add some of your valuable
> information to the documentation.
>
> Emmanuel Lecharny wrote:
>>
>> I will try to add some clarification about those partitions, because it's
>> not really something trivial.
>>
>> First, let's assume we are talking about 1.5 here.
>
> Yes we do.
>
>> A partition is nothing more than, to take an analogy, a mounting point on
>> a Unix filesystem. It has a root name (a DN in our case, and the underlying
>> storage can be whatever we want, byt from now on, a JDBM based storage (we
>> can perfectly set a partition to use BDB JE, as Kiran implemented it, or
>> Oracle, or MySql, as some of our users did).
>>
>> The main difference is that a partition _must_ be associated with an
>> entry. It's npt different from what you have on OpenLdap anyway.
>>
>> So let's sum up. You need :
>> - a name ("dc=example,dc=com", for instance)
>> - an alias in the configuration file ("example", here)
>> - an underlying storage ( <jdbmPartition id="example"...)
>> - and an entry which has to be injected in the server :
>> dn: dc=example,dc=com
>> ObjectClass: top
>> ObjectClass: domain
>> dc: example
>>
>> and that's pretty much it.
>
> As far as I understand it, the alias in the configuration file is not
> necessaryeven a configuration file is not necessary (if you run ApacheDS in
> embedded mode).
>
> The entry representing the name/suffix has not to be injected into the
> server, if the underlying partition implementation already has this entry.
>
> Emmanuel, would you please take a look at my minimal partition
> implementation,
>
> http://cwiki.apache.org/confluence/display/DIRxSBOX/Draft+-+How+to+write+a+simple+custom+partition+for+ApacheDS
>
> which simply carries one entry
>
> dn: ou=helloWorld
> objectClass: top
> objectClass: organizationalUnit
> description: hello, world
> description: a minimal partition
> ou: helloWorld
>
> It has not been injected, because it is the only one it has, and it is
> created during the init-method.
>
> Any feedback would be helpful. Is this a valid partition (and yes, I have
> created something completely useless from a storage point of view -- it is
> just a starting point for all the guys trying to write their own partition).
>
> I already have some questions. But I plan to stabilize my minimal example
> first in order to have something for the docs. In have a second example,
> which publishes the System.properties and their values as a tree in order to
> show more methods from the partition interface.
>
> Greetings from Hamburg,
>    Stefan
>
>
>

Re: Partition configurations [was Re: [CONF] Apache Directory SandBox: Draft - ... ]

Posted by Stefan Zoerner <st...@labeo.de>.
Hi Emmanuel,

Thanks for snapping in the discussion! I will add some of your valuable 
information to the documentation.

Emmanuel Lecharny wrote:
> I will try to add some clarification about those partitions, because 
> it's not really something trivial.
> 
> First, let's assume we are talking about 1.5 here.

Yes we do.

> A partition is nothing more than, to take an analogy, a mounting point 
> on a Unix filesystem. It has a root name (a DN in our case, and the 
> underlying storage can be whatever we want, byt from now on, a JDBM 
> based storage (we can perfectly set a partition to use BDB JE, as Kiran 
> implemented it, or Oracle, or MySql, as some of our users did).
> 
> The main difference is that a partition _must_ be associated with an 
> entry. It's npt different from what you have on OpenLdap anyway.
> 
> So let's sum up. You need :
> - a name ("dc=example,dc=com", for instance)
> - an alias in the configuration file ("example", here)
> - an underlying storage ( <jdbmPartition id="example"...)
> - and an entry which has to be injected in the server :
> dn: dc=example,dc=com
> ObjectClass: top
> ObjectClass: domain
> dc: example
> 
> and that's pretty much it.

As far as I understand it, the alias in the configuration file is not 
necessaryeven a configuration file is not necessary (if you run ApacheDS 
in embedded mode).

The entry representing the name/suffix has not to be injected into the 
server, if the underlying partition implementation already has this entry.

Emmanuel, would you please take a look at my minimal partition 
implementation,

http://cwiki.apache.org/confluence/display/DIRxSBOX/Draft+-+How+to+write+a+simple+custom+partition+for+ApacheDS

which simply carries one entry

dn: ou=helloWorld
objectClass: top
objectClass: organizationalUnit
description: hello, world
description: a minimal partition
ou: helloWorld

It has not been injected, because it is the only one it has, and it is 
created during the init-method.

Any feedback would be helpful. Is this a valid partition (and yes, I 
have created something completely useless from a storage point of view 
-- it is just a starting point for all the guys trying to write their 
own partition).

I already have some questions. But I plan to stabilize my minimal 
example first in order to have something for the docs. In have a second 
example, which publishes the System.properties and their values as a 
tree in order to show more methods from the partition interface.

Greetings from Hamburg,
     Stefan



Re: Partition configurations [was Re: [CONF] Apache Directory SandBox: Draft - ... ]

Posted by Emmanuel Lecharny <el...@gmail.com>.
Hi guys,

I will try to add some clarification about those partitions, because 
it's not really something trivial.

First, let's assume we are talking about 1.5 here.

A partition is nothing more than, to take an analogy, a mounting point 
on a Unix filesystem. It has a root name (a DN in our case, and the 
underlying storage can be whatever we want, byt from now on, a JDBM 
based storage (we can perfectly set a partition to use BDB JE, as Kiran 
implemented it, or Oracle, or MySql, as some of our users did).

The main difference is that a partition _must_ be associated with an 
entry. It's npt different from what you have on OpenLdap anyway.

So let's sum up. You need :
- a name ("dc=example,dc=com", for instance)
- an alias in the configuration file ("example", here)
- an underlying storage ( <jdbmPartition id="example"...)
- and an entry which has to be injected in the server :
dn: dc=example,dc=com
ObjectClass: top
ObjectClass: domain
dc: example

and that's pretty much it.

The problem is that we have change the way those elements are described 
in the server.xml file (because this file is changing all along the 
versions, until we reach a stable state : 2.0-RC1). So here are some of 
the issues we have :
- we don't anymore define the entry within the server.xml file, which is 
a bit of a burden for our users
- it's not obvious to understand that <jdbmPartion ... > is associated 
with a JDBM backend, not is it obvious what should we use for an Oracle 
database
- Not all of our users can clearly understand the server.xml content, 
because it's still a moving target (and I really can't blame the poor 
guys who try to update the documentation when we lazzy developpers don't 
do any effort to keep it stable :/ )
- Mixing xbeans with spring is certainly not helping us...

So we are in trouble here. The question is how could we aleviat our 
users pain... We mentioned many times the idea to store configuration in 
the DIT. But in order to do that, we have to have a stabilized 
configuration, too...

Now, some light at the end of the tunnel : we are almost done with the 
big refactoring in the server : ServerEntry, MINA 2, and such, are 
pretty much done. There are a few more steps to do in order to get a 2.0 
out of the oven, and we have to keep this target in mind now. But we 
also have to keep in mind that we have users, and the more closer we 
will be to 2.0, the more precise must be the documentation, because it's 
not an easy and free task.

I think some items can be discussed, and may be the best would be to 
create issues about each parts which is painfull in JIRA (under the DIR 
keyword).

wdyt ?

-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org



Re: Partition configurations [was Re: [CONF] Apache Directory SandBox: Draft - ... ]

Posted by Stefan Zoerner <st...@labeo.de>.
Hi Felix,

Felix Knecht wrote:
> Following the ML I recently saw multiple questions about writing a custom partition so there seems to be an interest in
> this topic.
> 
> Now I'm just following Stefans (big thanks about this howto!) and hit on the needs to change the server.xml when adding
> a custom partition.

You have found my first tries very fast :-)

For the others (warning: work in progress!):
http://cwiki.apache.org/confluence/display/DIRxSBOX/Draft+-+How+to+write+a+simple+custom+partition+for+ApacheDS

I have just started to write this tutorial on te topic; I have noticed 
the many questions as well. And I was curious how the most simple custom 
partition (hello world) would look like.

> I'd like to raise a discussion about this.
> Is it really needed to do this?!

I think, adding some lines to the server.xml is OK, and I guess it is 
also necessary to do that currently. A custom partition is not really a 
module. You may use the same custom partition implementation several 
times within one server configuration. This is what we currently do with 
the JDBM partition implementation.

> Isn't a new partition just a module to plug into an existing server? Wouldn't it be nice just to drop in the jar
> containing the custom partition and restart the server?
> No need to change any configurations?

As I said; it would be OK for me to add the partitions to the 
configuration manually . This is not a regular task. Currently, it is 
not easy the modify the server.xml, because it is a xbean / Spring 
hybrid, and it is not very well documented.

> Why not let spring detecting all the existing partition modules for us?

The functionality you described would be feasible with OSGi as well. 
This is a step (ApacheDS as OSGi bundle(s)) we plan for ages. A custom 
partition implementation would exactly come into play like you have 
described. But it has to be figured out how the instances (== 
partitions) are handles. The best way would probably be to store the 
whole configuration within the DIT anyway. No need for XML and Spring 
anymore. But this is probably not reachable in the near future.

Greetings from Hamburg,
     Stefan