You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Johan Asterholm <as...@gmail.com> on 2007/01/22 15:04:57 UTC
Partitions
Hi,
I'm trying to develop LDAP integration towards an application, but I have a
hard time to understand how it all should work.
To learn how it should be done I'm trying to develop a backend which works
towards a HashMap where the hash Map includes a key (Identifier) And then
another hashmap with attributes (They are not "Attributes" but simply String
names of the attribute). I prefer this way since this is quite similar to
what i need to do when integrating towards or application.
so searching for uid=1,ou=user,cn=example,cn=com would point to a row in
the hashmap, but ou=user,cn=example,cn=com is static.
Is there any example where a simple backend like this is done? Maybe not
with a HashMap but something similar?
This would help me tremendously.
BR / Johan Asterholm
Re: Partitions
Posted by Johan Asterholm <as...@gmail.com>.
Just updating the list with the latest mail. Anything pointing me in the
right direction would be appreciated.
BR / Johan Asterholm
2007/1/22, Emmanuel Lecharny <el...@gmail.com>:
>
> oh... I don't see a path to solve your problem in a couple of days. Woudl
> it be a couple of weeks, may be...
>
> Anyone ?
>
> Sorry :(
>
> On 1/22/07, Johan Asterholm <asterholm@gmail.com > wrote:
> >
> > Hi,
> >
> > Basically i have a couple of days to solve this so its quite urgent for
> > me.
> >
> > BR / Johan
> >
> >
> > 2007/1/22, Emmanuel Lecharny <el...@gmail.com>:
> > >
> > > ok, get it. So you want your application to be the backend, basically.
> > >
> > > I know this is possible, but I think you will have to implemant
> > > something like 11 interfaces to make it works.
> > >
> > > Hopefully, Alex may give you more informations about it. I also have
> > > to look at this part, and ths could be the perfect occasion.
> > >
> > > Is this urgent ?
> > >
> > > On 1/22/07, Johan Asterholm <asterholm@gmail.com > wrote:
> > > >
> > > > Hi,
> > > >
> > > > Im interesting in using ADS as a mapping between LDAP and my
> > > > application. to get LDAP support. so basically just the frontend.
> > > >
> > > > I have looked arround for examples on how to write the backend but i
> > > > havent realy found soemthing usefull.
> > > >
> > > > BR / Johan Asterholm
> > > >
> > > >
> > > > 2007/1/22, Emmanuel Lecharny <el...@gmail.com>:
> > > > >
> > > > > ok.
> > > > >
> > > > > But do you need ADS at all? Or are you just interested by storing
> > > > > DNs into a backend ?
> > > > >
> > > > > On 1/22/07, Johan Asterholm < asterholm@gmail.com > wrote:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > By backend can store up to several milion records but its quite
> > > > > > fast in answering (about ~20 MS) when having 2 milion records. The
> > > > > > application will handle caching itself so no caching wold be necessary.
> > > > > > Basically what I'm looking for is just an example for writing my backend.
> > > > > >
> > > > > > BR / Johan Asterholm
> > > > > >
> > > > > >
> > > > > > 2007/1/22, Emmanuel Lecharny <el...@gmail.com>:
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > the backend is using its own structure atm. It's based on
> > > > > > > JDBM, which is a B-tree storage.
> > > > > > >
> > > > > > > The DN are also stored in a B-Tree, and this is not
> > > > > > > necessarily optimal. We may want to go for a H-tree instead of a B-tree for
> > > > > > > these DN (B-tree and H-tree storages are available in JDBM)
> > > > > > >
> > > > > > > So basically, it's just a question to associate a H-tree to
> > > > > > > the DN storage.
> > > > > > >
> > > > > > > Ok, this is theory. In the 'real world', this would need more
> > > > > > > analysis, and we must evaluate the impact of such a modification on the code
> > > > > > > base.
> > > > > > >
> > > > > > > Now, I have a question : how big is your data set ? If it's a
> > > > > > > few thousnads, there would be a very little increase in performance doing
> > > > > > > that : you have to consider that the cache system will very quickly keep in
> > > > > > > memory all the DN of your database. (you might want to increase the cache
> > > > > > > size for this kind of objects in your partition :
> > > > > > > ...
> > > > > > > <property name="indexedAttributes">
> > > > > > > <set>
> > > > > > > <bean class="
> > > > > > > org.apache.directory.server.core.partition.impl.btree.MutableIndexConfiguration
> > > > > > > ">
> > > > > > > <property name="attributeId" value="
> > > > > > > 1.3.6.1.4.1.18060.0.4.1.2.1" />
> > > > > > > <property name="cacheSize" value="100" />
> > > > > > > </bean>
> > > > > > > ...
> > > > > > >
> > > > > > > In this default configuration, 100 DNs are kept into the
> > > > > > > cache. Just increase it to fit your needs.
> > > > > > >
> > > > > > > If in cache, grabbing a DN is really fast, wether you use
> > > > > > > H-tree or not.
> > > > > > >
> > > > > > > You will just have to consider if this would be of interest to
> > > > > > > modify the backend (with all the burden it suppose) instead of increasing
> > > > > > > the cache. And this is again a question of how many DN will you have.
> > > > > > >
> > > > > > > I hope this help.
> > > > > > >
> > > > > > > Feel free to go further, we really have this Hashing in our
> > > > > > > radar.
> > > > > > >
> > > > > > > Emmanuel
> > > > > > >
> > > > > > > On 1/22/07, Johan Asterholm < asterholm@gmail.com > wrote:
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I'm trying to develop LDAP integration towards an
> > > > > > > > application, but I have a hard time to understand how it all should work.
> > > > > > > >
> > > > > > > > To learn how it should be done I'm trying to develop a
> > > > > > > > backend which works towards a HashMap where the hash Map includes a key
> > > > > > > > (Identifier) And then another hashmap with attributes (They are not
> > > > > > > > "Attributes" but simply String names of the attribute). I prefer this way
> > > > > > > > since this is quite similar to what i need to do when integrating towards or
> > > > > > > > application.
> > > > > > > >
> > > > > > > > so searching for uid=1,ou=user,cn=example,cn=com would
> > > > > > > > point to a row in the hashmap, but ou=user,cn=example,cn=com is static.
> > > > > > > >
> > > > > > > > Is there any example where a simple backend like this is
> > > > > > > > done? Maybe not with a HashMap but something similar?
> > > > > > > >
> > > > > > > > This would help me tremendously.
> > > > > > > >
> > > > > > > > BR / Johan Asterholm
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Cordialement,
> > > > > > > Emmanuel Lécharny
> > > > > > > www.iktek.com
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Cordialement,
> > > > > Emmanuel Lécharny
> > > > > www.iktek.com
> > > > >
> > > >
> > > >
> > >
> > >
> > > --
> > > Cordialement,
> > > Emmanuel Lécharny
> > > www.iktek.com
> > >
> >
> >
>
>
> --
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
Re: Partitions
Posted by Johan Asterholm <as...@gmail.com>.
Hi,
By backend can store up to several milion records but its quite fast in
answering (about ~20 MS) when having 2 milion records. The application will
handle caching itself so no caching wold be necessary. Basically what I'm
looking for is just an example for writing my backend.
BR / Johan Asterholm
2007/1/22, Emmanuel Lecharny <el...@gmail.com>:
>
> Hi,
>
> the backend is using its own structure atm. It's based on JDBM, which is a
> B-tree storage.
>
> The DN are also stored in a B-Tree, and this is not necessarily optimal.
> We may want to go for a H-tree instead of a B-tree for these DN (B-tree and
> H-tree storages are available in JDBM)
>
> So basically, it's just a question to associate a H-tree to the DN
> storage.
>
> Ok, this is theory. In the 'real world', this would need more analysis,
> and we must evaluate the impact of such a modification on the code base.
>
> Now, I have a question : how big is your data set ? If it's a few
> thousnads, there would be a very little increase in performance doing that :
> you have to consider that the cache system will very quickly keep in memory
> all the DN of your database. (you might want to increase the cache size for
> this kind of objects in your partition :
> ...
> <property name="indexedAttributes">
> <set>
> <bean class="
> org.apache.directory.server.core.partition.impl.btree.MutableIndexConfiguration
> ">
> <property name="attributeId" value=" 1.3.6.1.4.1.18060.0.4.1.2.1"
> />
> <property name="cacheSize" value="100" />
> </bean>
> ...
>
> In this default configuration, 100 DNs are kept into the cache. Just
> increase it to fit your needs.
>
> If in cache, grabbing a DN is really fast, wether you use H-tree or not.
>
> You will just have to consider if this would be of interest to modify the
> backend (with all the burden it suppose) instead of increasing the cache.
> And this is again a question of how many DN will you have.
>
> I hope this help.
>
> Feel free to go further, we really have this Hashing in our radar.
>
> Emmanuel
>
> On 1/22/07, Johan Asterholm < asterholm@gmail.com> wrote:
> >
> > Hi,
> >
> > I'm trying to develop LDAP integration towards an application, but I
> > have a hard time to understand how it all should work.
> >
> > To learn how it should be done I'm trying to develop a backend which
> > works towards a HashMap where the hash Map includes a key (Identifier) And
> > then another hashmap with attributes (They are not "Attributes" but simply
> > String names of the attribute). I prefer this way since this is quite
> > similar to what i need to do when integrating towards or application.
> >
> > so searching for uid=1,ou=user,cn=example,cn=com would point to a row
> > in the hashmap, but ou=user,cn=example,cn=com is static.
> >
> > Is there any example where a simple backend like this is done? Maybe not
> > with a HashMap but something similar?
> >
> > This would help me tremendously.
> >
> > BR / Johan Asterholm
> >
>
>
>
> --
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
Re: Partitions
Posted by Emmanuel Lecharny <el...@gmail.com>.
hey,
I may have missed something :
may be you already have all the DN stored in a HashMap, and you want to use
this storage within ADS. In this case, this is really a diffferent story. We
have also something in our roadmap that will allow any other backend to be
used, as soon as you write the correct interface (which already exists, btw,
but I am not able to give you a clear explaination on how to use it atm due
to my limited knowledge of this part of the server :(
Emmanuel
On 1/22/07, Emmanuel Lecharny <el...@gmail.com> wrote:
>
> Hi,
>
> the backend is using its own structure atm. It's based on JDBM, which is a
> B-tree storage.
>
> The DN are also stored in a B-Tree, and this is not necessarily optimal.
> We may want to go for a H-tree instead of a B-tree for these DN (B-tree and
> H-tree storages are available in JDBM)
>
> So basically, it's just a question to associate a H-tree to the DN
> storage.
>
> Ok, this is theory. In the 'real world', this would need more analysis,
> and we must evaluate the impact of such a modification on the code base.
>
> Now, I have a question : how big is your data set ? If it's a few
> thousnads, there would be a very little increase in performance doing that :
> you have to consider that the cache system will very quickly keep in memory
> all the DN of your database. (you might want to increase the cache size for
> this kind of objects in your partition :
> ...
> <property name="indexedAttributes">
> <set>
> <bean class="
> org.apache.directory.server.core.partition.impl.btree.MutableIndexConfiguration
> ">
> <property name="attributeId" value=" 1.3.6.1.4.1.18060.0.4.1.2.1"
> />
> <property name="cacheSize" value="100" />
> </bean>
> ...
>
> In this default configuration, 100 DNs are kept into the cache. Just
> increase it to fit your needs.
>
> If in cache, grabbing a DN is really fast, wether you use H-tree or not.
>
> You will just have to consider if this would be of interest to modify the
> backend (with all the burden it suppose) instead of increasing the cache.
> And this is again a question of how many DN will you have.
>
> I hope this help.
>
> Feel free to go further, we really have this Hashing in our radar.
>
> Emmanuel
>
> On 1/22/07, Johan Asterholm < asterholm@gmail.com> wrote:
> >
> > Hi,
> >
> > I'm trying to develop LDAP integration towards an application, but I
> > have a hard time to understand how it all should work.
> >
> > To learn how it should be done I'm trying to develop a backend which
> > works towards a HashMap where the hash Map includes a key (Identifier) And
> > then another hashmap with attributes (They are not "Attributes" but simply
> > String names of the attribute). I prefer this way since this is quite
> > similar to what i need to do when integrating towards or application.
> >
> > so searching for uid=1,ou=user,cn=example,cn=com would point to a row
> > in the hashmap, but ou=user,cn=example,cn=com is static.
> >
> > Is there any example where a simple backend like this is done? Maybe not
> > with a HashMap but something similar?
> >
> > This would help me tremendously.
> >
> > BR / Johan Asterholm
> >
>
>
>
> --
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
--
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Partitions
Posted by Emmanuel Lecharny <el...@gmail.com>.
Hi,
the backend is using its own structure atm. It's based on JDBM, which is a
B-tree storage.
The DN are also stored in a B-Tree, and this is not necessarily optimal. We
may want to go for a H-tree instead of a B-tree for these DN (B-tree and
H-tree storages are available in JDBM)
So basically, it's just a question to associate a H-tree to the DN storage.
Ok, this is theory. In the 'real world', this would need more analysis, and
we must evaluate the impact of such a modification on the code base.
Now, I have a question : how big is your data set ? If it's a few thousnads,
there would be a very little increase in performance doing that : you have
to consider that the cache system will very quickly keep in memory all the
DN of your database. (you might want to increase the cache size for this
kind of objects in your partition :
...
<property name="indexedAttributes">
<set>
<bean class="
org.apache.directory.server.core.partition.impl.btree.MutableIndexConfiguration
">
<property name="attributeId" value="1.3.6.1.4.1.18060.0.4.1.2.1"
/>
<property name="cacheSize" value="100" />
</bean>
...
In this default configuration, 100 DNs are kept into the cache. Just
increase it to fit your needs.
If in cache, grabbing a DN is really fast, wether you use H-tree or not.
You will just have to consider if this would be of interest to modify the
backend (with all the burden it suppose) instead of increasing the cache.
And this is again a question of how many DN will you have.
I hope this help.
Feel free to go further, we really have this Hashing in our radar.
Emmanuel
On 1/22/07, Johan Asterholm <as...@gmail.com> wrote:
>
> Hi,
>
> I'm trying to develop LDAP integration towards an application, but I have
> a hard time to understand how it all should work.
>
> To learn how it should be done I'm trying to develop a backend which works
> towards a HashMap where the hash Map includes a key (Identifier) And then
> another hashmap with attributes (They are not "Attributes" but simply String
> names of the attribute). I prefer this way since this is quite similar to
> what i need to do when integrating towards or application.
>
> so searching for uid=1,ou=user,cn=example,cn=com would point to a row in
> the hashmap, but ou=user,cn=example,cn=com is static.
>
> Is there any example where a simple backend like this is done? Maybe not
> with a HashMap but something similar?
>
> This would help me tremendously.
>
> BR / Johan Asterholm
>
--
Cordialement,
Emmanuel Lécharny
www.iktek.com