You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@directory.apache.org by Ca...@ibs-ag.com on 2012/02/29 19:14:04 UTC
Quesiton about best way to index attributes
Hi, We have apacheds 2.0-M5 db setup with all ~90k users under one OU. Yes, I know they should be stored with some hierarchy but that's not an option for this
particular customer. I want to optimize searching by lastname, (sn), firstname (givenName) and displayName. I have created indexes under the partition and
we see ads-indexcachesize is set by default to 100. Assuming the value correlates to a value for the attribute, Can I set this to 50,000 for each of the attribs?.
I can dedicate ~1GB of heap space for the ApacheDS JVM.
Also, is there a way to rebuild the indexes in 2.0? I can't seem to find how that's done.
For example In the UI someone might enter "Smit" looking for Smith.
This gets converted into the following query with the OU as the baseDN.
(&
(objectclass=inetOrgPerson)
(|
(displayname=Smit*)
(sn=Smit*)
(givenName=Smit*)
)
)
Lastly, would we be better off performing three separate queries (for displayname,sn,givenName) or will the logical OR search above break the moment it finds a match? Thanks.
Re: Quesiton about best way to index attributes
Posted by Kiran Ayyagari <ka...@apache.org>.
On Thu, Mar 1, 2012 at 11:44 AM, Emmanuel Lécharny <el...@gmail.com> wrote:
> Le 2/29/12 7:50 PM, Kiran Ayyagari a écrit :
>
>> On Wed, Feb 29, 2012 at 11:44 PM,<Ca...@ibs-ag.com> wrote:
>>>
>>> Hi, We have apacheds 2.0-M5 db setup with all ~90k users under one OU.
>>> Yes, I know they should be stored with some hierarchy but that's not an
>>> option for this
>>> particular customer.
>
> Having 90K users in one single OU is *not* a bad practice. You may want to
> setup a hierarchy, but this is not mandatory.
>
>
>>> I want to optimize searching by lastname, (sn), firstname (givenName)
>>> and displayName. I have created indexes under the partition and
>>> we see ads-indexcachesize is set by default to 100. Assuming the value
>>> correlates to a value for the attribute, Can I set this to 50,000 for each
>>> of the attribs?.
>>
>> that won't help much, setting up more memory for the JVM will help
>
> You still can increase the number to a higher value, but you must know that
> as soon as the higher BTree pages will be loaded, the speedup will be lower.
> With 100 pages loaded, you have most of the BTree loaded.
>
> If you have enough memory, just increase this value to 50 000, should not
> harm.
>
>
> In any case, I would suggest that once you have increased those values for
> each index that you do a full search to load all of them in memory, to check
> that you don't get an OOM, before running in production.
>
>>
>> Also, is there a way to rebuild the indexes in 2.0? I can't seem to find
>> how that's done.
>>
>> no, one way is to delete the index and restart the server followed by
>> adding the index and restart, then the server will automatically build
>> the index
>> during startup
>
> Kiran : haven't we added some utility tool to do that ? (It's a bit far in
> the past.. If we don't have them, we must add them)
>
yes, I have added but was not maintained after that
> I'm wondering if we don't have a JIRA for that...
>
AFAIR there is one for generic tooling
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
--
Kiran Ayyagari
Re: Quesiton about best way to index attributes
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 2/29/12 7:50 PM, Kiran Ayyagari a écrit :
> On Wed, Feb 29, 2012 at 11:44 PM,<Ca...@ibs-ag.com> wrote:
>> Hi, We have apacheds 2.0-M5 db setup with all ~90k users under one OU. Yes, I know they should be stored with some hierarchy but that's not an option for this
>> particular customer.
Having 90K users in one single OU is *not* a bad practice. You may want
to setup a hierarchy, but this is not mandatory.
>> I want to optimize searching by lastname, (sn), firstname (givenName) and displayName. I have created indexes under the partition and
>> we see ads-indexcachesize is set by default to 100. Assuming the value correlates to a value for the attribute, Can I set this to 50,000 for each of the attribs?.
> that won't help much, setting up more memory for the JVM will help
You still can increase the number to a higher value, but you must know
that as soon as the higher BTree pages will be loaded, the speedup will
be lower. With 100 pages loaded, you have most of the BTree loaded.
If you have enough memory, just increase this value to 50 000, should
not harm.
In any case, I would suggest that once you have increased those values
for each index that you do a full search to load all of them in memory,
to check that you don't get an OOM, before running in production.
>
> Also, is there a way to rebuild the indexes in 2.0? I can't seem to find how that's done.
>
> no, one way is to delete the index and restart the server followed by
> adding the index and restart, then the server will automatically build
> the index
> during startup
Kiran : haven't we added some utility tool to do that ? (It's a bit far
in the past.. If we don't have them, we must add them)
I'm wondering if we don't have a JIRA for that...
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Quesiton about best way to index attributes
Posted by Kiran Ayyagari <ka...@apache.org>.
On Wed, Feb 29, 2012 at 11:44 PM, <Ca...@ibs-ag.com> wrote:
> Hi, We have apacheds 2.0-M5 db setup with all ~90k users under one OU. Yes, I know they should be stored with some hierarchy but that's not an option for this
> particular customer. I want to optimize searching by lastname, (sn), firstname (givenName) and displayName. I have created indexes under the partition and
> we see ads-indexcachesize is set by default to 100. Assuming the value correlates to a value for the attribute, Can I set this to 50,000 for each of the attribs?.
that won't help much, setting up more memory for the JVM will help
> I can dedicate ~1GB of heap space for the ApacheDS JVM.
>
cool, then set the -Xms and -Xmx flags in the java options
> Also, is there a way to rebuild the indexes in 2.0? I can't seem to find how that's done.
>
no, one way is to delete the index and restart the server followed by
adding the index and restart, then the server will automatically build
the index
during startup
> For example In the UI someone might enter "Smit" looking for Smith.
>
> This gets converted into the following query with the OU as the baseDN.
>
> (&
> (objectclass=inetOrgPerson)
> (|
> (displayname=Smit*)
> (sn=Smit*)
> (givenName=Smit*)
> )
> )
>
> Lastly, would we be better off performing three separate queries (for displayname,sn,givenName) or will the logical OR search above break the moment it finds a match? Thanks.
the logical OR breaks as soon as it finds a match
--
Kiran Ayyagari