You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Stefan Seelmann <se...@apache.org> on 2010/03/07 12:54:33 UTC
Search performance
Hi dev,
for the ApacheDS 1.5.6 release I created the installers and tested the
zip archive.
I injected an ou and 10000 inetOrgPerson entries [1]. When I run the
following ldapsearch command it takes approx. one second to get a result:
ldapsearch -h localhost -p 10389 -D "uid=admin,ou=system" -w secret -b
"dc=example,dc=com" -s sub "(uid=user.9999)"
I also used the searchrate tool from slamd and I only get 1 search per
second:
./searchrate.sh -h localhost -p 10389 -D "uid=admin,ou=system" -w secret
-b "dc=example,dc=com" -s sub -f "(uid=user.9999)" -t 1
Then I tried to tune the server: I set -Xmx1024m to give it more Ram.
And I increased the cache size in server.xml [2]. However with
searchrate I still don't get more then 3 searches per second.
I hope that I just forgot some additional tuning parameter.
Kind Regards,
Stefan
[1] http://people.apache.org/~seelmann/com-example-10000.ldif.gz (1MB!)
[2] http://pastebin.com/xMjR9zrK
Re: Search performance
Posted by Alex Karasulu <ak...@gmail.com>.
On Sun, Mar 7, 2010 at 3:22 PM, Stefan Seelmann <se...@apache.org> wrote:
> Ok, I think I got the issue. When doing a "normal" search with a filter like
> (uid=user.9999) the referral interceptor expands that filter to
> (|((uid=user.9999))(objectClass=referral)).
>
> The objectClass index however is a system index and not used by the
> optimizer, see [1] and [2].
>
> It should be easy to fix that, I'll try to fix it, stand by...
Darn I thought we fixed this issue of the optimizer not using the
objectClass by making it a system index. I guess I'm outdated and
confused on this topic.
Good job though in catching this issue and thanks for the fix.
>
> [1] http://www.mail-archive.com/dev@directory.apache.org/msg27163.html
> [2] https://issues.apache.org/jira/browse/DIRSERVER-1465
>
>> One guess: it may have something to do with referrals. If I add a "-M" to
>> the ldapsearch command (which sends the ManageDsaIT control) the search is
>> faster. I'll try to debug into the server.
>>
>>
>> Alex Karasulu wrote:
>>>
>>> Wow this is horrendous - what's the cause of this any idea? We were at
>>> 5000K searches per second before.
>>>
>>> On Sun, Mar 7, 2010 at 1:54 PM, Stefan Seelmann <se...@apache.org>
>>> wrote:
>>>>
>>>> Hi dev,
>>>>
>>>> for the ApacheDS 1.5.6 release I created the installers and tested the
>>>> zip
>>>> archive.
>>>>
>>>> I injected an ou and 10000 inetOrgPerson entries [1]. When I run the
>>>> following ldapsearch command it takes approx. one second to get a
>>>> result:
>>>>
>>>> ldapsearch -h localhost -p 10389 -D "uid=admin,ou=system" -w secret -b
>>>> "dc=example,dc=com" -s sub "(uid=user.9999)"
>>>>
>>>> I also used the searchrate tool from slamd and I only get 1 search per
>>>> second:
>>>>
>>>> ./searchrate.sh -h localhost -p 10389 -D "uid=admin,ou=system" -w secret
>>>> -b
>>>> "dc=example,dc=com" -s sub -f "(uid=user.9999)" -t 1
>>>>
>>>>
>>>> Then I tried to tune the server: I set -Xmx1024m to give it more Ram.
>>>> And I
>>>> increased the cache size in server.xml [2]. However with searchrate I
>>>> still
>>>> don't get more then 3 searches per second.
>>>>
>>>>
>>>> I hope that I just forgot some additional tuning parameter.
>>>>
>>>>
>>>> Kind Regards,
>>>> Stefan
>>>>
>>>>
>>>> [1] http://people.apache.org/~seelmann/com-example-10000.ldif.gz (1MB!)
>>>> [2] http://pastebin.com/xMjR9zrK
>>>>
>>>
>>>
>>>
>>
>
>
--
Alex Karasulu
My Blog :: http://www.jroller.com/akarasulu/
Apache Directory Server :: http://directory.apache.org
Apache MINA :: http://mina.apache.org
Re: Search performance
Posted by Stefan Seelmann <se...@apache.org>.
Ok, I think I got the issue. When doing a "normal" search with a filter
like (uid=user.9999) the referral interceptor expands that filter to
(|((uid=user.9999))(objectClass=referral)).
The objectClass index however is a system index and not used by the
optimizer, see [1] and [2].
It should be easy to fix that, I'll try to fix it, stand by...
Kind Regards,
Stefan
[1] http://www.mail-archive.com/dev@directory.apache.org/msg27163.html
[2] https://issues.apache.org/jira/browse/DIRSERVER-1465
> One guess: it may have something to do with referrals. If I add a "-M"
> to the ldapsearch command (which sends the ManageDsaIT control) the
> search is faster. I'll try to debug into the server.
>
>
> Alex Karasulu wrote:
>> Wow this is horrendous - what's the cause of this any idea? We were at
>> 5000K searches per second before.
>>
>> On Sun, Mar 7, 2010 at 1:54 PM, Stefan Seelmann <se...@apache.org>
>> wrote:
>>> Hi dev,
>>>
>>> for the ApacheDS 1.5.6 release I created the installers and tested
>>> the zip
>>> archive.
>>>
>>> I injected an ou and 10000 inetOrgPerson entries [1]. When I run the
>>> following ldapsearch command it takes approx. one second to get a
>>> result:
>>>
>>> ldapsearch -h localhost -p 10389 -D "uid=admin,ou=system" -w secret -b
>>> "dc=example,dc=com" -s sub "(uid=user.9999)"
>>>
>>> I also used the searchrate tool from slamd and I only get 1 search per
>>> second:
>>>
>>> ./searchrate.sh -h localhost -p 10389 -D "uid=admin,ou=system" -w
>>> secret -b
>>> "dc=example,dc=com" -s sub -f "(uid=user.9999)" -t 1
>>>
>>>
>>> Then I tried to tune the server: I set -Xmx1024m to give it more Ram.
>>> And I
>>> increased the cache size in server.xml [2]. However with searchrate I
>>> still
>>> don't get more then 3 searches per second.
>>>
>>>
>>> I hope that I just forgot some additional tuning parameter.
>>>
>>>
>>> Kind Regards,
>>> Stefan
>>>
>>>
>>> [1] http://people.apache.org/~seelmann/com-example-10000.ldif.gz (1MB!)
>>> [2] http://pastebin.com/xMjR9zrK
>>>
>>
>>
>>
>
Re: Search performance
Posted by Stefan Seelmann <se...@apache.org>.
One guess: it may have something to do with referrals. If I add a "-M"
to the ldapsearch command (which sends the ManageDsaIT control) the
search is faster. I'll try to debug into the server.
Alex Karasulu wrote:
> Wow this is horrendous - what's the cause of this any idea? We were at
> 5000K searches per second before.
>
> On Sun, Mar 7, 2010 at 1:54 PM, Stefan Seelmann <se...@apache.org> wrote:
>> Hi dev,
>>
>> for the ApacheDS 1.5.6 release I created the installers and tested the zip
>> archive.
>>
>> I injected an ou and 10000 inetOrgPerson entries [1]. When I run the
>> following ldapsearch command it takes approx. one second to get a result:
>>
>> ldapsearch -h localhost -p 10389 -D "uid=admin,ou=system" -w secret -b
>> "dc=example,dc=com" -s sub "(uid=user.9999)"
>>
>> I also used the searchrate tool from slamd and I only get 1 search per
>> second:
>>
>> ./searchrate.sh -h localhost -p 10389 -D "uid=admin,ou=system" -w secret -b
>> "dc=example,dc=com" -s sub -f "(uid=user.9999)" -t 1
>>
>>
>> Then I tried to tune the server: I set -Xmx1024m to give it more Ram. And I
>> increased the cache size in server.xml [2]. However with searchrate I still
>> don't get more then 3 searches per second.
>>
>>
>> I hope that I just forgot some additional tuning parameter.
>>
>>
>> Kind Regards,
>> Stefan
>>
>>
>> [1] http://people.apache.org/~seelmann/com-example-10000.ldif.gz (1MB!)
>> [2] http://pastebin.com/xMjR9zrK
>>
>
>
>
Re: Search performance
Posted by Alex Karasulu <ak...@gmail.com>.
Wow this is horrendous - what's the cause of this any idea? We were at
5000K searches per second before.
On Sun, Mar 7, 2010 at 1:54 PM, Stefan Seelmann <se...@apache.org> wrote:
> Hi dev,
>
> for the ApacheDS 1.5.6 release I created the installers and tested the zip
> archive.
>
> I injected an ou and 10000 inetOrgPerson entries [1]. When I run the
> following ldapsearch command it takes approx. one second to get a result:
>
> ldapsearch -h localhost -p 10389 -D "uid=admin,ou=system" -w secret -b
> "dc=example,dc=com" -s sub "(uid=user.9999)"
>
> I also used the searchrate tool from slamd and I only get 1 search per
> second:
>
> ./searchrate.sh -h localhost -p 10389 -D "uid=admin,ou=system" -w secret -b
> "dc=example,dc=com" -s sub -f "(uid=user.9999)" -t 1
>
>
> Then I tried to tune the server: I set -Xmx1024m to give it more Ram. And I
> increased the cache size in server.xml [2]. However with searchrate I still
> don't get more then 3 searches per second.
>
>
> I hope that I just forgot some additional tuning parameter.
>
>
> Kind Regards,
> Stefan
>
>
> [1] http://people.apache.org/~seelmann/com-example-10000.ldif.gz (1MB!)
> [2] http://pastebin.com/xMjR9zrK
>
--
Alex Karasulu
My Blog :: http://www.jroller.com/akarasulu/
Apache Directory Server :: http://directory.apache.org
Apache MINA :: http://mina.apache.org