Performance : ApacheDS and web applications

[Wim V <wi...@pizzastop.be>]

Hi,


I want to use apachDS for user persistence in a web application.
How is the performance of apache DS (embedded or not) in the following
situation, which can exist if users are allowed to "register themselves" in
the directory :

- Large number of users (inetOrgPerson) in all in one ou (f.e. users)
- rdn consist of only one attribute, which is uid

Let's say I have 1.000.000 users or more and they all live in 
ou=sales, ou=css, ou=users, dc=theonlinecompany, dc=com
with a dn like 
uid=tomsr19878, ou=sales, ou=css, ou=users, dc=theonlinecompany, dc=com
(unless they are promoted)

A badly structured tree, that is.


More specifically my questions are these :

- What is performance in terms of lookup (number/second) of credentials and
authorities/group memberships?

- What is application performance in the given situation related to using
the classic backend :
		- Database server
		- 3 tables : users, groups (or units) and authorities
		- All 3 tables indexed.

- How does this situation affect performance of lookups in other units of
the same directory?

- Does apache support indexing of units (or something else maybe?)
(Was apacheDS in some way designed keeping this in mind?)

- Did anyone practically test ApacheDS in these conditions yet?
What were the results?


Thanks to everyone for all comments.


Wim Verreycken

Re: Performance : ApacheDS and web applications

[Alex Karasulu <ak...@apache.org>]

On Thu, Apr 3, 2008 at 10:40 AM, Emmanuel Lecharny <el...@gmail.com>
wrote:

> Wim V wrote:
>
>
> >
> Depends. We have reached 12 000 random search requests per second on a 16
> way CPU,


8 way!!!!

Re: Performance : ApacheDS and web applications

[Emmanuel Lecharny <el...@gmail.com>]

Wim V wrote:
> Hi,
>   
Hi,

Let's try to give you some answer, assuming that there are not absolute, 
as they will depend on many different factors, including the server you 
will use...
>
> I want to use apachDS for user persistence in a web application.
> How is the performance of apache DS (embedded or not) in the following
> situation, which can exist if users are allowed to "register themselves" in
> the directory :
>
> - Large number of users (inetOrgPerson) in all in one ou (f.e. users)
> - rdn consist of only one attribute, which is uid
>
> Let's say I have 1.000.000 users or more and they all live in 
> ou=sales, ou=css, ou=users, dc=theonlinecompany, dc=com
> with a dn like 
> uid=tomsr19878, ou=sales, ou=css, ou=users, dc=theonlinecompany, dc=com
> (unless they are promoted)
>   
Ok, Typically what we are using when we do some performance tests using 
MakeLdif and Slamd.

> A badly structured tree, that is.
>   
Well, not that much. I would say, a typical LDAP tree.
>
> More specifically my questions are these :
>
> - What is performance in terms of lookup (number/second) of credentials and
> authorities/group memberships?
>   
Depends. We have reached 12 000 random search requests per second on a 
16 way CPU, on a server loaded with 4 million entries. You can have 
better performance using the server embedded, as you won't have the 
network layer, which is quite costly (no more network latency, no more 
ASN.1 BER encoding/decoding). I would say you may gain 30% more in this 
case.
> - What is application performance in the given situation related to using
> the classic backend :
> 		- Database server
> 		- 3 tables : users, groups (or units) and authorities
> 		- All 3 tables indexed.
>   
No idea. Never tested it...
> - How does this situation affect performance of lookups in other units of
> the same directory?
>   
Depends on what you store in the other partitions. The main issue is the 
cache : you can affect some cache to each partition, so if you define a 
partition specifically for your need, the impact on the other partitions 
would be noticable, but limited.
> - Does apache support indexing of units (or something else maybe?)
> (Was apacheDS in some way designed keeping this in mind?)
>   
What do you mean by "indexing of units" ? Can you define what you call a 
"unit" ?  ADS is using BTrees internally, which is a well known 
O(log2(N)) structure, FYI. You can set index on each attribute, if needed.
> - Did anyone practically test ApacheDS in these conditions yet?
>   
yes, as I said, the last time we did such tests was one month ago, with 
4 millions entries, and doing a random lookup on the server.
> What were the results?
>   
12 000 search/s, returning the full entry (with all the non-operational 
attributes).
>
> Thanks to everyone for all comments.
>   
Hope it helps.



-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org