You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@directory.apache.org by Howard Chu <hy...@symas.com> on 2007/05/03 18:06:08 UTC
SambaXP talk
For anyone curious, the slides from my presentation at the SambaXP
conference last week are now up on my web site.
http://highlandsun.com/hyc/SambaXP.pdf
Much of the material on malloc benchmarking was already presented at
SCALE5x earlier this year. New material in these slides include
benchmark results for OpenLDAP 2.3.34 vs FedoraDS 1.0.4, OpenDS 0.1-34,
and ApacheDS 1.0.1 on Linux 2.6. The machine used for these tests is the
same SunFire X4100 used in these tests last year
http://www.symas.com/benchmark-auth.shtml
Some earlier discussion of these results is also on the Connexitor blog:
http://www.connexitor.com/blog/pivot/entry.php?id=130
http://www.connexitor.com/blog/pivot/entry.php?id=131
We didn't test Microsoft ActiveDirectory because we don't have a 64 bit
build of it available, nor do we have a 64 bit Windows system available.
I suppose we can run those tests and publish those results sometime down
the road. If anybody is interested in helping to run more tests along
these lines, feel free to contact me. (Judging from the information
available here
http://www.microsoft.com/downloads/details.aspx?FamilyID=52e7c3bd-570a-475c-96e0-316dc821e3e7&displaylang=en
I don't think there'd be any different news on that front anyway.)
This round of benchmarking was quite educational. We discovered a memory
leak in FedoraDS (and reported that to their maintainers, of course).
Analyzing the results also shows that while FDS' entry cache is
reasonably effective, they have a performance bottleneck in their
frontend, most likely in connection management. I didn't profile it to
get a closer look, though I'm sure a profiler would make the culprit
obvious.
Also FDS is too memory hungry, which causes their server to run out of
memory much sooner than OpenLDAP (running on the identical machine, with
identical cache memory settings, indices, workload, etc...) so their
performance drops off quite sharply as database sizes increase and
memory becomes constrained. (This is something different from the malloc
degradation I was observing in OpenLDAP before, although FDS appears to
be affected by that as well.)
We also observed that Sun/Fedora's documentation and advice on
performance tuning for their servers is wrong, and we can obtain better
performance by ignoring their recommendations. Even though Sun, Fedora,
and OpenLDAP all use BerkeleyDB, it's obvious that they don't use it as
effectively as we do.
Given the extremely young age of the OpenDS code base I'd say they've
done a really good job thus far, even managing to beat FDS in one test.
But I'd also say they've gotten as good as they can possibly get with a
pure Java solution; indeed their future plans for entry caching require
support outside the JVM (e.g. using a tmpfs cache). Since they're still
at best 3x slower than OpenLDAP, it's unlikely they will ever achieve
their stated goal of delivering high performance with a Java code base.
Another thing to keep in mind - unlike FedoraDS (and Sun DSEE), OpenLDAP
is fully RFC compliant, does full schema checking, and fully implements
the X.500 data model. It does all that and is still over 3 times faster.
It's not just about speed; correctness still comes first. With good
engineering you don't have to sacrifice correctness to get performance.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Re: SambaXP talk
Posted by Emmanuel Lecharny <el...@apache.org>.
Hi Howard,
Howard Chu a écrit :
> For anyone curious, the slides from my presentation at the SambaXP
> conference last week are now up on my web site.
> http://highlandsun.com/hyc/SambaXP.pdf
Thanks for the slides, there are interesting numbers in it, though not
very pleasant for us ;)
>
> Much of the material on malloc benchmarking was already presented at
> SCALE5x earlier this year. New material in these slides include
> benchmark results for OpenLDAP 2.3.34 vs FedoraDS 1.0.4, OpenDS 0.1-34,
> and ApacheDS 1.0.1 on Linux 2.6. The machine used for these tests is the
> same SunFire X4100 used in these tests last year
> http://www.symas.com/benchmark-auth.shtml
>
> Some earlier discussion of these results is also on the Connexitor blog:
> http://www.connexitor.com/blog/pivot/entry.php?id=130
> http://www.connexitor.com/blog/pivot/entry.php?id=131
Is it possible that you post the configuration you have used for A2pache
DS (server.xml, and java version ?).
<snip/>
> Another thing to keep in mind - unlike FedoraDS (and Sun DSEE), OpenLDAP
> is fully RFC compliant, does full schema checking, and fully implements
> the X.500 data model. It does all that and is still over 3 times faster.
> It's not just about speed; correctness still comes first. With good
> engineering you don't have to sacrifice correctness to get performance.
Very true. We have worked on correctness first too, and we are not
finished yet, but performance is also somewhere back in our mind. It
takes time to get correctness, and if you are not starting with a lot of
experience like Sun guys did (they have SunDS running for a loooong time
before they started OpenDS), you have to dedicate a lot of time on
correctness.
Just expect to see ApacheDS improving by a large margin in the
performance field in the next few months (hey, we have a wide margin for
improvment ;)
It could be interesting if we can share the very same tests so that the
comparizon would be fair.
If you are going for a new injector tool, then may be we can share some
ideas about it. Basically, I buy the idea.
Thanks for the feedback !
PS: keyboard is dead :( RIP.
Re: SambaXP talk
Posted by Stefan Zoerner <sz...@apache.org>.
Hi Howard!
Howard Chu wrote:
> Another thing to keep in mind - unlike FedoraDS (and Sun DSEE), OpenLDAP
> is fully RFC compliant, does full schema checking, and fully implements
> the X.500 data model. It does all that and is still over 3 times faster.
> It's not just about speed; correctness still comes first. With good
> engineering you don't have to sacrifice correctness to get performance.
I just wanted to note that I 100% agree with your statement about how
important standard compliance is; especially in the area I work (EAI).
It therefore perfectly fits that correctness was one of the primary
goals ApacheDS 1.0 planned to achieve.
It is difficult to measure standard compliance. Passing the LDAP
certification of the Open Group
http://www.opengroup.org/certification/directory-home.html
was one idea we had to get feedback in this area.
ApacheDS 1.0 get certified by the Open Group in Oct. 2006, and we will
continue to release exclusively versions which fulfill their
requirements (knowing that the tests are not complete; and therefore
can't *prove* correctness).
Greetings from Hamburg,
Stefan
---8<---
Stefan Zoerner (szoerner@apache.org)
Committer :: PMC Member
Apache Directory Project
http://directory.apache.org