[ServerEntry] First performance tests

[Emmanuel Lecharny <el...@gmail.com>]

Hi guys,

now that we have removed references to Attributes into the core, I have 
run my favorite benchmark to see which kind of performance we get. This 
is of course very preliminary, and done in order to see where we have 
performance problems.

You have to keep in mind that this switch was not intended to increase 
the performance, but to improve the correctness of the server, by 
simplifying attributes manipulation (mainly attributes comparisons and 
value manipulations).

So far, the performance I got are far worst than the best performances I 
obtained with 1.5.1 : 1650 req/s compared to 4800 req/s. This is just 
one third of the previous result !

So where do we eat ms ?

Mainly in the new API, which is good (as it's not optimized at all). The 
main issue is the way we treat ObjectClasses. There are overkilling 
tasks done more than once, for instance when we are adding a new 
ObjectClass. The ancestor is computed many times, and we are creating 
far too many arrays and Lists. This has to be changed.
FYI, The ServerEntry.put( ServerAttrinute[] ) method cost 40% of the 
time. Far too much ! (The inner method addObjectClass( ObjectClass ) is 
responsible for this.

The DefaultOidRegistry.getOid( String ) is called more than 1 million 
times when doing around 2500 searches, for a global cost which represent 
33% of the server time. This has to be improved too. This is due to the 
fact that we transform the String to a lower cased string almost for 
every call, when we should deal with OIDs. As such elements should have 
been normalized very soon in the chain, this should not happen.

Last, not least, we have a lot of calls to the 
ServerEntryUtils.toServerEntry() method, which is used to transform an 
Attributes to a ServerEntry (as we are still storing Attributes in the 
backend. This is terribly costly (30% of the server time). This will 
disappear as soon as the backend will handle ServerEntry instead of 
attributes.

Hopefully, we will be able to fix those points very quickly.

Remember that this is just a preliminary check, FYI.

-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org

[ServerEntry] First performance tests take 4

[Emmanuel Lecharny <el...@gmail.com>]

Some few "minor" improvement permitted some gain : up to 3170 req/s. 
(remember we ware at 1650 req/s yesturday, with a target at 4800 req/s). 
This is twice faster with a few fixes.

We are closing the gap with the 1.5.1 version, and we still have a lot 
of work to do to suppress some duplication of work done.

Some insights :
- MINA counts for 5% of all the time.
- ASN.1 decoding represents 7%, but mainly due to DN parsing (70% out of 
it).
- ASN.1 encoding counts for 3%
- DN parsing represents 15% of the total time. This is because we parse 
far too many times the DNs. As soon as we will store serialized versions 
of DNs into the backend, we will be able to divide by at least 2 this time.
- The SearchRequest handling costs 87% of the time, of which you should 
deduce the DN parsing (10%)

Much more intersting :
- CollectiveAttribute filtering represents around 30% of all the time. 
Pretty costly for a feature almost nobody use !

Those informations just tell that we have many places we can dig to 
improve the server :)

-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org

Re: [ServerEntry] First performance tests take 3...

[Howard Chu <hy...@symas.com>]

Emmanuel Lecharny wrote:
> Some more  !
>
> Another easy improvement, I now get 2950 req/s.
>
> As usual, but this is sometime important to insist : "First, write
> correct code, then improve it" .

> "Premature optimization is the root of all evil". D. Knuth.

No disagreement here!  ;)
http://www.openldap.org/devel/programming.html

Though sometimes, when you headed off in the wrong direction in the first 
place, incremental development won't be enough to bring you back. (Referring 
to my latest warning on openldap-devel...)
-- 
   -- Howard Chu
   Chief Architect, Symas Corp.  http://www.symas.com
   Director, Highland Sun        http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP     http://www.openldap.org/project/

[ServerEntry] First performance tests take 3...

[Emmanuel Lecharny <el...@gmail.com>]

Some more  !

Another easy improvement, I now get 2950 req/s.

As usual, but this is sometime important to insist : "First, write 
correct code, then improve it" .

"Premature optimization is the root of all evil". D. Knuth.

-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org

Re: [ServerEntry] First performance tests take 2...

[Emmanuel Lecharny <el...@gmail.com>]

Hi again.

I have spent some time to analyse the perf traces, and I found some easy 
improvement (which could have been done in 1.5.1 to...).

I'm now up to 2500 req/s, simply by modifying the way MatchingRules and 
Syntax are fetched (using a static fields spares some useless lookups).

More to come later !

PS : I'm still using my own test : 10000 entries, one bind, 10000 random 
searches done on an indexed attribute, cache set to max. OpenLDAP do 
9500 req/s, for the record. Laptop, centrino 1.7 Ghz, 2 Gm ram.

Emmanuel Lecharny wrote:

-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org