[ServerEntry] Next step

[Emmanuel Lecharny <el...@apache.org>]

Hi guys,

I have finished to modify the code to get the ServerEntry serialized 
into the backend. It was not really easy, but it's done. It also has 
some impact :
- I had to modify JDBM in order to be able to deserialize the entries 
(just added an accessor to a class)

A mail has been sent to Alex Boivert to see if he can provide a new 
version for this jar (we are currently using 1.0.0) and he replied that 
he will try to find some time in a very busy schedule to deliver a 
1.0.1, which will have to be deployed to a maven repo. Another coiple of 
days...

Here is what I suggest at this point : I have created a sub-project 
(apacheds-jdbm) containing the JDBM code with almost all of the tests 
(except the performance tests), and it works pretty well. I will commit 
this code and my modification in the bigbang branch, as I have to move 
on to the next step : removing the JNDI layer.

The reason why I'm impatient to do that is that I have a 13 000 lines 
long diff pending, and I'm afraid to do some modifications on the server 
with around 100 impacted files, as it may break the currently working 
server...

FYI, I have run some perf test on my laptop, and here are the compared 
results for 1.5.2 and bigband (as always, local test, not meaningfull, 
blah blah, for your eyes only :)

1.5.2       : 3506 random search req/s
bigbang : 4296 random search req/s

I also profiled the server and found interesting numbers :

LdapDN handling : 37.6% of the CPU (out of which around 18% can be 
squeeze when the JNDI layer will have been removed)

ServerEntry handling : 19.60% of the CPU

ASN.1 codec : 10,15%

This is 67,4% of all the consumed CPU in tasks which are difficult to 
improve (if we except the 18% we can squeeze from some useless double 
LdapDN parsing), the rest is spreaded in small methods.

A blind guess is that it will be quite challenging to gain a lot now. My 
own expectation is that we should be able to go up to 6000 searches 
req/s on my laptop, but then, we will have to check the LdapDN parsing 
again.

It would be very cool if someone can double check the ServerEntry clone 
method to see if it's optimal, and if we can improve its code (or even 
find some bug or bad logic in it).

I was also thinking that some lazzy cloning might be enough (ie, cloning 
only the attribute container, not the attribute values themselves, as in 
many case, we are just removing attributes from the entry before 
returning it, keeping the attributes unmodified. That would save a few %)

That's it, I'm now waiting for Eris to be up and running...

thanks for your attention !

-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org

Re: [ServerEntry] Next step

[Alex Karasulu <ak...@apache.org>]

Thanks for the update Emmanuel.  I would agree with committing it all even
though we live with a fork for just a few days.

Alex

On Wed, Apr 30, 2008 at 7:24 AM, Emmanuel Lecharny <el...@apache.org>
wrote:

> Hi guys,
>
> I have finished to modify the code to get the ServerEntry serialized into
> the backend. It was not really easy, but it's done. It also has some impact
> :
> - I had to modify JDBM in order to be able to deserialize the entries
> (just added an accessor to a class)
>
> A mail has been sent to Alex Boivert to see if he can provide a new
> version for this jar (we are currently using 1.0.0) and he replied that he
> will try to find some time in a very busy schedule to deliver a 1.0.1, which
> will have to be deployed to a maven repo. Another coiple of days...
>
> Here is what I suggest at this point : I have created a sub-project
> (apacheds-jdbm) containing the JDBM code with almost all of the tests
> (except the performance tests), and it works pretty well. I will commit this
> code and my modification in the bigbang branch, as I have to move on to the
> next step : removing the JNDI layer.
>
> The reason why I'm impatient to do that is that I have a 13 000 lines long
> diff pending, and I'm afraid to do some modifications on the server with
> around 100 impacted files, as it may break the currently working server...
>
> FYI, I have run some perf test on my laptop, and here are the compared
> results for 1.5.2 and bigband (as always, local test, not meaningfull, blah
> blah, for your eyes only :)
>
> 1.5.2       : 3506 random search req/s
> bigbang : 4296 random search req/s
>
> I also profiled the server and found interesting numbers :
>
> LdapDN handling : 37.6% of the CPU (out of which around 18% can be squeeze
> when the JNDI layer will have been removed)
>
> ServerEntry handling : 19.60% of the CPU
>
> ASN.1 codec : 10,15%
>
> This is 67,4% of all the consumed CPU in tasks which are difficult to
> improve (if we except the 18% we can squeeze from some useless double LdapDN
> parsing), the rest is spreaded in small methods.
>
> A blind guess is that it will be quite challenging to gain a lot now. My
> own expectation is that we should be able to go up to 6000 searches req/s on
> my laptop, but then, we will have to check the LdapDN parsing again.
>
> It would be very cool if someone can double check the ServerEntry clone
> method to see if it's optimal, and if we can improve its code (or even find
> some bug or bad logic in it).
>
> I was also thinking that some lazzy cloning might be enough (ie, cloning
> only the attribute container, not the attribute values themselves, as in
> many case, we are just removing attributes from the entry before returning
> it, keeping the attributes unmodified. That would save a few %)
>
> That's it, I'm now waiting for Eris to be up and running...
>
> thanks for your attention !
>
> --
> --
> cordialement, regards,
> Emmanuel Lécharny
> www.iktek.com
> directory.apache.org
>
>
>