You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by "Emmanuel Lecharny (JIRA)" <di...@incubator.apache.org> on 2006/03/16 00:44:58 UTC
[jira] Created: (DIR-182) Improvment in the way DN are treated
Improvment in the way DN are treated
------------------------------------
Key: DIR-182
URL: http://issues.apache.org/jira/browse/DIR-182
Project: Directory
Type: Improvement
Reporter: Emmanuel Lecharny
We have mesured that DN parsing is the hot spot in the server : we spend around 80 % of server time to parse DNs.
After having studied the way BindRequest does handle DN, we can say that for a single bind for the user "uid=admin,ou=system", we have more than 40 parsing done. Here is a list of all the parsing in the DnComparator.compare() method :
(The '*' in front of each line shows where a parsing of both DN is done)
* Compare 'ou=groups,ou=system' and 'uid=admin,ou=system'
* Compare 'prefnodename=sysPrefRoot,ou=system' and 'uid=admin,ou=system'
Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
* Compare 'ou=groups,ou=system' and 'uid=admin,ou=system'
* Compare 'prefnodename=sysPrefRoot,ou=system' and 'uid=admin,ou=system'
Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
* Compare 'ou=groups,ou=system' and 'ou=system'
* Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
* Compare 'ou=configuration,ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
* Compare 'ou=groups,ou=system' and 'ou=system'
* Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
* Compare 'ou=configuration,ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
* Compare 'ou=groups,ou=system' and 'ou=system'
* Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
* Compare 'ou=configuration,ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
* Compare 'ou=groups,ou=system' and 'ou=system'
* Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
* Compare 'ou=configuration,ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
* Compare 'ou=groups,ou=system' and 'ou=system'
* Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
* Compare 'ou=configuration,ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
If we can change this compare function to work with Name instead of String we could have a huge improvment : more that 35x faster
(I wrote a micro-bench to mesure the difference : around 36 times faster).
As this is not the single place where we can gain time (we have a lot or String -> DN and DN -> String conversions during the Bind process), we would be able to handle Bind Request much much faster, too (being aware that we have a lot of other costly operaions that will nor allow a x40 improvment).
My personnal guess is that we should be able to handle around 5000 binds per second against the current 300, but this is just a guess :)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
Re: [jira] Created: (DIR-182) Improvment in the way DN are treated
Posted by Alex Karasulu <ao...@bellsouth.net>.
Emmanuel,
Not a big deal but can you move this issue to DIRAPACHEDS? Were we not
going to use DIR for general issues for the TLP rather than for product
related issues?
Alex
Emmanuel Lecharny (JIRA) wrote:
> Improvment in the way DN are treated
> ------------------------------------
>
> Key: DIR-182
> URL: http://issues.apache.org/jira/browse/DIR-182
> Project: Directory
> Type: Improvement
> Reporter: Emmanuel Lecharny
>
>
> We have mesured that DN parsing is the hot spot in the server : we spend around 80 % of server time to parse DNs.
>
> After having studied the way BindRequest does handle DN, we can say that for a single bind for the user "uid=admin,ou=system", we have more than 40 parsing done. Here is a list of all the parsing in the DnComparator.compare() method :
> (The '*' in front of each line shows where a parsing of both DN is done)
> * Compare 'ou=groups,ou=system' and 'uid=admin,ou=system'
> * Compare 'prefnodename=sysPrefRoot,ou=system' and 'uid=admin,ou=system'
> Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
> Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
> * Compare 'ou=groups,ou=system' and 'uid=admin,ou=system'
> * Compare 'prefnodename=sysPrefRoot,ou=system' and 'uid=admin,ou=system'
> Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
> Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
>
> If we can change this compare function to work with Name instead of String we could have a huge improvment : more that 35x faster
> (I wrote a micro-bench to mesure the difference : around 36 times faster).
>
> As this is not the single place where we can gain time (we have a lot or String -> DN and DN -> String conversions during the Bind process), we would be able to handle Bind Request much much faster, too (being aware that we have a lot of other costly operaions that will nor allow a x40 improvment).
>
> My personnal guess is that we should be able to handle around 5000 binds per second against the current 300, but this is just a guess :)
>
>
>
[jira] Commented: (DIR-182) Improvment in the way DN are treated
Posted by "Emmanuel Lecharny (JIRA)" <di...@incubator.apache.org>.
[ http://issues.apache.org/jira/browse/DIR-182?page=comments#action_12370651 ]
Emmanuel Lecharny commented on DIR-182:
---------------------------------------
To be checked, but replacing :
...
if ( ( ( String ) obj1 ).compareTo( ( String ) obj2 ) == 0 )
{
return 0;
}
else
{
try
{
synchronized ( parserMutex )
{
dn1 = parser.parse( ( String ) obj1 );
}
}
catch ( NamingException ne )
{
throw new IllegalArgumentException( "first argument (" + obj1 + ") was not a distinguished name" );
}
}
...
by
...
return -( ( String ) obj1 ).compareTo( ( String ) obj2 );
...
in org.apache.directory.shared.ldap.schema.DnComparator.compare( Object o1, Object o2) method seems to be enough to really boost ApachedDS performance ! (Note the negation in front of the string comparizon, because DN are compared in the reverse order)
It has te be seriously checked though...
> Improvment in the way DN are treated
> ------------------------------------
>
> Key: DIR-182
> URL: http://issues.apache.org/jira/browse/DIR-182
> Project: Directory
> Type: Improvement
> Reporter: Emmanuel Lecharny
>
> We have mesured that DN parsing is the hot spot in the server : we spend around 80 % of server time to parse DNs.
> After having studied the way BindRequest does handle DN, we can say that for a single bind for the user "uid=admin,ou=system", we have more than 40 parsing done. Here is a list of all the parsing in the DnComparator.compare() method :
> (The '*' in front of each line shows where a parsing of both DN is done)
> * Compare 'ou=groups,ou=system' and 'uid=admin,ou=system'
> * Compare 'prefnodename=sysPrefRoot,ou=system' and 'uid=admin,ou=system'
> Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
> Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
> * Compare 'ou=groups,ou=system' and 'uid=admin,ou=system'
> * Compare 'prefnodename=sysPrefRoot,ou=system' and 'uid=admin,ou=system'
> Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
> Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> If we can change this compare function to work with Name instead of String we could have a huge improvment : more that 35x faster
> (I wrote a micro-bench to mesure the difference : around 36 times faster).
> As this is not the single place where we can gain time (we have a lot or String -> DN and DN -> String conversions during the Bind process), we would be able to handle Bind Request much much faster, too (being aware that we have a lot of other costly operaions that will nor allow a x40 improvment).
> My personnal guess is that we should be able to handle around 5000 binds per second against the current 300, but this is just a guess :)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (DIR-182) Improvment in the way DN are treated
Posted by "Emmanuel Lecharny (JIRA)" <di...@incubator.apache.org>.
[ http://issues.apache.org/jira/browse/DIR-182?page=comments#action_12370650 ]
Emmanuel Lecharny commented on DIR-182:
---------------------------------------
A better solution would be to do a String comparizon into the DnComparator. It makes sense as the DN has already been normalized before being stored into the backend.
The performance increase is dramatic : for a bind request, the comparizon cost will be done around 670 times faster !
This approach has been already partially implemented month ago and is effective when two DN are equals : the server speed has been increased and is now 4 times faster than it was. In the previous list, all the DN that does not have a '*' are compared as strings. We just have to do the same thing for other elements.
> Improvment in the way DN are treated
> ------------------------------------
>
> Key: DIR-182
> URL: http://issues.apache.org/jira/browse/DIR-182
> Project: Directory
> Type: Improvement
> Reporter: Emmanuel Lecharny
>
> We have mesured that DN parsing is the hot spot in the server : we spend around 80 % of server time to parse DNs.
> After having studied the way BindRequest does handle DN, we can say that for a single bind for the user "uid=admin,ou=system", we have more than 40 parsing done. Here is a list of all the parsing in the DnComparator.compare() method :
> (The '*' in front of each line shows where a parsing of both DN is done)
> * Compare 'ou=groups,ou=system' and 'uid=admin,ou=system'
> * Compare 'prefnodename=sysPrefRoot,ou=system' and 'uid=admin,ou=system'
> Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
> Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
> * Compare 'ou=groups,ou=system' and 'uid=admin,ou=system'
> * Compare 'prefnodename=sysPrefRoot,ou=system' and 'uid=admin,ou=system'
> Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
> Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> If we can change this compare function to work with Name instead of String we could have a huge improvment : more that 35x faster
> (I wrote a micro-bench to mesure the difference : around 36 times faster).
> As this is not the single place where we can gain time (we have a lot or String -> DN and DN -> String conversions during the Bind process), we would be able to handle Bind Request much much faster, too (being aware that we have a lot of other costly operaions that will nor allow a x40 improvment).
> My personnal guess is that we should be able to handle around 5000 binds per second against the current 300, but this is just a guess :)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Closed: (DIRSERVER-600) Improvment in the way DN are treated
Posted by "Emmanuel Lecharny (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/DIRSERVER-600?page=all ]
Emmanuel Lecharny closed DIRSERVER-600.
---------------------------------------
Resolution: Fixed
Has been modified and fixed a while ago !
> Improvment in the way DN are treated
> ------------------------------------
>
> Key: DIRSERVER-600
> URL: http://issues.apache.org/jira/browse/DIRSERVER-600
> Project: Directory ApacheDS
> Issue Type: Improvement
> Reporter: Emmanuel Lecharny
>
> We have mesured that DN parsing is the hot spot in the server : we spend around 80 % of server time to parse DNs.
> After having studied the way BindRequest does handle DN, we can say that for a single bind for the user "uid=admin,ou=system", we have more than 40 parsing done. Here is a list of all the parsing in the DnComparator.compare() method :
> (The '*' in front of each line shows where a parsing of both DN is done)
> * Compare 'ou=groups,ou=system' and 'uid=admin,ou=system'
> * Compare 'prefnodename=sysPrefRoot,ou=system' and 'uid=admin,ou=system'
> Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
> Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
> * Compare 'ou=groups,ou=system' and 'uid=admin,ou=system'
> * Compare 'prefnodename=sysPrefRoot,ou=system' and 'uid=admin,ou=system'
> Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
> Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> * Compare 'ou=groups,ou=system' and 'ou=system'
> * Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
> * Compare 'ou=configuration,ou=system' and 'ou=system'
> Compare 'ou=system' and 'ou=system'
> If we can change this compare function to work with Name instead of String we could have a huge improvment : more that 35x faster
> (I wrote a micro-bench to mesure the difference : around 36 times faster).
> As this is not the single place where we can gain time (we have a lot or String -> DN and DN -> String conversions during the Bind process), we would be able to handle Bind Request much much faster, too (being aware that we have a lot of other costly operaions that will nor allow a x40 improvment).
> My personnal guess is that we should be able to handle around 5000 binds per second against the current 300, but this is just a guess :)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira