You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Emmanuel Lécharny <el...@gmail.com> on 2014/08/31 12:31:04 UTC
LdifReader speedup
Hi !
just coming back from a week of vacations...
I have spent some time yesterday analysing the peformances of the
BulkLoader. Most of the time is spent in the parsing of entries by the
LdifReader class. 25% is spent in the PrepareString method, and 10% in
the GeneralizedTimeNormalizer.
If we don't call the PrepareString.normalize method, assuming that we
don't have special strings, we save around 18%.
A faster verison of the GeneralizedTimeNormalizer provides a 9% gain (it
has been committed).
At this point, the question is to know if we should call the
PrepareString methods or not. Not calling it provides a huge speedup,
but OTOH, it's an important phase.
I'm still investigating some other speedups.
Re: LdifReader speedup
Posted by Jeff MAURY <je...@jeffmaury.com>.
You can look to my Golo Eclipse based IDE: github.com/golo-lang/gldt
Jeff
On Tue, Sep 2, 2014 at 11:39 AM, Emmanuel Lécharny <el...@gmail.com>
wrote:
> Le 02/09/14 10:29, Jeff MAURY a écrit :
> > If you want to facilitate Eclipse based development for ADS Studio, as a
> > long Tycho user, I would recommend manifest first approach.
> Do you have pointers/links ?
>
>
--
Jeff MAURY
"Legacy code" often differs from its suggested alternative by actually
working and scaling.
- Bjarne Stroustrup
http://www.jeffmaury.com
http://riadiscuss.jeffmaury.com
http://www.twitter.com/jeffmaury
Re: LdifReader speedup
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 02/09/14 10:29, Jeff MAURY a écrit :
> If you want to facilitate Eclipse based development for ADS Studio, as a
> long Tycho user, I would recommend manifest first approach.
Do you have pointers/links ?
Re: LdifReader speedup
Posted by Jeff MAURY <je...@jeffmaury.com>.
If you want to facilitate Eclipse based development for ADS Studio, as a
long Tycho user, I would recommend manifest first approach.
Jeff
On Tue, Sep 2, 2014 at 7:45 AM, Stefan Seelmann <ma...@stefan-seelmann.de>
wrote:
> On 09/01/2014 02:45 PM, Emmanuel Lécharny wrote:
> > I'm currently working on stabilizing the build which is not passing, due
> > to some modifications made here and there (my fault), then the enxt step
> > for me would be to fix Ldap Studio, which badly needs a result. We have
> > modified the configuration schema, which results on some failures when
> > we try to load it, for instance. We also have to check the plugins we
> > use, as the GUI just crashes.
> >
> > Any help in this area would be very welcome.
> >
>
> I started to look into Tycho, looking into the demo and some real-world
> projects, learning if 'pom-first' or 'manifest-first' approach suites
> better for us.
>
> Kind Regards,
> Stefan
>
>
--
Jeff MAURY
"Legacy code" often differs from its suggested alternative by actually
working and scaling.
- Bjarne Stroustrup
http://www.jeffmaury.com
http://riadiscuss.jeffmaury.com
http://www.twitter.com/jeffmaury
Re: LdifReader speedup
Posted by Stefan Seelmann <ma...@stefan-seelmann.de>.
On 09/01/2014 02:45 PM, Emmanuel Lécharny wrote:
> I'm currently working on stabilizing the build which is not passing, due
> to some modifications made here and there (my fault), then the enxt step
> for me would be to fix Ldap Studio, which badly needs a result. We have
> modified the configuration schema, which results on some failures when
> we try to load it, for instance. We also have to check the plugins we
> use, as the GUI just crashes.
>
> Any help in this area would be very welcome.
>
I started to look into Tycho, looking into the demo and some real-world
projects, learning if 'pom-first' or 'manifest-first' approach suites
better for us.
Kind Regards,
Stefan
Re: LdifReader speedup
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 01/09/14 14:45, Emmanuel Lécharny a écrit :
> Le 31/08/14 12:31, Emmanuel Lécharny a écrit :
>> Hi !
>>
>> just coming back from a week of vacations...
>>
>> I have spent some time yesterday analysing the peformances of the
>> BulkLoader. Most of the time is spent in the parsing of entries by the
>> LdifReader class. 25% is spent in the PrepareString method, and 10% in
>> the GeneralizedTimeNormalizer.
>>
>> If we don't call the PrepareString.normalize method, assuming that we
>> don't have special strings, we save around 18%.
>> A faster verison of the GeneralizedTimeNormalizer provides a 9% gain (it
>> has been committed).
>>
>> At this point, the question is to know if we should call the
>> PrepareString methods or not. Not calling it provides a huge speedup,
>> but OTOH, it's an important phase.
>>
>> I'm still investigating some other speedups.
> Ok, I have imrpoved a few methods and classes (Csn.isValid() is now 36
> times faster, GeneralizedTime.normalize() is 5 times faster) and cleaned
> up some code. As a result, we can read entries from a LDIF files 10% faster.
>
> I'm currently working on stabilizing the build which is not passing, due
> to some modifications made here and there (my fault), then the enxt step
> for me would be to fix Ldap Studio, which badly needs a result.
s/result/release/
Re: LdifReader speedup
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 31/08/14 12:31, Emmanuel Lécharny a écrit :
> Hi !
>
> just coming back from a week of vacations...
>
> I have spent some time yesterday analysing the peformances of the
> BulkLoader. Most of the time is spent in the parsing of entries by the
> LdifReader class. 25% is spent in the PrepareString method, and 10% in
> the GeneralizedTimeNormalizer.
>
> If we don't call the PrepareString.normalize method, assuming that we
> don't have special strings, we save around 18%.
> A faster verison of the GeneralizedTimeNormalizer provides a 9% gain (it
> has been committed).
>
> At this point, the question is to know if we should call the
> PrepareString methods or not. Not calling it provides a huge speedup,
> but OTOH, it's an important phase.
>
> I'm still investigating some other speedups.
Ok, I have imrpoved a few methods and classes (Csn.isValid() is now 36
times faster, GeneralizedTime.normalize() is 5 times faster) and cleaned
up some code. As a result, we can read entries from a LDIF files 10% faster.
I'm currently working on stabilizing the build which is not passing, due
to some modifications made here and there (my fault), then the enxt step
for me would be to fix Ldap Studio, which badly needs a result. We have
modified the configuration schema, which results on some failures when
we try to load it, for instance. We also have to check the plugins we
use, as the GUI just crashes.
Any help in this area would be very welcome.