ApacheDS 64-bit Heap Limit

[Hakan Çelik <ce...@gmail.com>]

Hi team,

We're testing ApacheDS for storing our ~500k customer base.

We couldn't go beyond 4GB heap size for ApacheDS (version 2.0 M21). 4GB was
not enough for our data, frequent GCs were scaring us for a possible memory
related shutdown. Btw backend became corrupted one time because of a
permgen space issue. We had to start from scratch :(  So an unexpected
shutdown can be really really messy.

Anyway, i've checked Tanuki wrapper release notes, i saw they've removed
4GB limit as of 3.3.0 version. (
https://wrapper.tanukisoftware.com/doc/english/release-notes.html#3.3.0)

As a solution i've upgraded my ApacheDS instance's *lib/libwrapper.so*,
*lib/wrapper.jar*, *bin/wrapper* to the latest (3.5.29 - community),
removed old wrapper jar, kept wrapper configs as is.. It looks like working
fine now.

Please feel free to give any feedbacks/warnings about this. And kindly, you
may consider updating this dependency version to latest for future releases.

Regards,
Hakan

Re: ApacheDS 64-bit Heap Limit

[Emmanuel Lécharny <el...@gmail.com>]

Le 01/08/16 � 10:20, Hakan �elik a �crit :
> Hi team,

Hi Hakan,
>
> We're testing ApacheDS for storing our ~500k customer base.
whoa ! It should have take hours to inject all of them ! FTR, we have
worked on a bulk loader, but only with the Mavibot partition, which is
not yet the default partition. That will allow you to inject data *way*
faster (we are ttalking orders of magnitude fastern here...).
>
> We couldn't go beyond 4GB heap size for ApacheDS (version 2.0 M21). 4GB was
> not enough for our data, frequent GCs were scaring us for a possible memory
> related shutdown. Btw backend became corrupted one time because of a
> permgen space issue. We had to start from scratch :(  So an unexpected
> shutdown can be really really messy.
Indeed. In 2.0-M23 we have added a 'repair' mode that can be used to fix
this corruption problem (you just have to use 'repair' instead of
'start' when launching the server, with the path to the partition to repair)

But the long term for us is to re-place the backend (JDBM) with Mavibot,
somethinhg we are working on for now 2 years, but it requires one or 2
months of work to complete, and we don't have them atm :/ (volunteer
based work is the pb here : would we have some financement to get it
done, that would be a game changer !)

>
> Anyway, i've checked Tanuki wrapper release notes, i saw they've removed
> 4GB limit as of 3.3.0 version. (
> https://wrapper.tanukisoftware.com/doc/english/release-notes.html#3.3.0)

three things :
- I think you can go beyond this limit in linux boxes. Are you usng
windows ?
- We intend to switch to another wrapper, like YAJSW, which does not
have the same limits
- You can also use teh default installer, which does not depend on the
Tanuki wrapper.

>
> As a solution i've upgraded my ApacheDS instance's *lib/libwrapper.so*,
> *lib/wrapper.jar*, *bin/wrapper* to the latest (3.5.29 - community),
> removed old wrapper jar, kept wrapper configs as is.. It looks like working
> fine now.

That will work. Sadly, we can't use this version, because it license
(GPL 2/3) is not compatible with The ASF AL 2.0 icense :/
>
> Please feel free to give any feedbacks/warnings about this. And kindly, you
> may consider updating this dependency version to latest for future releases.
Not option :/

Thanks for the feedback, and keep following the project, because the
switch to mavibot will solve a *lot* of the issues you are facing !