Re: Slow backup with large number of folders in Datastore

[Clay Ferguson <wc...@gmail.com>]

The key thing to avoid is having ANY nodes with a very large number of
children (i "think" this is still an unresolved "challenge" that is not
been "addressed" in Oak). The developers debate about whether it's a
problem or not actually. Regardless, make sure you are both STORING and
PROCESSING in CHUNKS. Try not to hold a single session open for a long time
without saving it, or large reads/writes, before closing again. Means you
need to not only break up your nodes into smaller chunks, but make your
worker thread(s) bite off only limited "chunks" of work at a time to eat
away at the task bit by bit. Much less memory is used, etc, if you do this
approach. (similar to 'buffering')

Best regards,
Clay Ferguson
wclayf@gmail.com


On Thu, Mar 31, 2016 at 10:37 AM, Imtiaz <ma...@free.fr> wrote:

> Hello,
> I'm trying to optimize the backup of my datastore in JackRabbit 2.2.9,
> which
> currently takes too long (>24h).
> I have a very huge number of folders, and about 1 folder/file in my
> Datastore. I'd like to increase the number of files per folder, so that my
> backup could be faster.
> My node/properties organisation is: MM-YYYY/DD/container.
> I saw on the Datastore wiki, on Future Improvements section (last updated
> 2011) :" If possible the number of directories (and maybe files) should be
> reduced to improve performance. "
>
> Is there any news on this (maybe I missed something in the changelogs)? Is
> there a way to decrease the number of folder, or to change the file
> organisation?
>
> Thank you,
>
>
>
>
> --
> View this message in context:
> http://jackrabbit.510166.n4.nabble.com/Slow-backup-with-large-number-of-folders-in-Datastore-tp4663847.html
> Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
>