You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jackrabbit.apache.org by zhonglf <zh...@gmail.com> on 2008/07/18 11:07:21 UTC
Too slow - adding 1,000 node into the repositroy
I am using a single session instance to add a large number of nodes in a
"flat" structure (the nodes are consist of about 7 simple properties
(Strings mostly, and a documents)). At first, adding 1,000 node took 200
seconds, when repository was empty.Until the size of nodes in the repository
over 100,000, adding 1,000 node took more than 10 minutes. Why ? Any ideas
to improve it?
PS: I didn't add all nodes in one parent node, instead of using a node as a
folder for per 200 children nodes.
--
View this message in context: http://www.nabble.com/Too-slow---adding-1%2C000-node-into-the-repositroy-tp18525321p18525321.html
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
Re: Too slow - adding 1,000 node into the repositroy
Posted by zhonglf <zh...@gmail.com>.
Thank you, i will try to config the file data store right now.
Alexander Klimetschek wrote:
>
> Are you using a bundle DB persistence manager (recommended)? And do
> you have binary properties (in which case a DataStore is useful)? Have
> a look at these wiki pages:
>
> http://wiki.apache.org/jackrabbit/PersistenceManagerFAQ
> http://wiki.apache.org/jackrabbit/DataStore
>
> Regards,
> Alex
>
> On Fri, Jul 18, 2008 at 11:07 AM, zhonglf <zh...@gmail.com> wrote:
>>
>> I am using a single session instance to add a large number of nodes in a
>> "flat" structure (the nodes are consist of about 7 simple properties
>> (Strings mostly, and a documents)). At first, adding 1,000 node took 200
>> seconds, when repository was empty.Until the size of nodes in the
>> repository
>> over 100,000, adding 1,000 node took more than 10 minutes. Why ? Any
>> ideas
>> to improve it?
>>
>> PS: I didn't add all nodes in one parent node, instead of using a node as
>> a
>> folder for per 200 children nodes.
>> --
>> View this message in context:
>> http://www.nabble.com/Too-slow---adding-1%2C000-node-into-the-repositroy-tp18525321p18525321.html
>> Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> Alexander Klimetschek
> alexander.klimetschek@day.com
>
>
--
View this message in context: http://www.nabble.com/Too-slow---adding-1%2C000-node-into-the-repositroy-tp18525321p18525622.html
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
Re: Too slow - adding 1,000 node into the repositroy
Posted by zhonglf <zh...@gmail.com>.
Alexander Klimetschek wrote:
>
> Are you using a bundle DB persistence manager (recommended)? And do
> you have binary properties (in which case a DataStore is useful)? Have
> a look at these wiki pages:
>
> http://wiki.apache.org/jackrabbit/PersistenceManagerFAQ
> http://wiki.apache.org/jackrabbit/DataStore
>
> Regards,
> Alex
>
> --
> Alexander Klimetschek
> alexander.klimetschek@day.com
>
>
Hi, Alex, i hava done both of them, but no much better :(
--
View this message in context: http://www.nabble.com/Too-slow---adding-1%2C000-node-into-the-repositroy-tp18525321p18565020.html
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
Re: Too slow - adding 1,000 node into the repositroy
Posted by Alexander Klimetschek <ak...@day.com>.
Are you using a bundle DB persistence manager (recommended)? And do
you have binary properties (in which case a DataStore is useful)? Have
a look at these wiki pages:
http://wiki.apache.org/jackrabbit/PersistenceManagerFAQ
http://wiki.apache.org/jackrabbit/DataStore
Regards,
Alex
On Fri, Jul 18, 2008 at 11:07 AM, zhonglf <zh...@gmail.com> wrote:
>
> I am using a single session instance to add a large number of nodes in a
> "flat" structure (the nodes are consist of about 7 simple properties
> (Strings mostly, and a documents)). At first, adding 1,000 node took 200
> seconds, when repository was empty.Until the size of nodes in the repository
> over 100,000, adding 1,000 node took more than 10 minutes. Why ? Any ideas
> to improve it?
>
> PS: I didn't add all nodes in one parent node, instead of using a node as a
> folder for per 200 children nodes.
> --
> View this message in context: http://www.nabble.com/Too-slow---adding-1%2C000-node-into-the-repositroy-tp18525321p18525321.html
> Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
>
>
--
Alexander Klimetschek
alexander.klimetschek@day.com
Re: Too slow - adding 1,000 node into the repositroy
Posted by Stefan Guggisberg <st...@day.com>.
On Mon, Jul 21, 2008 at 11:20 AM, zhonglf <zh...@gmail.com> wrote:
>
>
>
> Stefan Guggisberg-2 wrote:
>>
>> jackrabbit is not optimized for very flat node hierarchies (e.g. with more
>> than
>> 30k child nodes per node). if you create a deeper structure you'll
>> experience a
>> lot better performance.
>>
>> anyway, your figures seem to be *very* high (200s for 1k nodes)...
>> are you using any of the foillowing?
>> - references
>> - versioning
>> - large binaries
>>
>> cheers
>> stefan
>>
>>
>
> yes, all of them. Is this the reason?
probably yes. performance WRT large binaries should be significantly
improved by using the DataStore.
referential integrity and versioning are very powerful features but they
come at a certain cost. make sure you're only using them where needed.
other factors that might have a great impact on performance is the choice
of persistence manager and backend. bunde db persistence manager in
combination with embedded derby currently gets you the best performance.
if you change the persistence manager configuration make sure
modify the workspace.xml as well and not just the repository.xml.
cheers
stefan
>
>
> --
> View this message in context: http://www.nabble.com/Too-slow---adding-1%2C000-node-into-the-repositroy-tp18525321p18564929.html
> Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
>
>
Re: Too slow - adding 1,000 node into the repositroy
Posted by zhonglf <zh...@gmail.com>.
Stefan Guggisberg-2 wrote:
>
> jackrabbit is not optimized for very flat node hierarchies (e.g. with more
> than
> 30k child nodes per node). if you create a deeper structure you'll
> experience a
> lot better performance.
>
> anyway, your figures seem to be *very* high (200s for 1k nodes)...
> are you using any of the foillowing?
> - references
> - versioning
> - large binaries
>
> cheers
> stefan
>
>
yes, all of them. Is this the reason?
--
View this message in context: http://www.nabble.com/Too-slow---adding-1%2C000-node-into-the-repositroy-tp18525321p18564929.html
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
Re: Too slow - adding 1,000 node into the repositroy
Posted by Stefan Guggisberg <st...@day.com>.
On Fri, Jul 18, 2008 at 11:07 AM, zhonglf <zh...@gmail.com> wrote:
>
> I am using a single session instance to add a large number of nodes in a
> "flat" structure (the nodes are consist of about 7 simple properties
> (Strings mostly, and a documents)). At first, adding 1,000 node took 200
> seconds, when repository was empty.Until the size of nodes in the repository
> over 100,000, adding 1,000 node took more than 10 minutes. Why ? Any ideas
> to improve it?
jackrabbit is not optimized for very flat node hierarchies (e.g. with more than
30k child nodes per node). if you create a deeper structure you'll experience a
lot better performance.
anyway, your figures seem to be *very* high (200s for 1k nodes)...
are you using any of the foillowing?
- references
- versioning
- large binaries
cheers
stefan
>
> PS: I didn't add all nodes in one parent node, instead of using a node as a
> folder for per 200 children nodes.
> --
> View this message in context: http://www.nabble.com/Too-slow---adding-1%2C000-node-into-the-repositroy-tp18525321p18525321.html
> Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
>
>