[jira] [Commented] (OAK-10147) Many move operations may consume a lot of memory

["Marcel Reutegger (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-10147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17722139#comment-17722139 ] 

Marcel Reutegger commented on OAK-10147:
----------------------------------------

Added an article via [#935|https://github.com/apache/jackrabbit-oak/pull/935] that explains how move operations are implemented.

https://jackrabbit.apache.org/oak/docs/architecture/move.html

I'm also familiar with cases where memory usage was high and move operations were involved. Please note, memory usage may be high even when changes are saved periodically. The problem are references to Nodes acquired before move operations happen and then those Nodes are not accessed. In one case, application code was traversing the repository through Sling API and a series of chained and wrapped iterators that returned Sling Resources backed by JCR Nodes. Traversal was depth first, but also pre-fetched the next Node at each level. While traversing, the application moved nodes and periodically called Session.save(). The pre-fetched node under the root of the traversal start then accumulated memory for all those move operations because it was not accessed until late in the entire traversal.

The behaviour is independent of the NodeStore implementation in use. Though, there are certainly differences how much memory is retained by NodeBuilder and NodeState implementations.

> Many move operations may consume a lot of memory
> ------------------------------------------------
>
>                 Key: OAK-10147
>                 URL: https://issues.apache.org/jira/browse/OAK-10147
>             Project: Jackrabbit Oak
>          Issue Type: Task
>          Components: core
>            Reporter: Julian Reschke
>            Assignee: Marcel Reutegger
>            Priority: Major
>
> We have encountered an issue where a component did a huge number of move operations in transient space, and oak-core's "MutableRoot" class consumed ~800 MB for ca 3000 move operations. This seems to be a lot.
> We should:
> 1. Write a test / benchmark that reproduces the issue
> 2. Check whether it's specific to a certain store implemention
> 3. See whether the memory footprint can be reduced



--
This message was sent by Atlassian Jira
(v8.20.10#820010)