[jira] Updated: (JCR-937) CacheManager max memory size

["Martijn Hendriks (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/JCR-937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Martijn Hendriks updated JCR-937:
---------------------------------

    Attachment: CacheManagerTest.java

Currently, the estimation of the sizes of the caches that the CacheManager manages is not very accurate. The sizes of properties are estimated in PropertyState.calculateMemoryFootprint() to be 350 + values.length * 100. This is really bad when you have a lot of non-binary properties that are significantly larger than 100 bytes.

The attached unit test gives a testcase with only one concurrent session which is closed properly. Running this test (-Xmx=512M) and checking a heap-dump triggered by an OutOfMemoryError with YourKit shows the following:
(1) The CacheManager lets the cache of the SISM grow way beyond the default max total cache size of 16 MB, namely to 157 MB.
(2) Lucene classes have a retained size of approx 323 MB.

Is the Lucene memory consumption due to the extremely long String properties, or has it to do with the test setup?

I'd like to reopen this issue, but I don't see a button for it.

> CacheManager max memory size
> ----------------------------
>
>                 Key: JCR-937
>                 URL: https://issues.apache.org/jira/browse/JCR-937
>             Project: Jackrabbit
>          Issue Type: Bug
>          Components: jackrabbit-core
>    Affects Versions: 1.3
>            Reporter: Xiaohua Lu
>            Assignee: Thomas Mueller
>            Priority: Minor
>         Attachments: CacheManagerTest.java
>
>
> I have ran into OutOfMemory a couple of times with Jackrabbit (cluster, 4 nodes, each has 1G mem heap size). 
> After adding some debug into the CacheManager, I noticed that maxMemorySize (default to 16M) is not really honored during resizeAll check.  Each individual MLRUItemStateCache seems to honor the size, but the total number/size of MLRUItemStateCache is not. If you put some print statement of totalMemoryUsed and unusedMemory, you can see that totalMemoryUsed is more than 16M and unusedMemory is negative. 
> The other problem we have noticed during the profiling is there are a lots of other in memory objects that are consuming memory but not included in CacheManager caches control. One example is CachingHierarchyManager which consumed 58M out of 242M through its use of PathMap. If CacheManager's maxSize can control the total cache size used by Jackrabbit, that would be easier from a management's perspective. (btw, upper_limit in CachingHierarchyManager is hardcoded and can't be control from outside)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.