[jira] Commented: (JCR-1037) Memory leak causing performance problems

["Antonio Carballo (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/JCR-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12517349 ] 

Antonio Carballo commented on JCR-1037:
---------------------------------------

We downloaded v1.3.1 sources and after debugging the CacheManager class we can conclusively say that the "caching" algorithm is the problem. Our trace (see attachment) shows that the CacheManager keeps growing its pool and memory consumption as more files are pushed into the system. This problem was also reported in JCR-937 incident report back in May. Our conclusions about this problem agree with the incident's reporter.

We advice that this class be modified to perform 'pruning' operations on the pool to get rid of Cache objects. This will then restrain the pool's growth to the one defined by the "maxMemory" variable.

So, what's the problem? Simple. The CacheManager class only trims the memory within the Cache objects. It does not get rid of any Cache objects. Thus, the pool of Cache objects keeps growing and growing and growing. No attempt is being made to limit the pool's memory usage (combined Cache objects) to its maximum ("maxMemory").

What's the solution? Simple. Implement a true LRU or ACR algorithm to truly manage the pool and force the upper level code to retrieve the Cache object from its pool (CacheManager's). If the upper level classes are mismanaging the Cache objects, this bug will not be fixed as outlined in this paragraph. Is the MLRUItemStateCache

This bug is a showstopper to us due to its effect on performance during heavy loads. If you have any questions regarding our research, please feel free to contact us.

Sincerely,

/Antonio C.


> Memory leak causing performance problems
> ----------------------------------------
>
>                 Key: JCR-1037
>                 URL: https://issues.apache.org/jira/browse/JCR-1037
>             Project: Jackrabbit
>          Issue Type: Bug
>          Components: Jackrabbit API
>    Affects Versions: 1.2.1, 1.2.2, 1.2.3, 1.3
>         Environment: Tomcat 6.0, XP Pro w/1Gb
>            Reporter: Antonio Carballo
>
> Folks,
> We have been running tests on JCR v1.3 and v1.2.1 for the past two weeks. The system keeps running out of memory after X number of documents are added. Our initial test consisted of about 50 documents and gradually increased to about 150 documents. The size of the documents ranged from 1K to 9MB. We later changed the test to consist of files with less than 1K in length with the same result. Increasing the heap size delays the error but the outcome is always the same (Servlet runs out of heap memory.)
> Using JProbe we found a high number of references created by the caching sub-system (SessionItemStateManager.java, SharedItemStateManager.java, LocalItemStateManager.java).  We changed the caching parameters using CacheManager (min 64K - max 16MB). This change only delayed the error. Servlet eventually runs out of heap memory.
> We are more than happy to share our findings (even source code and test data) with the Jackrabbit team. Please let us know how you wish to proceed.
> Sincerely,
> Antonio Carballo

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.