You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Laxman (JIRA)" <ji...@apache.org> on 2010/02/27 10:42:05 UTC
[jira] Created: (SOLR-1798) Memory leak in FastLUCache
Memory leak in FastLUCache
--------------------------
Key: SOLR-1798
URL: https://issues.apache.org/jira/browse/SOLR-1798
Project: Solr
Issue Type: Bug
Components: search
Affects Versions: 1.4, 1.5
Reporter: Laxman
Fix For: 1.5
Every time a commit happens two Stats instances [org.apache.solr.common.util.ConcurrentLRUCache.Stats] are leaking.
Following code to maintain cumulative cache statistics causing this Stats object leak.
{noformat}
cumulativeStats = (List<ConcurrentLRUCache.Stats>) persistence;
cumulativeStats.add(cache.getStats());
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (SOLR-1798) Memory leak in FastLRUCache
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley reassigned SOLR-1798:
----------------------------------
Assignee: Yonik Seeley
> Memory leak in FastLRUCache
> ---------------------------
>
> Key: SOLR-1798
> URL: https://issues.apache.org/jira/browse/SOLR-1798
> Project: Solr
> Issue Type: Bug
> Components: search
> Affects Versions: 1.4, 1.5
> Reporter: Laxman
> Assignee: Yonik Seeley
> Fix For: 1.5
>
>
> Every time a commit happens two Stats instances [org.apache.solr.common.util.ConcurrentLRUCache.Stats] are leaking.
> Following code [org.apache.solr.search.FastLRUCache] to maintain cumulative cache statistics causing this Stats object leak.
> {noformat}
> cumulativeStats = (List<ConcurrentLRUCache.Stats>) persistence;
> cumulativeStats.add(cache.getStats());
> {noformat}
> Everytime a *commit* happens a new cache object is getting created and its stats is added to the list which is not released at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1798) Memory leak in FastLRUCache
Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842380#action_12842380 ]
Noble Paul commented on SOLR-1798:
----------------------------------
bq.do you remember why a List was used here?
It was used to collect the cumulative data. I guess the original LRU cache was doing it the same way. It was a wrong decision to make
> Memory leak in FastLRUCache
> ---------------------------
>
> Key: SOLR-1798
> URL: https://issues.apache.org/jira/browse/SOLR-1798
> Project: Solr
> Issue Type: Bug
> Components: search
> Affects Versions: 1.4, 1.5
> Reporter: Laxman
> Assignee: Yonik Seeley
> Fix For: 1.5
>
> Attachments: SOLR-1798.patch, SOLR-1798.patch
>
>
> Every time a commit happens two Stats instances [org.apache.solr.common.util.ConcurrentLRUCache.Stats] are leaking.
> Following code [org.apache.solr.search.FastLRUCache] to maintain cumulative cache statistics causing this Stats object leak.
> {noformat}
> cumulativeStats = (List<ConcurrentLRUCache.Stats>) persistence;
> cumulativeStats.add(cache.getStats());
> {noformat}
> Everytime a *commit* happens a new cache object is getting created and its stats is added to the list which is not released at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1798) Memory leak in FastLRUCache
Posted by "Laxman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Laxman updated SOLR-1798:
-------------------------
Summary: Memory leak in FastLRUCache (was: Memory leak in FastLUCache)
> Memory leak in FastLRUCache
> ---------------------------
>
> Key: SOLR-1798
> URL: https://issues.apache.org/jira/browse/SOLR-1798
> Project: Solr
> Issue Type: Bug
> Components: search
> Affects Versions: 1.4, 1.5
> Reporter: Laxman
> Fix For: 1.5
>
>
> Every time a commit happens two Stats instances [org.apache.solr.common.util.ConcurrentLRUCache.Stats] are leaking.
> Following code [org.apache.solr.search.FastLRUCache] to maintain cumulative cache statistics causing this Stats object leak.
> {noformat}
> cumulativeStats = (List<ConcurrentLRUCache.Stats>) persistence;
> cumulativeStats.add(cache.getStats());
> {noformat}
> Everytime a *commit* happens a new cache object is getting created and its stats is added to the list which is not released at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1798) Memory leak in FastLRUCache
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley updated SOLR-1798:
-------------------------------
Attachment: SOLR-1798.patch
New patch that fixes the cumulative stats by adding in an entry in the list for cumulative stats and incrementing them for every cache that is closed.
Longer term, we might want a hash set or something (if we start having per-segment caches) but more changes will be necessary then anyway.
> Memory leak in FastLRUCache
> ---------------------------
>
> Key: SOLR-1798
> URL: https://issues.apache.org/jira/browse/SOLR-1798
> Project: Solr
> Issue Type: Bug
> Components: search
> Affects Versions: 1.4, 1.5
> Reporter: Laxman
> Assignee: Yonik Seeley
> Fix For: 1.5
>
> Attachments: SOLR-1798.patch, SOLR-1798.patch
>
>
> Every time a commit happens two Stats instances [org.apache.solr.common.util.ConcurrentLRUCache.Stats] are leaking.
> Following code [org.apache.solr.search.FastLRUCache] to maintain cumulative cache statistics causing this Stats object leak.
> {noformat}
> cumulativeStats = (List<ConcurrentLRUCache.Stats>) persistence;
> cumulativeStats.add(cache.getStats());
> {noformat}
> Everytime a *commit* happens a new cache object is getting created and its stats is added to the list which is not released at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1798) Memory leak in FastLUCache
Posted by "Laxman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12839253#action_12839253 ]
Laxman commented on SOLR-1798:
------------------------------
Thanks for quick reponse Yonik.
Following is our analysis so far which may help us in resolving the issue.
1) For every commit cache instances are getting created.[org.apache.solr.search.SolrIndexSearcher]
2) For each cache config we are maintaining a list of Statistics for aggregation.
*One more observation*
>From the Solr stats page I see all the cache stats as 0 values. Following line of code from FastLRUCache.init looks incorrect.
{noformat}
cache.setAlive(false);
{noformat}
Because of the above line, all the cache instances created from FastLRUCache are not maintaining any Stats info.
If this is correct, I dont understand why are we maintaining a list of stats containing zero values.
Or did I miss something important here?
> Memory leak in FastLUCache
> --------------------------
>
> Key: SOLR-1798
> URL: https://issues.apache.org/jira/browse/SOLR-1798
> Project: Solr
> Issue Type: Bug
> Components: search
> Affects Versions: 1.4, 1.5
> Reporter: Laxman
> Fix For: 1.5
>
>
> Every time a commit happens two Stats instances [org.apache.solr.common.util.ConcurrentLRUCache.Stats] are leaking.
> Following code [org.apache.solr.search.FastLRUCache] to maintain cumulative cache statistics causing this Stats object leak.
> {noformat}
> cumulativeStats = (List<ConcurrentLRUCache.Stats>) persistence;
> cumulativeStats.add(cache.getStats());
> {noformat}
> Everytime a *commit* happens a new cache object is getting created and its stats is added to the list which is not released at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1798) Memory leak in FastLRUCache
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842319#action_12842319 ]
Yonik Seeley commented on SOLR-1798:
------------------------------------
Hmmm, right - fixes the memory leak, but breaks cumulative stats.
> Memory leak in FastLRUCache
> ---------------------------
>
> Key: SOLR-1798
> URL: https://issues.apache.org/jira/browse/SOLR-1798
> Project: Solr
> Issue Type: Bug
> Components: search
> Affects Versions: 1.4, 1.5
> Reporter: Laxman
> Assignee: Yonik Seeley
> Fix For: 1.5
>
> Attachments: SOLR-1798.patch
>
>
> Every time a commit happens two Stats instances [org.apache.solr.common.util.ConcurrentLRUCache.Stats] are leaking.
> Following code [org.apache.solr.search.FastLRUCache] to maintain cumulative cache statistics causing this Stats object leak.
> {noformat}
> cumulativeStats = (List<ConcurrentLRUCache.Stats>) persistence;
> cumulativeStats.add(cache.getStats());
> {noformat}
> Everytime a *commit* happens a new cache object is getting created and its stats is added to the list which is not released at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-1798) Memory leak in FastLRUCache
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley resolved SOLR-1798.
--------------------------------
Resolution: Fixed
committed.
> Memory leak in FastLRUCache
> ---------------------------
>
> Key: SOLR-1798
> URL: https://issues.apache.org/jira/browse/SOLR-1798
> Project: Solr
> Issue Type: Bug
> Components: search
> Affects Versions: 1.4, 1.5
> Reporter: Laxman
> Assignee: Yonik Seeley
> Fix For: 1.5
>
> Attachments: SOLR-1798.patch, SOLR-1798.patch
>
>
> Every time a commit happens two Stats instances [org.apache.solr.common.util.ConcurrentLRUCache.Stats] are leaking.
> Following code [org.apache.solr.search.FastLRUCache] to maintain cumulative cache statistics causing this Stats object leak.
> {noformat}
> cumulativeStats = (List<ConcurrentLRUCache.Stats>) persistence;
> cumulativeStats.add(cache.getStats());
> {noformat}
> Everytime a *commit* happens a new cache object is getting created and its stats is added to the list which is not released at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1798) Memory leak in FastLUCache
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12839251#action_12839251 ]
Yonik Seeley commented on SOLR-1798:
------------------------------------
Thanks Laxman, that does look incorrect.
Noble - do you remember why a List was used here?
> Memory leak in FastLUCache
> --------------------------
>
> Key: SOLR-1798
> URL: https://issues.apache.org/jira/browse/SOLR-1798
> Project: Solr
> Issue Type: Bug
> Components: search
> Affects Versions: 1.4, 1.5
> Reporter: Laxman
> Fix For: 1.5
>
>
> Every time a commit happens two Stats instances [org.apache.solr.common.util.ConcurrentLRUCache.Stats] are leaking.
> Following code [org.apache.solr.search.FastLRUCache] to maintain cumulative cache statistics causing this Stats object leak.
> {noformat}
> cumulativeStats = (List<ConcurrentLRUCache.Stats>) persistence;
> cumulativeStats.add(cache.getStats());
> {noformat}
> Everytime a *commit* happens a new cache object is getting created and its stats is added to the list which is not released at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1798) Memory leak in FastLRUCache
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yonik Seeley updated SOLR-1798:
-------------------------------
Attachment: SOLR-1798.patch
I think the fix is as simple as this patch.
Verifying that it fixes it is slightly harder - I'll try breaking out the profiler.
> Memory leak in FastLRUCache
> ---------------------------
>
> Key: SOLR-1798
> URL: https://issues.apache.org/jira/browse/SOLR-1798
> Project: Solr
> Issue Type: Bug
> Components: search
> Affects Versions: 1.4, 1.5
> Reporter: Laxman
> Assignee: Yonik Seeley
> Fix For: 1.5
>
> Attachments: SOLR-1798.patch
>
>
> Every time a commit happens two Stats instances [org.apache.solr.common.util.ConcurrentLRUCache.Stats] are leaking.
> Following code [org.apache.solr.search.FastLRUCache] to maintain cumulative cache statistics causing this Stats object leak.
> {noformat}
> cumulativeStats = (List<ConcurrentLRUCache.Stats>) persistence;
> cumulativeStats.add(cache.getStats());
> {noformat}
> Everytime a *commit* happens a new cache object is getting created and its stats is added to the list which is not released at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1798) Memory leak in FastLUCache
Posted by "Laxman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Laxman updated SOLR-1798:
-------------------------
Description:
Every time a commit happens two Stats instances [org.apache.solr.common.util.ConcurrentLRUCache.Stats] are leaking.
Following code [org.apache.solr.search.FastLRUCache] to maintain cumulative cache statistics causing this Stats object leak.
{noformat}
cumulativeStats = (List<ConcurrentLRUCache.Stats>) persistence;
cumulativeStats.add(cache.getStats());
{noformat}
Everytime a *commit* happens a new cache object is getting created and its stats is added to the list which is not released at all.
was:
Every time a commit happens two Stats instances [org.apache.solr.common.util.ConcurrentLRUCache.Stats] are leaking.
Following code to maintain cumulative cache statistics causing this Stats object leak.
{noformat}
cumulativeStats = (List<ConcurrentLRUCache.Stats>) persistence;
cumulativeStats.add(cache.getStats());
{noformat}
> Memory leak in FastLUCache
> --------------------------
>
> Key: SOLR-1798
> URL: https://issues.apache.org/jira/browse/SOLR-1798
> Project: Solr
> Issue Type: Bug
> Components: search
> Affects Versions: 1.4, 1.5
> Reporter: Laxman
> Fix For: 1.5
>
>
> Every time a commit happens two Stats instances [org.apache.solr.common.util.ConcurrentLRUCache.Stats] are leaking.
> Following code [org.apache.solr.search.FastLRUCache] to maintain cumulative cache statistics causing this Stats object leak.
> {noformat}
> cumulativeStats = (List<ConcurrentLRUCache.Stats>) persistence;
> cumulativeStats.add(cache.getStats());
> {noformat}
> Everytime a *commit* happens a new cache object is getting created and its stats is added to the list which is not released at all.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.