You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Vinh Le (Jira)" <ji...@apache.org> on 2020/01/20 06:58:00 UTC
[jira] [Comment Edited] (SOLR-10506) Possible memory leak upon
collection reload
[ https://issues.apache.org/jira/browse/SOLR-10506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17019264#comment-17019264 ]
Vinh Le edited comment on SOLR-10506 at 1/20/20 6:57 AM:
---------------------------------------------------------
In 7.7.2, some SolrCore still are not released after being removed (and alias switched to another one)
!image-2020-01-20-14-51-26-411.png!
was (Author: vinhlh):
In 7.7.2, some SolrCore still are not released after being removed (actually alias switched to another one)
!image-2020-01-20-14-51-26-411.png!
> Possible memory leak upon collection reload
> -------------------------------------------
>
> Key: SOLR-10506
> URL: https://issues.apache.org/jira/browse/SOLR-10506
> Project: Solr
> Issue Type: Bug
> Components: Server
> Affects Versions: 6.5
> Reporter: Torsten Bøgh Köster
> Assignee: Christine Poerschke
> Priority: Major
> Fix For: 6.6.6, 7.0
>
> Attachments: SOLR-10506.patch, image-2020-01-20-14-51-26-411.png, solr_collection_reload_13_cores.png, solr_gc_path_via_zk_WatchManager.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Upon manual Solr Collection reloading, references to the closed {{SolrCore}} are not fully removed by the garbage collector as a strong reference to the {{ZkIndexSchemaReader}} is held in a ZooKeeper {{Watcher}} that watches for schema changes.
> In our case, this leads to a massive memory leak as managed resources are still referenced by the closed {{SolrCore}}. Our Solr cloud environment utilizes rather large managed resources (synonyms, stopwords). To reproduce, we fired out environment up and reloaded the collection 13 times. As a result we fully exhausted our heap. A closer look with the Yourkit profiler revealed 13 {{SolrCore}} instances, still holding strong references to the garbage collection root (see screenshot 1).
> Each {{SolrCore}} instance holds a single path with strong references to the gc root via a `Watcher` in `ZkIndexSchemaReader` (see screenshot 2). The {{ZkIndexSchemaReader}} registers a close hook in the {{SolrCore}} but the Zookeeper is not removed upon core close.
> We supplied a Github Pull Request (https://github.com/apache/lucene-solr/pull/197) that extracts the zookeeper `Watcher` as a static inner class. To eliminate the memory leak, the schema reader is held inside a `WeakReference` and the reference is explicitly removed on core close.
> Initially I wanted to supply a test case but unfortunately did not find a good starting point ...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org