You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Erick Erickson (JIRA)" <ji...@apache.org> on 2018/03/14 21:23:00 UTC
[jira] [Commented] (SOLR-11882) SolrMetric registries retain
references to SolrCores when closed
[ https://issues.apache.org/jira/browse/SOLR-11882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399440#comment-16399440 ]
Erick Erickson commented on SOLR-11882:
---------------------------------------
[~ab] OK, I think the light finally dawned. We're talking about two different cases and they both have to be handled.
1> transient core case, the one I'm started with. In this case, the core is closed out and _may_, some time in the near or far future be opened again. In this case the patch from 28-Jan is probably almost fine although there's still a (probably small but unacceptable) chance that a new version of the core would be opened before the closer thread got 'round to closing the old one.
2> reopening a core which is the case you're talking about in your comment 1-Feb.
In <2> there's no problem with cores accumulating due to the reference in the metrics code since they've been released by the new assignment already.
Does that make sense?
And is there a good way other than inspection to test any fixes I make?
Thanks!
> SolrMetric registries retain references to SolrCores when closed
> ----------------------------------------------------------------
>
> Key: SOLR-11882
> URL: https://issues.apache.org/jira/browse/SOLR-11882
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics, Server
> Affects Versions: 7.1
> Reporter: Eros Taborelli
> Assignee: Erick Erickson
> Priority: Major
> Attachments: SOLR-11882.patch, SOLR-11882.patch, SOLR-11882.patch, SOLR-11882.patch, create-cores.zip, solr-dump-full_Leak_Suspects.zip, solr.config.zip
>
>
> *Description:*
> Our setup involves using a lot of small cores (possibly hundred thousand), but working only on a few of them at any given time.
> We already followed all recommendations in this guide: [https://wiki.apache.org/solr/LotsOfCores]
> We noticed that after creating/loading around 1000-2000 empty cores, with no documents inside, the heap consumption went through the roof despite having set transientCacheSize to only 64 (heap size set to 12G).
> All cores are correctly set to loadOnStartup=false and transient=true, and we have verified via logs that the cores in excess are actually being closed.
> However, a reference remains in the org.apache.solr.metrics.SolrMetricManager#registries that is never removed until a core if fully unloaded.
> Restarting the JVM loads all cores in the admin UI, but doesn't populate the ConcurrentHashMap until a core is actually fully loaded.
> I reproduced the issue on a smaller scale (transientCacheSize = 5, heap size = 512m) and made a report (attached) using eclipse MAT.
> *Desired outcome:*
> When a transient core is closed, the references in the SolrMetricManager should be removed, in the same fashion the reporters for the core are also closed and removed.
> In alternative, a unloadOnClose=true|false flag could be implemented to fully unload a transient core when closed due to the cache size.
> *Note:*
> The documentation mentions everywhere that the unused cores will be unloaded, but it's misleading as the cores are never fully unloaded.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org