You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Andrzej Bialecki (Jira)" <ji...@apache.org> on 2019/09/25 12:15:00 UTC
[jira] [Created] (SOLR-13790) LRUStatsCache size explosion
Andrzej Bialecki created SOLR-13790:
----------------------------------------
Summary: LRUStatsCache size explosion
Key: SOLR-13790
URL: https://issues.apache.org/jira/browse/SOLR-13790
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 8.2, 7.7.2, 8.3
Reporter: Andrzej Bialecki
Assignee: Andrzej Bialecki
Fix For: 7.7.3, 8.3
On a sizeable cluster with multi-shard multi-replica collections, when {{LRUStatsCache}} was in use we encountered excessive memory usage, which consequently led to severe performance problems.
On a closer examination of the heapdumps it became apparent that when {{LRUStatsCache.addToPerShardTermStats}} is called it creates instances of {{FastLRUCache}} using the passed {{shard}} argument - however, the value of this argument is not a simple shard name but instead it's a randomly ordered list of ALL replica URLs for this shard.
As a result, due to the combinatoric number of possible keys, over time the map in {{LRUStatsCache.perShardTemStats}} grew to contain ~2 mln entries...
The fix seems to be simply to extract the shard name and cache using this name instead of the full string value of the {{shard}} parameter. Existing unit tests also need much improvement.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org