RE: Collection reload leaves dangling SolrCore instances

[Markus Jelsma <ma...@openindex.io>]

Hello,

It doesn't appear to be fixed on 7.4. During core reload there is the memory spike, afterwards i loose about 110 MB in OldGen. Forcing GC doesn't free the memory.

On reloads, it seems one instance per core is retained. So on my local box where all collections are in one instance, i can easily get OOME if i reload a couple of replica's.

So, do you have another already existing issue for me matching this problem? Or shall i open a new issue.

Many thanks,
Markus



 
 
-----Original message-----
> From:Markus Jelsma <ma...@openindex.io>
> Sent: Wednesday 2nd May 2018 18:01
> To: solr-user@lucene.apache.org; solr-user <so...@lucene.apache.org>
> Subject: RE: Collection reload leaves dangling SolrCore instances
> 
> Sounds just like it, i will check it out!
> 
> Thanks both!
> Markus
> 
>  
>  
> -----Original message-----
> > From:Erick Erickson <er...@gmail.com>
> > Sent: Wednesday 2nd May 2018 17:21
> > To: solr-user <so...@lucene.apache.org>
> > Subject: Re: Collection reload leaves dangling SolrCore instances
> > 
> > Markus:
> > 
> > You may well be hitting SOLR-11882.
> > 
> > On Wed, May 2, 2018 at 8:18 AM, Shawn Heisey <ap...@elyograg.org> wrote:
> > > On 5/2/2018 4:40 AM, Markus Jelsma wrote:
> > >> One of our collections, that is heavy with tons of TokenFilters using large dictionaries, has a lot of trouble dealing with collection reload. I removed all custom plugins from solrconfig, dumbed the schema down and removed all custom filters and replaced a customized decompounder with Lucene's vanilla filter, and the problem still exists.
> > >>
> > >> After collection reload a second SolrCore instance appears for each real core in use, each next reload causes the number of instances to grow. The dangling instances are eventually removed except for one or two. When working locally with for example two shards/one replica in one JVM, a single reload eats about 500 MB for each reload.
> > >>
> > >> How can we force Solr to remove those instances sooner? Forcing a GC won't do it so it seems Solr itself actively keeps some stale instances alive.
> > >
> > > Custom plugins, which you did mention, would be the most likely
> > > culprit.  Those sometimes have bugs where they don't properly close
> > > resources.  Are you absolutely sure that there is no custom software
> > > loading at all?  Removing the jars entirely (not just the config that
> > > might use the jars) might be required.
> > >
> > > Have you been able to get heap dumps and figure out what object is
> > > keeping the SolrCore alive?
> > >
> > > Thanks,
> > > Shawn
> > >
> > 
>