Re: Solr Nodes Killed During a ReIndexing Process on New VMs Out of Memory Error

[Shawn Heisey <ap...@elyograg.org>]

On 7/18/2018 7:10 AM, THADC wrote:
> We performed a full reindex for the first time against our largest database
> and on two new VMs dedicated to solr indexing. We have two solr nodes
> (solrCloud/solr7.3) with a zookeeper cluster. Several hours into the
> reindexing process, both solr nodes shut down with:
>
> java.long.OutOfMemoryError: Java heap space
>
> Running OOM killer script for process blah on port blah
>
> Does this definitely indicate we need more memory or could it just be a heap
> space adjustment issue? Is there a way to better diagnose what to do?

https://wiki.apache.org/solr/SolrPerformanceProblems#Java_Heap

There are exactly two ways to deal with OOME:  Increase the available 
amount of the resource that's running out (heap space in this case), or 
change something so the program requires less of that resource.  
Depending on the hardware and software configuration, either of these 
options might turn out to be impossible.

The rest of that wiki page discusses memory in general as well as heap 
memory.  If you have questions after reading the page, go ahead and ask 
them.

Thanks,
Shawn

Re: Solr Nodes Killed During a ReIndexing Process on New VMs Out of Memory Error

[THADC <ti...@gmail.com>]

Thanks, made heap size considerably larger and its fine now. Thank you



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr Nodes Killed During a ReIndexing Process on New VMs Out of Memory Error

[Shawn Heisey <ap...@elyograg.org>]

On 7/18/2018 8:31 AM, THADC wrote:
> Thanks for the reply. I read the link you provided. I am currently not
> specifying a heap size with solr so my understanding is that by default it
> will just grow automatically. If I add more physical memory to the VM
> without doing anything with heap size, won't that possibly fix the problem?

No, that is not how it works.  If Java is not given a heap size, then it 
will choose the heap size for you based on how much memory the machine 
has and its own internal algorithms, and limit itself to that amount.

Solr 5.0 and later, when started using the included scripts, asks Java 
for a 512MB heap by default.  This is an extremely small heap.  Nearly 
all Solr users must increase the heap size beyond 512MB.

Thanks,
Shawn

Re: Solr Nodes Killed During a ReIndexing Process on New VMs Out of Memory Error

[THADC <ti...@gmail.com>]

Thanks for the reply. I read the link you provided. I am currently not
specifying a heap size with solr so my understanding is that by default it
will just grow automatically. If I add more physical memory to the VM
without doing anything with heap size, won't that possibly fix the problem?

Thanks



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html