You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@solr.apache.org by Walter Underwood <wu...@wunderwood.org> on 2021/04/19 05:14:09 UTC

Re: Running out of 16GB heap during indexing

Any suggestions?

Looks like we are ditching SolrJ anyway, but it would be nice to have a bit more info.

wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Mar 29, 2021, at 1:18 PM, Walter Underwood <wu...@wunderwood.org> wrote:
> 
> Does this fix need changes on both the client and server side of SolrJ? Because of some older clusters, we are using SolrJ 4.4.10. We can upgrade to 6.6.2, but that predates this fix.
> 
> https://issues.apache.org/jira/browse/SOLR-14013
> 
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
> 
>> On Mar 29, 2021, at 11:33 AM, Walter Underwood <wu...@wunderwood.org> wrote:
>> 
>> We have a cluster that is running out of heap during indexing. Our older cluster does not have this problem. Both clusters have 8 shards and 3 replica sets, for a total of 24 hosts. Both use the same G1GC settings. Both have the same indexing feed of 57 million documents.
>> 
>> Old cluster:
>> Solr 6.6.2
>> 11 GB heap
>> Oracle Corporation OpenJDK 64-Bit Server VM 1.8.0_131 25.131-b11
>> 
>> New cluster:
>> Solr 8.7.0
>> 16 GB heap
>> Amazon.com Inc. OpenJDK 64-Bit Server VM 11.0.10 11.0.10+9-LTS
>> 
>> Cache sizes on the new cluster are pretty standard, filter and query result caches are 512, document cache is 2048. Documents aren’t huge, maybe average size a few kbytes. They are homework problems.
>> 
>> Any ideas? The 6.6.2 cluster has been running for a few years with no indexing problems.
>> 
>> wunder
>> Walter Underwood
>> wunder@wunderwood.org
>> http://observer.wunderwood.org/  (my blog)
>> 
>