Re: solr and dovecot: high load

[al...@qboxmail.it.INVALID]

Hello,

thank you for the answers. We reduced the heap size to 20GB and it has 
made the situation better: it didn't stop, but the I/O activity was 
still high.


We are a bit at a loss on what to monitor, maybe the garbage collector 
activity?

I'm not aware of any particular planned recurrent activity around 
midnight, but we'll have a closer look at dovecot.


We have another solr (that handles less users) that had a high I/O 
activity, which was reduced after increasing the physical memory from 
24GB to 48GB. Could this be a scalable solution?


Thank you,

AG


On 24/11/22 12:39, Jan Høydahl wrote:
> Hi,
>
> Why the large Solr heap? What is happing around midnight? A nightly batch index?
> Try lowering it from 30240m to e.g. 16g (or even lower) and see how it goes.
>
> Jan
>
>> 24. nov. 2022 kl. 12:21 skrev alessia.gagliano@qboxmail.it.invalid <al...@qboxmail.it.INVALID>:
>>
>> Hi,
>>
>> we have a strange issue with solr: sometimes, usually everyday at around midnight, solr becomes unreachable due to a high cpu load and high I/O activity.
>>
>>
>> Our configurations are:
>>
>>   - Solr 7.7.3 on CentOS 7
>>   - Dovecot 2.3.18 on CentOS 7
>>   - /var/solr is around 250GB
>>   - the Solr server has 40GB of ram / 2GB of swap (not used).
>>   - SOLR_HEAP="30240m"
>>   - sovecot is configured with "fts_autoindex = no"
>>
>> It shouldn't be a RAM problem,  as only 30% is used before restarting ( "memory":"8.6 GB (%30.4) of 28.3 GB"}).
>>
>>
>> Does anyone know what could be the cause of this behavior?
>>
>>
>> Thanks,
>>
>> AG
>>
>>

Re: solr and dovecot: high load

[Shawn Heisey <ap...@elyograg.org>]

On 12/1/22 02:47, alessia.gagliano@qboxmail.it.INVALID wrote:
> We have another solr (that handles less users) that had a high I/O 
> activity, which was reduced after increasing the physical memory from 
> 24GB to 48GB. Could this be a scalable solution?

More memory not consumed by programs will often greatly increase general 
performance, and Solr in particular relies on the disk cache.  Add 
enough memory and there will be almost zero I/O for queries, most of the 
I/O will be for index updates.

https://cwiki.apache.org/confluence/display/solr/solrperformanceproblems

Thanks,
Shawn