Re: Urgent help needed on Solr cloud(cont)

[SayantiGmail <sa...@gmail.com>]

Hi

This seems to be a bug in Solr 8.4 .Will this get resolved in higher versions or we need to update the stall time configuration as a workaround.

> On 12 Apr 2021, at 22:41, Carlos .Sponchiado <cs...@gmail.com> wrote:
> 
> 1. I found a similar issue in this version of Solr here
> https://github.com/clarin-eric/VLO/issues/291 , They suggest using
> solr.cloud.client.stallTime
> to mitigate it. But I think fixing the commit issue will solve this problem
> too.
> 4. The delete query was supposed to only mark the document inside each
> segment as deleted.
> 
> Do you know if the throughput of updating documents increased a lot? In the
> stats of SolrAdmin is possible to see how frequently commits are happening.
> If it is a lot, this fix of avoid send commit command and have hardCommit
> and SoftCommit configured can help you. Let's wait for other suggestions
> here too.
> 
>> Em seg., 12 de abr. de 2021 às 18:51, Rekha Sekhar <re...@gmail.com>
>> escreveu:
>> 
>> Hi,
>> 
>> Thank you for the information. We will try out the below suggestions.
>> 
>> I have a few more questions, which are facing in the application.
>> 
>> The application has *2 solr (v 8.4.1)* and *3 Zookeeper (v3.6.2) *running
>> in SolrCloud mode.
>> After running it for few days we could see below error in logs -
>> 2021-04-12 11:04:26.850 ERROR (qtp1632497828-1072) [c:datacore s:shard1
>> r:core_node4 x:datacore_shard1_replica_n2] o.a.s.u.SolrCmdDistributor
>> java.io.IOException: *Request processing has stalled for 90083ms with 100
>> remaining elements in the queue*.
>> and
>> 2021-04-12 09:00:36.350 ERROR (qtp1632497828-786) [c:datacore s:shard1
>> r:core_node4 x:datacore_shard1_replica_n2] o.a.s.s.HttpSolrCall
>> null:java.io.IOException: *Task queue processing has stalled for 90175 ms
>> with 92 remaining elements to process.*
>> 
>> 
>> 1. What do these error messages mean? How can we resolve this?
>> 2. After getting these messages,  the 2 Solr nodes show different document
>> count and delete count. It seems the 2 Solr nodes are not sync(screenshot
>> attached for reference).
>> 3. One of the nodes (not leader) goes to a recovering state forever.
>> 4.In the solr update requests of 1 Lakh records, there are few thousands of
>> delete query as well. Do  the delete query introduce more slowness in
>> synching the nodes
>> 
>> The above messages are coming frequently from both the Solr nodes and
>> finally one node goes to a recovering state forever.
>> 
>> Could you please help by answering the above queries.
>> 
>> Thanks,
>> Rekha
>> 
> 
> 
> -- 
> Abraços
> Carlos Sponchiado