Getting Reset cancel_stream_error on solr-8.5.2

["raj.yadav" <ra...@cse.ism.ac.in>]

Hey All,
We have updated our system from solr 5.4 to solr 8.5.2 and we are suddenly
seeing a lot of the below errors in our logs.

HttpChannelState org.eclipse.jetty.io.EofException: Reset
cancel_stream_error

Is this related to some system level or solr level config?

How do I find the cause of this?
How do I solve this?

*Solr Setup Details:*
Solr version => solr-8.5.2

GC setting: GC_TUNE=" -XX:+UseG1GC -XX:+PerfDisableSharedMem
-XX:+ParallelRefProcEnabled -XX:G1HeapRegionSize=8m -XX:MaxGCPauseMillis=150
-XX:InitiatingHeapOccupancyPercent=60 -XX:+UseLargePages -XX:+AggressiveOpts
"

Solr Collection details: (running in solrCloud mode) It has 6 shards, and
each shard has only one replica (which is also a leader) and replica type is
NRT. Total doc in collection: 77 million and each shard index size: 11 GB
and avg size/doc: 1.0Kb

Zookeeper => We are using external zookeeper ensemble (3 node cluster)

System Datails:
Centos (7.7); disk size: 250 GB; cpu: (8 vcpus, 64 GiB memory)


Solr OPs

Solr start command
#/var/solr-8.5.2/bin/solr start -c  -p solr_port  -z
zk_host1:zk_port,zk_host1:zk_port,zk_host1:zk_port -s
/var/node_my_collection_1/solr-8.5.2/server/solr -h <solr_ip> -m 26g 
-DzkClientTimeout=30000 -force



Creat Collection 
1.upload config to zookeper
#var/solr-8.5.2/server/scripts/cloud-scripts/./zkcli.sh -z
zk_host1:zk_port,zk_host1:zk_port,zk_host1:zk_port  -cmd upconfig -confname
my_collection  -confdir /<local_config_path>

2. Cretaed collection with 3 shards (shard1,shard2,shard3),
#curl 
"http://<solr_ip>:solr_port/solr/admin/collections?action=CREATE&name=my_collection&numShards=3&replicationFactor=1&maxShardsPerNode=1&collection.configName=my_collection&createNodeSet=solr_node1:solr_port,solr_node2:solr_port,solr_node3:solr_port"

3. Used SPLITSHARD command to split each shards into two half
(shard1_1,shard1_0,shard2_0,...)
e.g
 #curl
"http://<solr_ip>:solr_port/solr/admin/collections?action=SPLITSHARD&collection=my_collection&shard=shard1

4. Used DELETESHARD command to delete old shatds (shard1,shard2,shard3).
e.g
 #curl
"http://<solr_ip>:solr_port/solr/admin/collections?action=DELETESHARD&collection=my_collection&shard=shard1




--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html