solr cloud vers 7.6.0 Requested node XXXXX:8090_solr is not part of the cluster

[Jeff Courtade <co...@gmail.com>]

Hi,

I have a large solr deployment with solr 7.6.0

128 servers 5 zookeepers on separate systems

each solr host runs 2 solr instances one a primary shard one a secondary shard
port 8080 is always primary aka master replica
port 8090 is always a secondary replica

These are all NRT replicas

so to add insult to injury we never start the secondary replicas
except by accident as the servers cannot handle the load of both solrs
running at the same time.

so we have 128 primary shard replicas with a secondary replica that is
down all the time

we are seeing this in our cloud console under nodes

Requested node XXXXXXX:8090_solr is not part of the cluster

So the primary replica for that shard is fine and the 8090 replica
shows up in the console as down like all the other ones on port 8080.

the solr cluster is answering all queries.

does this make any difference at all to the cluster being able to
serve requests?

how could we jumpstart that node so this error goes away?