Why will slaves not replicate from master when Leader (Searching) and Leader (Replicable) do not match?

[Stian Brattland <st...@octetnest.no>]

I've got a Solr 9.1 master and slave. I have set up replication.

When the slave starts up, I do the following request on that *slave* to
retrieve the index from master :
http://localhost:8983/solr/[core]/replication?command=fetchindex. However,
this always gives the following error :

----------
2022-12-16 14:57:58.159 INFO  (explicit-fetchindex-cmd) []
o.a.s.h.IndexFetcher Leader's generation: 658
2022-12-16 14:57:58.160 INFO  (explicit-fetchindex-cmd) []
o.a.s.h.IndexFetcher Leader's version: 1671202320920
2022-12-16 14:57:58.160 INFO  (explicit-fetchindex-cmd) []
o.a.s.h.IndexFetcher Follower's generation: 1
2022-12-16 14:57:58.160 INFO  (explicit-fetchindex-cmd) []
o.a.s.h.IndexFetcher Follower's version: 0
2022-12-16 14:57:58.160 INFO  (explicit-fetchindex-cmd) []
o.a.s.h.IndexFetcher Starting replication process
2022-12-16 14:57:58.164 ERROR (explicit-fetchindex-cmd) []
o.a.s.h.IndexFetcher No files to download for index generation: 658
----------

The slave refuses to replicate from the master until I have run
http://localhost:8983/solr/[core]/update?optimize=true on the *master*.

However, I then need to hurry up and initiate a replication from the slave,
because the slave will only replicate the master if Leader (Searching) and
Leader (Replicable) matches.

Screenshot from the master :
[image: image.png]

As soon as Leader (Searching) and Leader (Replicable) no longer match, the
slave again refuse to replicate:
[image: image.png]

The slave probably refuse to replicate simply because it is unable to. If I
request
http://localhost:8983/solr/[core]/replication?command=filelist&generation=667
on the *master*, I get :

{
  "responseHeader":{
    "status":0,
    "QTime":0},
  "status":"ERROR",
  "message":"invalid index generation"}

The Leader (Searching) value increases all the time, because indexing and
commits are happening.

I find it hard to believe that this is by design. Should slaves really not
be able to replicate from the master when Leader (Searching) and Leader
(Replicable) do not match?

Regards,
Stian Brattland