numdocs is different on replicas in a shard

[Webster Homer <we...@sial.com>]

This morning I was told that there was something screwy with one of our
collections.
This collection has 2 shards and 2 replicas per shard. Each replica has a
different value for numDocs!
Datacenter #1
shard1_replica1    1513053
shard1_replica2    1512653
shard2_replica1    1512296
shard2_replica2    1512487

We have 2 copies of this collection that are populated via cdcr from a
common collection. Both copies show the same thing. They run in different
datacenters in the cloud
Datacenter #2
shard1_replica1    1513054
shard1_replica2    1512903
shard2_replica1    1512452
shard2_replica2    1512487

We are running Solr 7.2.0 in SolrCloud mode.
This collection is populated by CDCR, auto commits are enabled.

I don't see any errors in the logs. I manually sent a commit to the
collection, the above numbers are after the commit.

The source collection has only 2 replicas per shard
shard1_replica1   1513054
shard2_replica1   1512487


What could cause this? How can I address it? How do we prevent it from
happening again?

-- 


This message and any attachment are confidential and may be
privileged or 
otherwise protected from disclosure. If you are not the intended
recipient, 
you must not copy this message or attachment or disclose the
contents to 
any other person. If you have received this transmission in error,
please 
notify the sender immediately and delete the message and any attachment

from your system. Merck KGaA, Darmstadt, Germany and any of its 
subsidiaries do
not accept liability for any omissions or errors in this 
message which may
arise as a result of E-Mail-transmission or for damages 
resulting from any
unauthorized changes of the content of this message and 
any attachment thereto.
Merck KGaA, Darmstadt, Germany and any of its 
subsidiaries do not guarantee
that this message is free of viruses and does 
not accept liability for any
damages caused by any virus transmitted 
therewith.



Click http://www.emdgroup.com/disclaimer 
<http://www.emdgroup.com/disclaimer> to access the
German, French, Spanish 
and Portuguese versions of this disclaimer.