Is there any detailed condition on which the snapshot pull recovery will occur?

[forest_soup <ta...@gmail.com>]

We have a SolrCloud with solr v5.3.2. 
collection1 contains 1 shard with 2 replicas on solr nodes: solr1 and solr2
respectively.
In solrconfig.xml, there are updateLog config and uploaded to ZK and
effective:
    <updateLog>
      <str name="dir">${solr.ulog.dir:}</str>
      <int
name="numVersionBuckets">${solr.ulog.numVersionBuckets:65536}</int>
      <int name="numRecordsToKeep">1000</int>
      <int name="maxNumLogsToKeep">100</int>
    </updateLog>

We know with these settings, at first solr1 down and solr2 active, and solr2
received more than 1000 updates, after solr1 is restarted, the recovery of
the replica in solr1 will be snapshot pull.

But we noticed a case with below steps:
1, At first solr1 and solr2 are active and both replicas has lots of data;
2, solr2 is shutdown;
3, update to solr1 with less than 1000 updates;
4, solr1 is shutdown;
5, the replica's data dir in solr2 are missing due to bad device or
mis-deletion;
6, solr2 is startup;
7, update to solr2 with about 2 or 3 updates;
8, solr1 is startup;
9, we noticed both replicas in solr1 and solr2 have only those 2 or 3
update's data in step #7. 
Lots of data lost!

It seems the recovery in solr1 is snapshot pull from solr2. 
Our questions:
1, Is there any explanation on this case?
2, Is there any detailed condition on which the snapshot pull recovery will
occur? 

Thanks!



--
View this message in context: http://lucene.472066.n3.nabble.com/Is-there-any-detailed-condition-on-which-the-snapshot-pull-recovery-will-occur-tp4271311.html
Sent from the Solr - User mailing list archive at Nabble.com.