Merging / Syncing two SOLR indexes

[Gilles Comeau <gi...@polecat.co>]

Hi all,

I have two SOLR shards of about 20 GB  (server1_2015_02_01 and server2_2015_02_01 are the shard names, representing core 2015_02_01).  For historical reasons we are not using replication, instead we are writing from a source to each of the shards.

I've found that not all the rows have made it to both shards for a core.  There are only a few thousand missing in each case.

We're looking for a way to "sync" the shards.   There is a unique key.   We've tried two techniques - one is reading out all the IDs from each and then writing the missing ones.   The dirtier one was just writing everything from one to the other and then back.

Does anyone know of a smarter technique to sync up two indexes?

Cheers,
Gilles