Solr Slaves Sync

[OLX - Pablo Garrido <sy...@olx.com>]

Hello

	We have a 3 Solr Servers replication schema, one Master and 2 Slaves,
commits are done every 5 minutes on the Master and an optimize is done
once a day during midnight, snapshots are copied via rsync to Slaves are
done every 10 minutes, we are facing serious problems when doing the
sync after the optimize and keeping Slaves serving queries as usual,
active connections to Slaves increase highly during Optimize Snapshot
sync, is there any way we can tune this process ? we try this process :

 	1. stopping Sync process on one Slave
	2. taking the other one out of the LB pool
	3. do the sync on this offline Slave
	4. after sync is over add back to LB Pool synced Slave
	5. take other Slave out from LB Pool
	6. start sync process on the offline Slave
	8 add back synced Slave to LB Pool

following these steps we sometimes face high active connections when
moving Slaves back to LB Pool. Has anybody faced this situation in
production envs ? Thanks

	Pablo 

Re: Solr Slaves Sync

[Chris Hostetter <ho...@fucit.org>]

: As far as I can tell, there is no need to remove a slave from a pool while

assuming you are using the replication scripts provided with Solr, this is 
correct.  (i suppose if you were using windows or some other odd 
replication system, there *might* be a reason for taking them out of 
rotaiton)

as long as you don't do anything wacky to produce a snapshot on the master 
in the middle of your <optimize> there is no reason to stop rsyncd on the 
master, or disable snappuller on the slaves.

the only time i ever "disable" replication on any machine is when i'm 
deploying a completley new set of configs, and i want to rebuild a new 
index from scratch on my master and test it out before any of my slaves 
see it (meanwhile they are still serveing requests using the old stale 
index)...
  http://wiki.apache.org/solr/CollectionRebuilding


-Hoss

Re: Solr Slaves Sync

[Matthew Runo <mr...@zappos.com>]

As far as I can tell, there is no need to remove a slave from a pool  
while performing the sync. It's all done in the background and doesn't  
change anything till the final <commit/> is ran to open a new searcher.

Thanks for your time!

Matthew Runo
Software Engineer, Zappos.com
mruno@zappos.com - 702-943-7833

On Sep 4, 2008, at 10:46 AM, OLX - Pablo Garrido wrote:

> Hello
>
> 	We have a 3 Solr Servers replication schema, one Master and 2 Slaves,
> commits are done every 5 minutes on the Master and an optimize is done
> once a day during midnight, snapshots are copied via rsync to Slaves  
> are
> done every 10 minutes, we are facing serious problems when doing the
> sync after the optimize and keeping Slaves serving queries as usual,
> active connections to Slaves increase highly during Optimize Snapshot
> sync, is there any way we can tune this process ? we try this  
> process :
>
> 	1. stopping Sync process on one Slave
> 	2. taking the other one out of the LB pool
> 	3. do the sync on this offline Slave
> 	4. after sync is over add back to LB Pool synced Slave
> 	5. take other Slave out from LB Pool
> 	6. start sync process on the offline Slave
> 	8 add back synced Slave to LB Pool
>
> following these steps we sometimes face high active connections when
> moving Slaves back to LB Pool. Has anybody faced this situation in
> production envs ? Thanks
>
> 	Pablo
>