Replication in Master Slave Solr setup

[vracks <v....@gmail.com>]

Basic Questions about the Replication in Master Slave Solr Setup.

1) Can Master push the changes to Slaves using the replication handler

2) If the Answer to the above question is no, then what is use of having the
option of replicateAfter in the replicationHandler, since only the Slave is
going to poll the master at a particular interval.
If the answer to the above question is yes, then wanted to know the master
knows about the Slave instances to which to push the changes. Since the
replication handler dont have any options to mention the slaves.If the
answer to the above question is yes, which is best option for replication
push or pull.

Sample Master Slave replication handler setup

<requestHandler name="/replication" class="solr.ReplicationHandler">
  <lst name="master">
    <str name="replicateAfter">commit</str>
    <str name="confFiles">schema.xml,stopwords.txt,synonyms.txt</str>
  </lst>
  <lst name="slave">
    <str
name="masterUrl">http://master.solr.company.com:8983/solr/core_name/replication</str>
    <str name="pollInterval">00:00:60</str>
  </lst>
</requestHandler>

Please help me on understanding the replication architecture on Solr.





--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Replication in Master Slave Solr setup

[Erick Erickson <er...@gmail.com>]

The OP was making an invalid assumption I think, that the index would
replicate _whenever_ the index changed. But that's not necessarily
true, although it's the most common (and default) case.

From the ref guiide:

'If you use "startup", you need to have a "commit" and/or "optimize"
entry also if you want to trigger replication on future commits or
optimizes.'

So you can set up your master/slave installation to replicate on
master startup but never again automatically _unless_ replicateAfter
is also set to commit or optimize too. You could then explicitly send
a replication API "fetchIndex" command if you wanted total control of
when replications happened.

For instance, imagine a setup where you updated your master index over
the course of a day, but only wanted the results available for
yesterday on the slaves. Setting your poll interval wouldn't help
because that timer starts whenever you start your slave. Set
replicateAfter to onStartup then at midnight each night use the
replication fetchIndex API call on each of the slaves.

Somewhat of a corner case, but possible.

Best,
Erick


On Sun, Mar 18, 2018 at 10:06 PM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 3/17/2018 8:06 PM, vracks wrote:
>>
>> Basic Questions about the Replication in Master Slave Solr Setup.
>>
>> 1) Can Master push the changes to Slaves using the replication handler
>
>
> Replication is always pull -- the slave asks the master if there's anything
> to copy.
>
>> 2) If the Answer to the above question is no, then what is use of having
>> the
>> option of replicateAfter in the replicationHandler, since only the Slave
>> is
>> going to poll the master at a particular interval.
>
>
> The replicateAfter options control when the master will tell a polling slave
> that there is a change.
>
>> If the answer to the above question is yes, then wanted to know the master
>> knows about the Slave instances to which to push the changes.
>
>
> The master does not know about slaves until they connect. It does not push
> changes.
>
> Thanks,
> Shawn
>

Re: Replication in Master Slave Solr setup

[Shawn Heisey <ap...@elyograg.org>]

On 3/17/2018 8:06 PM, vracks wrote:
> Basic Questions about the Replication in Master Slave Solr Setup.
>
> 1) Can Master push the changes to Slaves using the replication handler

Replication is always pull -- the slave asks the master if there's 
anything to copy.

> 2) If the Answer to the above question is no, then what is use of having the
> option of replicateAfter in the replicationHandler, since only the Slave is
> going to poll the master at a particular interval.

The replicateAfter options control when the master will tell a polling 
slave that there is a change.

> If the answer to the above question is yes, then wanted to know the master
> knows about the Slave instances to which to push the changes.

The master does not know about slaves until they connect. It does not 
push changes.

Thanks,
Shawn