You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Tommaso Teofili <to...@gmail.com> on 2010/11/30 16:41:35 UTC
Dinamically change master
Hi all,
in a replication environment if the host where the master is running goes
down for some reason, is there a way to communicate to the slaves to point
to a different (backup) master without manually changing configuration (and
restarting the slaves or their cores)?
Basically I'd like to be able to change the replication master dinamically
inside the slaves.
Do you have any idea of how this could be achieved?
Thanks in advance for any help.
Regards,
Tommaso
Re: Dinamically change master
Posted by Tommaso Teofili <to...@gmail.com>.
Back with my master resiliency need, talking with Upayavira we discovered we
were proposing the same solution :-)
This can be useful if you don't have a VIP with master/backup polling
policy.
It goes like this: there are 2 host for indexing, one is the main and one is
the backup one, the backup one is slave of the main one and the main one is
also master of N hosts which will be used for searching. If the main master
goes down then the backup one will be used for indexing and/or serving
search slaves.
This last feature can be done defining an external properties file for each
search slave which will contain the URL to master (pointed inside the
replication request handler tag of solrconfig.xml), so if these search
slaves run on multi core one has only to change properties file URL and
issue a http://SLAVEURL/solr/admin/cores?action=RELOAD&core=core0 to get
polling the backup master.
Cheers,
Tommaso
2010/12/1 Tommaso Teofili <to...@gmail.com>
> Thanks Upayavira, that sounds very good.
>
> p.s.:
> I read that page some weeks ago and didn't get back to check on it.
>
>
> 2010/12/1 Upayavira <uv...@odoko.co.uk>
>
>> Note, all extracted from http://wiki.apache.org/solr/SolrReplication
>>
>> You'd put:
>>
>> <requestHandler name="/replication" class="solr.ReplicationHandler" >
>> <lst name="master">
>> <!--Replicate on 'startup' and 'commit'. 'optimize' is also a
>> valid value for replicateAfter. -->
>> <str name="replicateAfter">startup</str>
>> <str name="replicateAfter">commit</str>
>> </lst>
>> </requestHandler>
>>
>> into every box you want to be able to act as a master, then use:
>>
>> http://slave_host:port
>> /solr/replication?command=fetchindex&masterUrl=<your
>> master URL>
>>
>> As the above page says better than I can, "It is possible to pass on
>> extra attribute 'masterUrl' or other attributes like 'compression' (or
>> any other parameter which is specified in the <lst name="slave"> tag) to
>> do a one time replication from a master. This obviates the need for
>> hardcoding the master in the slave."
>>
>> HTH, Upayavira
>>
>> On Wed, 01 Dec 2010 06:24 +0100, "Tommaso Teofili"
>> <to...@gmail.com> wrote:
>> > Hi Upayavira,
>> > this is a good start for solving my problem, can you please tell how
>> does
>> > such a replication URL look like?
>> > Thanks,
>> > Tommaso
>> >
>> > 2010/12/1 Upayavira <uv...@odoko.co.uk>
>> >
>> > > Hi Tommaso,
>> > >
>> > > I believe you can tell each server to act as a master (which means it
>> > > can have its indexes pulled from it).
>> > >
>> > > You can then include the master hostname in the URL that triggers a
>> > > replication process. Thus, if you triggered replication from outside
>> > > solr, you'd have control over which master you pull from.
>> > >
>> > > Does this answer your question?
>> > >
>> > > Upayavira
>> > >
>> > >
>> > > On Tue, 30 Nov 2010 09:18 -0800, "Ken Krugler"
>> > > <kk...@transpac.com> wrote:
>> > > > Hi Tommaso,
>> > > >
>> > > > On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote:
>> > > >
>> > > > > Hi all,
>> > > > >
>> > > > > in a replication environment if the host where the master is
>> running
>> > > > > goes
>> > > > > down for some reason, is there a way to communicate to the slaves
>> to
>> > > > > point
>> > > > > to a different (backup) master without manually changing
>> > > > > configuration (and
>> > > > > restarting the slaves or their cores)?
>> > > > >
>> > > > > Basically I'd like to be able to change the replication master
>> > > > > dinamically
>> > > > > inside the slaves.
>> > > > >
>> > > > > Do you have any idea of how this could be achieved?
>> > > >
>> > > > One common approach is to use VIP (virtual IP) support provided by
>> > > > load balancers.
>> > > >
>> > > > Your slaves are configured to use a VIP to talk to the master, so
>> that
>> > > > it's easy to dynamically change which master they use, via updates
>> to
>> > > > the load balancer config.
>> > > >
>> > > > -- Ken
>> > > >
>> > > > --------------------------
>> > > > Ken Krugler
>> > > > +1 530-210-6378
>> > > > http://bixolabs.com
>> > > > e l a s t i c w e b m i n i n g
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > >
>> >
>>
>
>
Re: Dinamically change master
Posted by Tommaso Teofili <to...@gmail.com>.
Thanks Upayavira, that sounds very good.
p.s.:
I read that page some weeks ago and didn't get back to check on it.
2010/12/1 Upayavira <uv...@odoko.co.uk>
> Note, all extracted from http://wiki.apache.org/solr/SolrReplication
>
> You'd put:
>
> <requestHandler name="/replication" class="solr.ReplicationHandler" >
> <lst name="master">
> <!--Replicate on 'startup' and 'commit'. 'optimize' is also a
> valid value for replicateAfter. -->
> <str name="replicateAfter">startup</str>
> <str name="replicateAfter">commit</str>
> </lst>
> </requestHandler>
>
> into every box you want to be able to act as a master, then use:
>
> http://slave_host:port/solr/replication?command=fetchindex&masterUrl=<your
> master URL>
>
> As the above page says better than I can, "It is possible to pass on
> extra attribute 'masterUrl' or other attributes like 'compression' (or
> any other parameter which is specified in the <lst name="slave"> tag) to
> do a one time replication from a master. This obviates the need for
> hardcoding the master in the slave."
>
> HTH, Upayavira
>
> On Wed, 01 Dec 2010 06:24 +0100, "Tommaso Teofili"
> <to...@gmail.com> wrote:
> > Hi Upayavira,
> > this is a good start for solving my problem, can you please tell how does
> > such a replication URL look like?
> > Thanks,
> > Tommaso
> >
> > 2010/12/1 Upayavira <uv...@odoko.co.uk>
> >
> > > Hi Tommaso,
> > >
> > > I believe you can tell each server to act as a master (which means it
> > > can have its indexes pulled from it).
> > >
> > > You can then include the master hostname in the URL that triggers a
> > > replication process. Thus, if you triggered replication from outside
> > > solr, you'd have control over which master you pull from.
> > >
> > > Does this answer your question?
> > >
> > > Upayavira
> > >
> > >
> > > On Tue, 30 Nov 2010 09:18 -0800, "Ken Krugler"
> > > <kk...@transpac.com> wrote:
> > > > Hi Tommaso,
> > > >
> > > > On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > in a replication environment if the host where the master is
> running
> > > > > goes
> > > > > down for some reason, is there a way to communicate to the slaves
> to
> > > > > point
> > > > > to a different (backup) master without manually changing
> > > > > configuration (and
> > > > > restarting the slaves or their cores)?
> > > > >
> > > > > Basically I'd like to be able to change the replication master
> > > > > dinamically
> > > > > inside the slaves.
> > > > >
> > > > > Do you have any idea of how this could be achieved?
> > > >
> > > > One common approach is to use VIP (virtual IP) support provided by
> > > > load balancers.
> > > >
> > > > Your slaves are configured to use a VIP to talk to the master, so
> that
> > > > it's easy to dynamically change which master they use, via updates to
> > > > the load balancer config.
> > > >
> > > > -- Ken
> > > >
> > > > --------------------------
> > > > Ken Krugler
> > > > +1 530-210-6378
> > > > http://bixolabs.com
> > > > e l a s t i c w e b m i n i n g
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> >
>
Re: Dinamically change master
Posted by Upayavira <uv...@odoko.co.uk>.
Note, all extracted from http://wiki.apache.org/solr/SolrReplication
You'd put:
<requestHandler name="/replication" class="solr.ReplicationHandler" >
<lst name="master">
<!--Replicate on 'startup' and 'commit'. 'optimize' is also a
valid value for replicateAfter. -->
<str name="replicateAfter">startup</str>
<str name="replicateAfter">commit</str>
</lst>
</requestHandler>
into every box you want to be able to act as a master, then use:
http://slave_host:port/solr/replication?command=fetchindex&masterUrl=<your
master URL>
As the above page says better than I can, "It is possible to pass on
extra attribute 'masterUrl' or other attributes like 'compression' (or
any other parameter which is specified in the <lst name="slave"> tag) to
do a one time replication from a master. This obviates the need for
hardcoding the master in the slave."
HTH, Upayavira
On Wed, 01 Dec 2010 06:24 +0100, "Tommaso Teofili"
<to...@gmail.com> wrote:
> Hi Upayavira,
> this is a good start for solving my problem, can you please tell how does
> such a replication URL look like?
> Thanks,
> Tommaso
>
> 2010/12/1 Upayavira <uv...@odoko.co.uk>
>
> > Hi Tommaso,
> >
> > I believe you can tell each server to act as a master (which means it
> > can have its indexes pulled from it).
> >
> > You can then include the master hostname in the URL that triggers a
> > replication process. Thus, if you triggered replication from outside
> > solr, you'd have control over which master you pull from.
> >
> > Does this answer your question?
> >
> > Upayavira
> >
> >
> > On Tue, 30 Nov 2010 09:18 -0800, "Ken Krugler"
> > <kk...@transpac.com> wrote:
> > > Hi Tommaso,
> > >
> > > On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote:
> > >
> > > > Hi all,
> > > >
> > > > in a replication environment if the host where the master is running
> > > > goes
> > > > down for some reason, is there a way to communicate to the slaves to
> > > > point
> > > > to a different (backup) master without manually changing
> > > > configuration (and
> > > > restarting the slaves or their cores)?
> > > >
> > > > Basically I'd like to be able to change the replication master
> > > > dinamically
> > > > inside the slaves.
> > > >
> > > > Do you have any idea of how this could be achieved?
> > >
> > > One common approach is to use VIP (virtual IP) support provided by
> > > load balancers.
> > >
> > > Your slaves are configured to use a VIP to talk to the master, so that
> > > it's easy to dynamically change which master they use, via updates to
> > > the load balancer config.
> > >
> > > -- Ken
> > >
> > > --------------------------
> > > Ken Krugler
> > > +1 530-210-6378
> > > http://bixolabs.com
> > > e l a s t i c w e b m i n i n g
> > >
> > >
> > >
> > >
> > >
> > >
> >
>
Re: Dinamically change master
Posted by Tommaso Teofili <to...@gmail.com>.
Hi Upayavira,
this is a good start for solving my problem, can you please tell how does
such a replication URL look like?
Thanks,
Tommaso
2010/12/1 Upayavira <uv...@odoko.co.uk>
> Hi Tommaso,
>
> I believe you can tell each server to act as a master (which means it
> can have its indexes pulled from it).
>
> You can then include the master hostname in the URL that triggers a
> replication process. Thus, if you triggered replication from outside
> solr, you'd have control over which master you pull from.
>
> Does this answer your question?
>
> Upayavira
>
>
> On Tue, 30 Nov 2010 09:18 -0800, "Ken Krugler"
> <kk...@transpac.com> wrote:
> > Hi Tommaso,
> >
> > On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote:
> >
> > > Hi all,
> > >
> > > in a replication environment if the host where the master is running
> > > goes
> > > down for some reason, is there a way to communicate to the slaves to
> > > point
> > > to a different (backup) master without manually changing
> > > configuration (and
> > > restarting the slaves or their cores)?
> > >
> > > Basically I'd like to be able to change the replication master
> > > dinamically
> > > inside the slaves.
> > >
> > > Do you have any idea of how this could be achieved?
> >
> > One common approach is to use VIP (virtual IP) support provided by
> > load balancers.
> >
> > Your slaves are configured to use a VIP to talk to the master, so that
> > it's easy to dynamically change which master they use, via updates to
> > the load balancer config.
> >
> > -- Ken
> >
> > --------------------------
> > Ken Krugler
> > +1 530-210-6378
> > http://bixolabs.com
> > e l a s t i c w e b m i n i n g
> >
> >
> >
> >
> >
> >
>
Re: Dinamically change master
Posted by Upayavira <uv...@odoko.co.uk>.
Hi Tommaso,
I believe you can tell each server to act as a master (which means it
can have its indexes pulled from it).
You can then include the master hostname in the URL that triggers a
replication process. Thus, if you triggered replication from outside
solr, you'd have control over which master you pull from.
Does this answer your question?
Upayavira
On Tue, 30 Nov 2010 09:18 -0800, "Ken Krugler"
<kk...@transpac.com> wrote:
> Hi Tommaso,
>
> On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote:
>
> > Hi all,
> >
> > in a replication environment if the host where the master is running
> > goes
> > down for some reason, is there a way to communicate to the slaves to
> > point
> > to a different (backup) master without manually changing
> > configuration (and
> > restarting the slaves or their cores)?
> >
> > Basically I'd like to be able to change the replication master
> > dinamically
> > inside the slaves.
> >
> > Do you have any idea of how this could be achieved?
>
> One common approach is to use VIP (virtual IP) support provided by
> load balancers.
>
> Your slaves are configured to use a VIP to talk to the master, so that
> it's easy to dynamically change which master they use, via updates to
> the load balancer config.
>
> -- Ken
>
> --------------------------
> Ken Krugler
> +1 530-210-6378
> http://bixolabs.com
> e l a s t i c w e b m i n i n g
>
>
>
>
>
>
Re: Dinamically change master
Posted by Tommaso Teofili <to...@gmail.com>.
Hi,
Thanks Jacob and Ken for your replies.
I am not able to change project architecture to add Lucandra even if it
looks like a nice solution.
Going the VIP way can definitely an option even if I'd be more keen to solve
that "inside" Solr.
I am thinking to try and play with Collection Distribution scripts or doing
some custom stuff.
But, please, if you have any other advice let me know.
Thanks again.
Tommaso
2010/11/30 Ken Krugler <kk...@transpac.com>
> Hi Tommaso,
>
>
> On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote:
>
> Hi all,
>>
>> in a replication environment if the host where the master is running goes
>> down for some reason, is there a way to communicate to the slaves to point
>> to a different (backup) master without manually changing configuration
>> (and
>> restarting the slaves or their cores)?
>>
>> Basically I'd like to be able to change the replication master dinamically
>> inside the slaves.
>>
>> Do you have any idea of how this could be achieved?
>>
>
> One common approach is to use VIP (virtual IP) support provided by load
> balancers.
>
> Your slaves are configured to use a VIP to talk to the master, so that it's
> easy to dynamically change which master they use, via updates to the load
> balancer config.
>
> -- Ken
>
> --------------------------
> Ken Krugler
> +1 530-210-6378
> http://bixolabs.com
> e l a s t i c w e b m i n i n g
>
>
>
>
>
>
Re: Dinamically change master
Posted by Ken Krugler <kk...@transpac.com>.
Hi Tommaso,
On Nov 30, 2010, at 7:41am, Tommaso Teofili wrote:
> Hi all,
>
> in a replication environment if the host where the master is running
> goes
> down for some reason, is there a way to communicate to the slaves to
> point
> to a different (backup) master without manually changing
> configuration (and
> restarting the slaves or their cores)?
>
> Basically I'd like to be able to change the replication master
> dinamically
> inside the slaves.
>
> Do you have any idea of how this could be achieved?
One common approach is to use VIP (virtual IP) support provided by
load balancers.
Your slaves are configured to use a VIP to talk to the master, so that
it's easy to dynamically change which master they use, via updates to
the load balancer config.
-- Ken
--------------------------
Ken Krugler
+1 530-210-6378
http://bixolabs.com
e l a s t i c w e b m i n i n g
Re: Dinamically change master
Posted by Jacob Elder <je...@locamoda.com>.
Your best bet might be to look into Lucandra:
https://github.com/tjake/Lucandra
On Tue, Nov 30, 2010 at 10:41 AM, Tommaso Teofili <tommaso.teofili@gmail.com
> wrote:
> Hi all,
>
> in a replication environment if the host where the master is running goes
> down for some reason, is there a way to communicate to the slaves to point
> to a different (backup) master without manually changing configuration (and
> restarting the slaves or their cores)?
>
> Basically I'd like to be able to change the replication master dinamically
> inside the slaves.
>
> Do you have any idea of how this could be achieved?
>
> Thanks in advance for any help.
> Regards,
> Tommaso
>
--
Jacob Elder
@jelder
(646) 535-3379