You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by kondamudims <ko...@gmail.com> on 2013/11/26 19:47:17 UTC
SOLR Master-Slave Repeater with Load balancer
We are trying to setup solr Master Slave repeater model, where we will have
two solr servers say S1 and S2 and a Load balancer LB to route all the
requests to either S1 or S2. S1 and S2 acts as both Master and
Slave(Repeater). In both the solr server configurations, in the
solrconfig.xml file for master url property if we provide Load balancer
host-name and port number then at any point there will be a self polling,
i.e. if LB is configured in such a way that all its requests will be routed
to S1, then while polling S1-->LB-->S1 and S2-->LB-->S1. Do you see any
issue with self polling(S1-->LB-->). We are mainly trying to achieve High
availability as we don't want to use Solr Cloud. Thanks in advance
--
View this message in context: http://lucene.472066.n3.nabble.com/SOLR-Master-Slave-Repeater-with-Load-balancer-tp4103363.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SOLR Master-Slave Repeater with Load balancer
Posted by Walter Underwood <wu...@wunderwood.org>.
Erick is right, you have been put in a terrible position.
You need to get agreement, in writing, that it is OK for search to go down when one server is out of service. This might be for scheduled maintenance or even a config update. When one server is down, search is down, period.
This requirement is like choosing a truck, but insisting that there is only budget for three tires.
You must, must, must communicate the risks associated with a two-server SolrCloud cluster.
wunder
On Dec 4, 2013, at 7:10 PM, Erick Erickson <er...@gmail.com> wrote:
> bq: but we have limitation on the number of servers that we can use due to
> budget
> concerns (limit is 2)
>
> really, really, really push back to your project managers on this. So what
> you need 3 machines for a ZooKeeper quorum? The needs of ZK are quite
> light, they don't need a powerful machine. Your managers are saying "for
> want of spending $1,000 on a machine, which we will waste 10 times that
> paying engineers to set up an old-style system, we can't go with
> SolrCloud". You can run the ZooKeeper instances in a separate JVM on your
> two servers and have a cheap machine running ZK for the third instance if
> necessary.
>
> Another rant finished.....
>
> Erick
>
>
> On Wed, Dec 4, 2013 at 6:07 PM, kondamudims <ko...@gmail.com> wrote:
>
>> Hi Erick,
>> Thanks a lot for your explanation. We initially considered Solr Cloud but
>> we
>> have limitation on the number of servers that we can use due to budget
>> concerns (limit is 2) Solr Cloud requires minimum 3. I have tried out the
>> solution you suggested and so far its going well and we are not doing self
>> polling concept.
>>
>>
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/SOLR-Master-Slave-Repeater-with-Load-balancer-tp4103363p4105017.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
--
Walter Underwood
wunder@wunderwood.org
Re: SOLR Master-Slave Repeater with Load balancer
Posted by Erick Erickson <er...@gmail.com>.
bq: but we have limitation on the number of servers that we can use due to
budget
concerns (limit is 2)
really, really, really push back to your project managers on this. So what
you need 3 machines for a ZooKeeper quorum? The needs of ZK are quite
light, they don't need a powerful machine. Your managers are saying "for
want of spending $1,000 on a machine, which we will waste 10 times that
paying engineers to set up an old-style system, we can't go with
SolrCloud". You can run the ZooKeeper instances in a separate JVM on your
two servers and have a cheap machine running ZK for the third instance if
necessary.
Another rant finished.....
Erick
On Wed, Dec 4, 2013 at 6:07 PM, kondamudims <ko...@gmail.com> wrote:
> Hi Erick,
> Thanks a lot for your explanation. We initially considered Solr Cloud but
> we
> have limitation on the number of servers that we can use due to budget
> concerns (limit is 2) Solr Cloud requires minimum 3. I have tried out the
> solution you suggested and so far its going well and we are not doing self
> polling concept.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/SOLR-Master-Slave-Repeater-with-Load-balancer-tp4103363p4105017.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: SOLR Master-Slave Repeater with Load balancer
Posted by kondamudims <ko...@gmail.com>.
Hi Erick,
Thanks a lot for your explanation. We initially considered Solr Cloud but we
have limitation on the number of servers that we can use due to budget
concerns (limit is 2) Solr Cloud requires minimum 3. I have tried out the
solution you suggested and so far its going well and we are not doing self
polling concept.
--
View this message in context: http://lucene.472066.n3.nabble.com/SOLR-Master-Slave-Repeater-with-Load-balancer-tp4103363p4105017.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SOLR Master-Slave Repeater with Load balancer
Posted by Erick Erickson <er...@gmail.com>.
Yes. This is going to hurt you a lot. The intent of M/S is that
you should be indexing to one, and only one machine, the
master. All slaves pull their indexes from the master. Frankly
I don't know quite what will happen in the configuration you're
talking about. I strongly recommend you do not do this.
HA is not at all difficult in a M/S situation. Just configure
as many slaves as you need, all pointing to the same master.
Then have your LB point to the slaves. And have your indexing
process(es) point to the master and only the master.
Bringing up a new slave is as simple as configuring a new machine,
pointing it to the master, waiting for replication to complete then
letting the LB know about the new machine.
What you don't get is fail-over if the master goes down. You can
promote one of the slaves to be a master but you have to put
a mechanism in place that allows you to "catch up" that index,
which is often just re-indexing everything from before the master
went down to the new master.
You said you don't want to go to SolrCloud, and that's up to
you. But automatic, robust HA/DR is hard. If your goal is to have
a fault-tolerant, automated recovery system I really urge you to
reconsider SolrCloud. The _point_ of SolrCloud is exactly
that. If you try to re-invent that process you'll be in for a lot of work.
There, rant finished :)
Erick
On Tue, Nov 26, 2013 at 1:47 PM, kondamudims <ko...@gmail.com> wrote:
> We are trying to setup solr Master Slave repeater model, where we will have
> two solr servers say S1 and S2 and a Load balancer LB to route all the
> requests to either S1 or S2. S1 and S2 acts as both Master and
> Slave(Repeater). In both the solr server configurations, in the
> solrconfig.xml file for master url property if we provide Load balancer
> host-name and port number then at any point there will be a self polling,
> i.e. if LB is configured in such a way that all its requests will be routed
> to S1, then while polling S1-->LB-->S1 and S2-->LB-->S1. Do you see any
> issue with self polling(S1-->LB-->). We are mainly trying to achieve High
> availability as we don't want to use Solr Cloud. Thanks in advance
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/SOLR-Master-Slave-Repeater-with-Load-balancer-tp4103363.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>