You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Marcin Rzewucki <mr...@gmail.com> on 2012/12/28 17:39:25 UTC
ZooKeeper ensemble behind load balancer
Hi,
Does Solr need connection to all of hosts in ZK ensemble or only to one of
them at a time ? I wonder if it is possible to use load balancer for ZK
ensemble and use only one address as zkHost for Solr ? Having load balancer
makes it easier to change ZK hosts while still using same address by Solr
(no need to restart Solr or change its configuration).
Thanks in advance.
Regards.
Re: ZooKeeper ensemble behind load balancer
Posted by Marcin Rzewucki <mr...@gmail.com>.
Right, that's a good idea. Thanks!
On 30 December 2012 17:41, Aloke Ghoshal <al...@gmail.com> wrote:
> Hi Marcin,
>
> Since you are thinking of this in the context of Amazon, I would suggest
> taking a different route. Assign an Elastic IP (EIP) to each EC2 instance
> running the ZK node & use the EIP in Solr. This way you could easily map
> the EIP to a new EC2 instance subsequently, if required, and the changes
> would get picked up by Solr automatically without a restart.
>
> Regards,
> Aloke
>
>
> On Sun, Dec 30, 2012 at 8:29 PM, Marcin Rzewucki <mrzewucki@gmail.com
> >wrote:
>
> > Hi,
> > Thanks for your replies. I don't change ZK hosts a lot of times. I'm
> using
> > Amazon platform and if a ZK host is not available I will start new EC2
> > instance and will have to plug it to Solr configuration (restart
> required).
> > In case of load balancer this would be much easier (no need to restart
> > solr).
> >
> > Regards.
> >
> >
> > On 30 December 2012 04:06, Anirudha Jadhav <an...@nyu.edu> wrote:
> >
> > > A zookeeper ensemble should be a fairly reliable, large enough no.of
> > > machines(3+ typically 5,7,9) for a quorum.
> > > So adding a load balancer on top will just add a hop and
> > > decrease performance, and also add a failure point in the system.
> > >
> > > that being said there needs to be a way to provide solr with a way to
> > > refresh conf. without restart.
> > >
> > > Solr takes a list of zk hosts on startup, If i am correct , uses one of
> > > them unless it fails or round robins.
> > >
> > > why do your zkhosts need to change a lot?
> > >
> > >
> > > On Sat, Dec 29, 2012 at 10:58 AM, Upayavira <uv...@odoko.co.uk> wrote:
> > >
> > > > I would suggest asking this on the zookeeper user list.
> > > >
> > > > And let us know here what you find out, I'd be interested.
> > > >
> > > > Note, zookeeper, as I understand it, uses its own protocol, so to
> some
> > > > reasonable extent it probablmy depends on yr load balancer. Also, as
> I
> > > > understand it, zookeeper maintains active connections to solr hosts,
> > > > which is not a common scenario for load balances as I understand it.
> > > >
> > > > Upayavira
> > > >
> > > > On Fri, Dec 28, 2012, at 04:39 PM, Marcin Rzewucki wrote:
> > > > > Hi,
> > > > >
> > > > > Does Solr need connection to all of hosts in ZK ensemble or only to
> > one
> > > > > of
> > > > > them at a time ? I wonder if it is possible to use load balancer
> for
> > ZK
> > > > > ensemble and use only one address as zkHost for Solr ? Having load
> > > > > balancer
> > > > > makes it easier to change ZK hosts while still using same address
> by
> > > Solr
> > > > > (no need to restart Solr or change its configuration).
> > > > >
> > > > > Thanks in advance.
> > > > > Regards.
> > > >
> > >
> > >
> > >
> > > --
> > > Anirudha P. Jadhav
> > >
> >
>
Re: ZooKeeper ensemble behind load balancer
Posted by Aloke Ghoshal <al...@gmail.com>.
Hi Marcin,
Since you are thinking of this in the context of Amazon, I would suggest
taking a different route. Assign an Elastic IP (EIP) to each EC2 instance
running the ZK node & use the EIP in Solr. This way you could easily map
the EIP to a new EC2 instance subsequently, if required, and the changes
would get picked up by Solr automatically without a restart.
Regards,
Aloke
On Sun, Dec 30, 2012 at 8:29 PM, Marcin Rzewucki <mr...@gmail.com>wrote:
> Hi,
> Thanks for your replies. I don't change ZK hosts a lot of times. I'm using
> Amazon platform and if a ZK host is not available I will start new EC2
> instance and will have to plug it to Solr configuration (restart required).
> In case of load balancer this would be much easier (no need to restart
> solr).
>
> Regards.
>
>
> On 30 December 2012 04:06, Anirudha Jadhav <an...@nyu.edu> wrote:
>
> > A zookeeper ensemble should be a fairly reliable, large enough no.of
> > machines(3+ typically 5,7,9) for a quorum.
> > So adding a load balancer on top will just add a hop and
> > decrease performance, and also add a failure point in the system.
> >
> > that being said there needs to be a way to provide solr with a way to
> > refresh conf. without restart.
> >
> > Solr takes a list of zk hosts on startup, If i am correct , uses one of
> > them unless it fails or round robins.
> >
> > why do your zkhosts need to change a lot?
> >
> >
> > On Sat, Dec 29, 2012 at 10:58 AM, Upayavira <uv...@odoko.co.uk> wrote:
> >
> > > I would suggest asking this on the zookeeper user list.
> > >
> > > And let us know here what you find out, I'd be interested.
> > >
> > > Note, zookeeper, as I understand it, uses its own protocol, so to some
> > > reasonable extent it probablmy depends on yr load balancer. Also, as I
> > > understand it, zookeeper maintains active connections to solr hosts,
> > > which is not a common scenario for load balances as I understand it.
> > >
> > > Upayavira
> > >
> > > On Fri, Dec 28, 2012, at 04:39 PM, Marcin Rzewucki wrote:
> > > > Hi,
> > > >
> > > > Does Solr need connection to all of hosts in ZK ensemble or only to
> one
> > > > of
> > > > them at a time ? I wonder if it is possible to use load balancer for
> ZK
> > > > ensemble and use only one address as zkHost for Solr ? Having load
> > > > balancer
> > > > makes it easier to change ZK hosts while still using same address by
> > Solr
> > > > (no need to restart Solr or change its configuration).
> > > >
> > > > Thanks in advance.
> > > > Regards.
> > >
> >
> >
> >
> > --
> > Anirudha P. Jadhav
> >
>
Re: ZooKeeper ensemble behind load balancer
Posted by Marcin Rzewucki <mr...@gmail.com>.
Hi,
Thanks for your replies. I don't change ZK hosts a lot of times. I'm using
Amazon platform and if a ZK host is not available I will start new EC2
instance and will have to plug it to Solr configuration (restart required).
In case of load balancer this would be much easier (no need to restart
solr).
Regards.
On 30 December 2012 04:06, Anirudha Jadhav <an...@nyu.edu> wrote:
> A zookeeper ensemble should be a fairly reliable, large enough no.of
> machines(3+ typically 5,7,9) for a quorum.
> So adding a load balancer on top will just add a hop and
> decrease performance, and also add a failure point in the system.
>
> that being said there needs to be a way to provide solr with a way to
> refresh conf. without restart.
>
> Solr takes a list of zk hosts on startup, If i am correct , uses one of
> them unless it fails or round robins.
>
> why do your zkhosts need to change a lot?
>
>
> On Sat, Dec 29, 2012 at 10:58 AM, Upayavira <uv...@odoko.co.uk> wrote:
>
> > I would suggest asking this on the zookeeper user list.
> >
> > And let us know here what you find out, I'd be interested.
> >
> > Note, zookeeper, as I understand it, uses its own protocol, so to some
> > reasonable extent it probablmy depends on yr load balancer. Also, as I
> > understand it, zookeeper maintains active connections to solr hosts,
> > which is not a common scenario for load balances as I understand it.
> >
> > Upayavira
> >
> > On Fri, Dec 28, 2012, at 04:39 PM, Marcin Rzewucki wrote:
> > > Hi,
> > >
> > > Does Solr need connection to all of hosts in ZK ensemble or only to one
> > > of
> > > them at a time ? I wonder if it is possible to use load balancer for ZK
> > > ensemble and use only one address as zkHost for Solr ? Having load
> > > balancer
> > > makes it easier to change ZK hosts while still using same address by
> Solr
> > > (no need to restart Solr or change its configuration).
> > >
> > > Thanks in advance.
> > > Regards.
> >
>
>
>
> --
> Anirudha P. Jadhav
>
Re: ZooKeeper ensemble behind load balancer
Posted by Anirudha Jadhav <an...@nyu.edu>.
A zookeeper ensemble should be a fairly reliable, large enough no.of
machines(3+ typically 5,7,9) for a quorum.
So adding a load balancer on top will just add a hop and
decrease performance, and also add a failure point in the system.
that being said there needs to be a way to provide solr with a way to
refresh conf. without restart.
Solr takes a list of zk hosts on startup, If i am correct , uses one of
them unless it fails or round robins.
why do your zkhosts need to change a lot?
On Sat, Dec 29, 2012 at 10:58 AM, Upayavira <uv...@odoko.co.uk> wrote:
> I would suggest asking this on the zookeeper user list.
>
> And let us know here what you find out, I'd be interested.
>
> Note, zookeeper, as I understand it, uses its own protocol, so to some
> reasonable extent it probablmy depends on yr load balancer. Also, as I
> understand it, zookeeper maintains active connections to solr hosts,
> which is not a common scenario for load balances as I understand it.
>
> Upayavira
>
> On Fri, Dec 28, 2012, at 04:39 PM, Marcin Rzewucki wrote:
> > Hi,
> >
> > Does Solr need connection to all of hosts in ZK ensemble or only to one
> > of
> > them at a time ? I wonder if it is possible to use load balancer for ZK
> > ensemble and use only one address as zkHost for Solr ? Having load
> > balancer
> > makes it easier to change ZK hosts while still using same address by Solr
> > (no need to restart Solr or change its configuration).
> >
> > Thanks in advance.
> > Regards.
>
--
Anirudha P. Jadhav
Re: ZooKeeper ensemble behind load balancer
Posted by Upayavira <uv...@odoko.co.uk>.
I would suggest asking this on the zookeeper user list.
And let us know here what you find out, I'd be interested.
Note, zookeeper, as I understand it, uses its own protocol, so to some
reasonable extent it probablmy depends on yr load balancer. Also, as I
understand it, zookeeper maintains active connections to solr hosts,
which is not a common scenario for load balances as I understand it.
Upayavira
On Fri, Dec 28, 2012, at 04:39 PM, Marcin Rzewucki wrote:
> Hi,
>
> Does Solr need connection to all of hosts in ZK ensemble or only to one
> of
> them at a time ? I wonder if it is possible to use load balancer for ZK
> ensemble and use only one address as zkHost for Solr ? Having load
> balancer
> makes it easier to change ZK hosts while still using same address by Solr
> (no need to restart Solr or change its configuration).
>
> Thanks in advance.
> Regards.