Geographically redundant ZooKeeper instances

[Cameron McKenzie <mc...@gmail.com>]

Hi,
I'm prototyping a solution using ZooKeeper that relies on geographic
redundancy. Specifically, we require two geographically redundant sites,
but I'm not sure how this should be configured with ZooKeepers quorum
requirements. The number of clients and amount of data being written and
consumed will be quite small (in terms of what ZooKeeper can handle), so
from capacity point of view we can get away with 3 ZK nodes.

How should these be configured though to achieve geographical redundancy?
We can't just have 2 ZK nodes at one site and 1 at the other, as this means
that if the site with 2 nodes explodes then the other site cannot take over.

The only way I can think of achieving this redundancy is by having 3 sites,
each with a single ZK node. This is not ideal though as it means that we
need another site purely for ZK.

Am I missing something here?
cheers
Cam

Re: Geographically redundant ZooKeeper instances

[Cameron McKenzie <mc...@gmail.com>]

Thanks Camille,
I have read your blog post, but was hoping that given that my case was
slightly simpler than yours that there was some other simpler alternative.

Thanks for the quick reply.
cheers
Cam



On Mon, Sep 16, 2013 at 8:34 AM, Camille Fournier <ca...@apache.org>wrote:

> No, to do this you need a tiebreaker node at a tertiary site. I wrote a
> blog post on this a while back, you may find it useful:
>
> http://whilefalse.blogspot.com/2012/12/building-global-highly-available.html
>
> Best,
> Camille
>
>
> On Sun, Sep 15, 2013 at 6:32 PM, Cameron McKenzie <mckenzie.cam@gmail.com
> >wrote:
>
> > Hi,
> > I'm prototyping a solution using ZooKeeper that relies on geographic
> > redundancy. Specifically, we require two geographically redundant sites,
> > but I'm not sure how this should be configured with ZooKeepers quorum
> > requirements. The number of clients and amount of data being written and
> > consumed will be quite small (in terms of what ZooKeeper can handle), so
> > from capacity point of view we can get away with 3 ZK nodes.
> >
> > How should these be configured though to achieve geographical redundancy?
> > We can't just have 2 ZK nodes at one site and 1 at the other, as this
> means
> > that if the site with 2 nodes explodes then the other site cannot take
> > over.
> >
> > The only way I can think of achieving this redundancy is by having 3
> sites,
> > each with a single ZK node. This is not ideal though as it means that we
> > need another site purely for ZK.
> >
> > Am I missing something here?
> > cheers
> > Cam
> >
>

Re: Geographically redundant ZooKeeper instances

[Camille Fournier <ca...@apache.org>]

No, to do this you need a tiebreaker node at a tertiary site. I wrote a
blog post on this a while back, you may find it useful:
http://whilefalse.blogspot.com/2012/12/building-global-highly-available.html

Best,
Camille


On Sun, Sep 15, 2013 at 6:32 PM, Cameron McKenzie <mc...@gmail.com>wrote:

> Hi,
> I'm prototyping a solution using ZooKeeper that relies on geographic
> redundancy. Specifically, we require two geographically redundant sites,
> but I'm not sure how this should be configured with ZooKeepers quorum
> requirements. The number of clients and amount of data being written and
> consumed will be quite small (in terms of what ZooKeeper can handle), so
> from capacity point of view we can get away with 3 ZK nodes.
>
> How should these be configured though to achieve geographical redundancy?
> We can't just have 2 ZK nodes at one site and 1 at the other, as this means
> that if the site with 2 nodes explodes then the other site cannot take
> over.
>
> The only way I can think of achieving this redundancy is by having 3 sites,
> each with a single ZK node. This is not ideal though as it means that we
> need another site purely for ZK.
>
> Am I missing something here?
> cheers
> Cam
>