You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Kaushal Shriyan <ka...@gmail.com> on 2015/05/29 19:08:01 UTC
Meaning of quorum and ensemble
Hi,
I have a question regarding quorum in context to
http://stackoverflow.com/questions/13022244/zookeeper-reliability-three-versus-five-nodes
Zookeeper requires that you have a quorum of servers up, where quorum is
ceil(N/2). For a 3 server ensemble, that means 2 servers must be up at any
time, for a 5 server ensemble, 3 servers need to be up at any time.
1. what does quorum mean?
2. so does ceil(N/2) mean 3+1/2 = 4/2 which is 2 servers in context to 3
server ensemble and 5+1/2 = 6/2 which is 3 servers in context to 5 server
ensemble if the ceil(N/2) is a ceiling function?
3. what does ensemble mean in context to Apache zookeeper?
I would appreciate your help.
Regards,
Kaushal
Re: Meaning of quorum and ensemble
Posted by Chris Nauroth <cn...@hortonworks.com>.
That's a very interesting question. With only 2 availability zones, it's
impossible to place the nodes such that a quorum can be satisfied during a
full outage of one of the availability zones. If we place 3 nodes in 1a
and 2 nodes in 1b, then an outage of 1a causes loss of quorum. This seems
to imply that you'd need to deploy in an AWS region that supports 3 or
more availability zones. Then, you could deploy 2 nodes in zone a, 2
nodes in zone b, and 1 node in zone c. Loss of zone a or b still would
leave 3/5 nodes running. Loss of zone c still would leave 4/5 nodes
running.
I haven't deployed ZooKeeper in AWS personally, so I'm not well-versed in
this. Maybe others on the list who have deployed in AWS could chime in to
describe their deployment architecture.
--Chris Nauroth
On 5/29/15, 12:04 PM, "Kaushal Shriyan" <ka...@gmail.com> wrote:
>Thanks Chris for the explanation. Please help me understand how would I
>implement a 3 node or a 5 node ensemble in a AWS region for example
>Singapore Region which has two Availability Zone 1a and 1b if either of
>the
>zone becomes unavailable due to say for example Network Glitch or not
>reachable.
>
>Regards,
>
>Kaushal
> On 29 May 2015 23:41, "Chris Nauroth" <cn...@hortonworks.com> wrote:
>
>> Hello Kaushal,
>>
>> 1. "Quorum" refers to the minimum number of nodes that must agree on a
>> transaction before it is considered committed.
>> 2. Yes, your understanding of ceil(N/2) is correct. It sounds like your
>> formulation of it is more like floor((N+1)/2). The two formulations are
>> equivalent, so whichever one is more intuitive to you is valid.
>> 3. "Ensemble" refers to the full set of peer servers in a ZooKeeper
>> cluster.
>>
>> Tying this back to your examples, a 3-node ensemble requires a quorum
>>of 2
>> servers running to commit a transaction. A 5-node ensemble requires a
>> quorum of 3 servers running to commit a transaction. An administrator
>>can
>> deploy a ZooKeeper cluster sized according to trade-offs between budget
>> (number of servers) and desired fault tolerance (number of simultaneous
>> server failures). A more detailed discussion is available in the
>> ZooKeeper Internals document:
>>
>> http://zookeeper.apache.org/doc/r3.4.6/zookeeperInternals.html
>>
>>
>> I hope this helps.
>>
>> --Chris Nauroth
>>
>>
>>
>>
>> On 5/29/15, 10:08 AM, "Kaushal Shriyan" <ka...@gmail.com>
>>wrote:
>>
>> >Hi,
>> >
>> >I have a question regarding quorum in context to
>> >
>>
>>http://stackoverflow.com/questions/13022244/zookeeper-reliability-three-v
>>e
>> >rsus-five-nodes
>> >
>> >Zookeeper requires that you have a quorum of servers up, where quorum
>>is
>> >ceil(N/2). For a 3 server ensemble, that means 2 servers must be up at
>>any
>> >time, for a 5 server ensemble, 3 servers need to be up at any time.
>> >
>> > 1. what does quorum mean?
>> > 2. so does ceil(N/2) mean 3+1/2 = 4/2 which is 2 servers in context
>>to
>> >3
>> > server ensemble and 5+1/2 = 6/2 which is 3 servers in context to 5
>> >server
>> > ensemble if the ceil(N/2) is a ceiling function?
>> > 3. what does ensemble mean in context to Apache zookeeper?
>> >
>> >I would appreciate your help.
>> >
>> >Regards,
>> >
>> >Kaushal
>>
>>
Re: Meaning of quorum and ensemble
Posted by Kaushal Shriyan <ka...@gmail.com>.
Thanks Chris for the explanation. Please help me understand how would I
implement a 3 node or a 5 node ensemble in a AWS region for example
Singapore Region which has two Availability Zone 1a and 1b if either of the
zone becomes unavailable due to say for example Network Glitch or not
reachable.
Regards,
Kaushal
On 29 May 2015 23:41, "Chris Nauroth" <cn...@hortonworks.com> wrote:
> Hello Kaushal,
>
> 1. "Quorum" refers to the minimum number of nodes that must agree on a
> transaction before it is considered committed.
> 2. Yes, your understanding of ceil(N/2) is correct. It sounds like your
> formulation of it is more like floor((N+1)/2). The two formulations are
> equivalent, so whichever one is more intuitive to you is valid.
> 3. "Ensemble" refers to the full set of peer servers in a ZooKeeper
> cluster.
>
> Tying this back to your examples, a 3-node ensemble requires a quorum of 2
> servers running to commit a transaction. A 5-node ensemble requires a
> quorum of 3 servers running to commit a transaction. An administrator can
> deploy a ZooKeeper cluster sized according to trade-offs between budget
> (number of servers) and desired fault tolerance (number of simultaneous
> server failures). A more detailed discussion is available in the
> ZooKeeper Internals document:
>
> http://zookeeper.apache.org/doc/r3.4.6/zookeeperInternals.html
>
>
> I hope this helps.
>
> --Chris Nauroth
>
>
>
>
> On 5/29/15, 10:08 AM, "Kaushal Shriyan" <ka...@gmail.com> wrote:
>
> >Hi,
> >
> >I have a question regarding quorum in context to
> >
> http://stackoverflow.com/questions/13022244/zookeeper-reliability-three-ve
> >rsus-five-nodes
> >
> >Zookeeper requires that you have a quorum of servers up, where quorum is
> >ceil(N/2). For a 3 server ensemble, that means 2 servers must be up at any
> >time, for a 5 server ensemble, 3 servers need to be up at any time.
> >
> > 1. what does quorum mean?
> > 2. so does ceil(N/2) mean 3+1/2 = 4/2 which is 2 servers in context to
> >3
> > server ensemble and 5+1/2 = 6/2 which is 3 servers in context to 5
> >server
> > ensemble if the ceil(N/2) is a ceiling function?
> > 3. what does ensemble mean in context to Apache zookeeper?
> >
> >I would appreciate your help.
> >
> >Regards,
> >
> >Kaushal
>
>
Re: Meaning of quorum and ensemble
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hello Kaushal,
1. "Quorum" refers to the minimum number of nodes that must agree on a
transaction before it is considered committed.
2. Yes, your understanding of ceil(N/2) is correct. It sounds like your
formulation of it is more like floor((N+1)/2). The two formulations are
equivalent, so whichever one is more intuitive to you is valid.
3. "Ensemble" refers to the full set of peer servers in a ZooKeeper
cluster.
Tying this back to your examples, a 3-node ensemble requires a quorum of 2
servers running to commit a transaction. A 5-node ensemble requires a
quorum of 3 servers running to commit a transaction. An administrator can
deploy a ZooKeeper cluster sized according to trade-offs between budget
(number of servers) and desired fault tolerance (number of simultaneous
server failures). A more detailed discussion is available in the
ZooKeeper Internals document:
http://zookeeper.apache.org/doc/r3.4.6/zookeeperInternals.html
I hope this helps.
--Chris Nauroth
On 5/29/15, 10:08 AM, "Kaushal Shriyan" <ka...@gmail.com> wrote:
>Hi,
>
>I have a question regarding quorum in context to
>http://stackoverflow.com/questions/13022244/zookeeper-reliability-three-ve
>rsus-five-nodes
>
>Zookeeper requires that you have a quorum of servers up, where quorum is
>ceil(N/2). For a 3 server ensemble, that means 2 servers must be up at any
>time, for a 5 server ensemble, 3 servers need to be up at any time.
>
> 1. what does quorum mean?
> 2. so does ceil(N/2) mean 3+1/2 = 4/2 which is 2 servers in context to
>3
> server ensemble and 5+1/2 = 6/2 which is 3 servers in context to 5
>server
> ensemble if the ceil(N/2) is a ceiling function?
> 3. what does ensemble mean in context to Apache zookeeper?
>
>I would appreciate your help.
>
>Regards,
>
>Kaushal