You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Imesha Sudasingha <im...@cse.mrt.ac.lk> on 2016/10/05 13:09:46 UTC
Zookeeper - Network partitioning behaviour
Hi all,
When I was going through zookeeper, I noticed the following scenarios,
- Suppose we have a 5 member quorum operating as normal. In a network
partitioning where the leader and a member node goes into one partition and
other 3 nodes get into the other network partition. I known then the side
with the majority will elect a new leader and carry on to serve requests.
But will the partition with a minority will also continue to serve read
requests until a write request comes to the leader in the minority
partition? How does zookeeper handle this occasion?
- In the above scenario, what will happen to the "watches" that have
already been registered to a node in the minority partition?
Can you clarify the above scenarios?
Thanks in advance!
--
*Imesha Sudasingha*
Undergraduate of Department of Computer Science and Engineering,
University of Moratuwa.
+94717086160
View in Linkedin <https://lk.linkedin.com/in/imeshasudasingha>
RE: EXTERNAL: Zookeeper - Network partitioning behaviour
Posted by "Daly, Jeff" <je...@lmco.com>.
-----Original Message-----
From: Imesha Sudasingha [mailto:imesha.13@cse.mrt.ac.lk]
Sent: Wednesday, October 05, 2016 9:10 AM
To: user@zookeeper.apache.org
Subject: EXTERNAL: Zookeeper - Network partitioning behaviour
Hi all,
When I was going through zookeeper, I noticed the following scenarios,
- Suppose we have a 5 member quorum operating as normal. In a network
partitioning where the leader and a member node goes into one partition and
other 3 nodes get into the other network partition. I known then the side
with the majority will elect a new leader and carry on to serve requests.
But will the partition with a minority will also continue to serve read
requests until a write request comes to the leader in the minority
partition? How does zookeeper handle this occasion?
- In the above scenario, what will happen to the "watches" that have
already been registered to a node in the minority partition?
Can you clarify the above scenarios?
Thanks in advance!
--
*Imesha Sudasingha*
Undergraduate of Department of Computer Science and Engineering, University of Moratuwa.
+94717086160
View in Linkedin <https://lk.linkedin.com/in/imeshasudasingha>
Re: Zookeeper - Network partitioning behaviour
Posted by Imesha Sudasingha <im...@cse.mrt.ac.lk>.
Hi Rakesh,
Thanks for your great explanation. That is what I wanted to know.
Regards,
Imesha Sudasingha
On 6 October 2016 at 12:25, Rakesh Radhakrishnan <ra...@apache.org> wrote:
> Hi Imesha Sudasingha,
>
> For example, we have A,B,C,D,E five servers, that formed quorum and assume
> A is the Leader. Again assume network partition happened between
> A,B(minority region) and C,D,E. As we know, 3 is the majority factor to
> form quorum. Since Leader A is in the minority region, the entire quorum
> will get shutdown and all the servers will automatically moves to leader
> election phase. Now, A & B will do sending notifications each other and
> will never succeed to form quorum due to <3 factor. On the other side, C,D
> and E will participate each other and elect one of them as Leader.
>
> >>>>>But will the partition with a minority will also continue to serve
> read
> >>>>>requests until a write request comes to the leader in the minority
> >>>>>partition?
> After the partition, all the servers in the minority region will get
> shutdown and moves to leader election phase. All the client sessions
> connected to these servers will be disconnected and
> will receive "KeeperState.Disconnected" event to their watchers, if any
> registered.
>
> But ZooKeeper supports read-only server mode. In this mode, client can
> connect to the read-only server even when the server might be partitioned
> from the quorum.
>
> Reference:- Please read the section "Read Only Mode Server" in the apache
> document link, https://zookeeper.apache.org/doc/r3.4.9/zookeeperAdmin.html
>
>
> >>>>>In the above scenario, what will happen to the "watches" that have
> >>>>>already been registered to a node in the minority partition?
> Since the quorum is re-forming, all the client session watchers will
> receive "KeeperState.Disconnected" event. These client session will start
> sending connection request to all quorum servers(A,B,C,D,E) to re-establish
> the connection. In my above example, C,D,E re-forms quorum successfully and
> client sessions will reconnect to one of these servers in the quorum(am
> assuming zkclient has the C,D,E server host address configured). Clients
> automatically reset watches during successful session reconnect. There is a
> feature to disable this watch resetting, please read the configuration
> "zookeeper.disableAutoWatchReset" section in the following apache doc link
> to understand more on this,
> https://zookeeper.apache.org/doc/trunk/zookeeperProgrammers.html
>
> Regards,
> Rakesh
>
> On Wed, Oct 5, 2016 at 6:39 PM, Imesha Sudasingha <imesha.13@cse.mrt.ac.lk
> >
> wrote:
>
> > Hi all,
> >
> > When I was going through zookeeper, I noticed the following scenarios,
> >
> > - Suppose we have a 5 member quorum operating as normal. In a network
> > partitioning where the leader and a member node goes into one
> partition
> > and
> > other 3 nodes get into the other network partition. I known then the
> > side
> > with the majority will elect a new leader and carry on to serve
> > requests.
> > But will the partition with a minority will also continue to serve
> read
> > requests until a write request comes to the leader in the minority
> > partition? How does zookeeper handle this occasion?
> > - In the above scenario, what will happen to the "watches" that have
> > already been registered to a node in the minority partition?
> >
> > Can you clarify the above scenarios?
> > Thanks in advance!
> >
> > --
> > *Imesha Sudasingha*
> > Undergraduate of Department of Computer Science and Engineering,
> > University of Moratuwa.
> > +94717086160
> > View in Linkedin <https://lk.linkedin.com/in/imeshasudasingha>
> >
>
--
*Imesha Sudasingha*
Undergraduate of Department of Computer Science and Engineering,
University of Moratuwa.
+94717086160
View in Linkedin <https://lk.linkedin.com/in/imeshasudasingha>
Re: Zookeeper - Network partitioning behaviour
Posted by Rakesh Radhakrishnan <ra...@apache.org>.
Hi Imesha Sudasingha,
For example, we have A,B,C,D,E five servers, that formed quorum and assume
A is the Leader. Again assume network partition happened between
A,B(minority region) and C,D,E. As we know, 3 is the majority factor to
form quorum. Since Leader A is in the minority region, the entire quorum
will get shutdown and all the servers will automatically moves to leader
election phase. Now, A & B will do sending notifications each other and
will never succeed to form quorum due to <3 factor. On the other side, C,D
and E will participate each other and elect one of them as Leader.
>>>>>But will the partition with a minority will also continue to serve read
>>>>>requests until a write request comes to the leader in the minority
>>>>>partition?
After the partition, all the servers in the minority region will get
shutdown and moves to leader election phase. All the client sessions
connected to these servers will be disconnected and
will receive "KeeperState.Disconnected" event to their watchers, if any
registered.
But ZooKeeper supports read-only server mode. In this mode, client can
connect to the read-only server even when the server might be partitioned
from the quorum.
Reference:- Please read the section "Read Only Mode Server" in the apache
document link, https://zookeeper.apache.org/doc/r3.4.9/zookeeperAdmin.html
>>>>>In the above scenario, what will happen to the "watches" that have
>>>>>already been registered to a node in the minority partition?
Since the quorum is re-forming, all the client session watchers will
receive "KeeperState.Disconnected" event. These client session will start
sending connection request to all quorum servers(A,B,C,D,E) to re-establish
the connection. In my above example, C,D,E re-forms quorum successfully and
client sessions will reconnect to one of these servers in the quorum(am
assuming zkclient has the C,D,E server host address configured). Clients
automatically reset watches during successful session reconnect. There is a
feature to disable this watch resetting, please read the configuration
"zookeeper.disableAutoWatchReset" section in the following apache doc link
to understand more on this,
https://zookeeper.apache.org/doc/trunk/zookeeperProgrammers.html
Regards,
Rakesh
On Wed, Oct 5, 2016 at 6:39 PM, Imesha Sudasingha <im...@cse.mrt.ac.lk>
wrote:
> Hi all,
>
> When I was going through zookeeper, I noticed the following scenarios,
>
> - Suppose we have a 5 member quorum operating as normal. In a network
> partitioning where the leader and a member node goes into one partition
> and
> other 3 nodes get into the other network partition. I known then the
> side
> with the majority will elect a new leader and carry on to serve
> requests.
> But will the partition with a minority will also continue to serve read
> requests until a write request comes to the leader in the minority
> partition? How does zookeeper handle this occasion?
> - In the above scenario, what will happen to the "watches" that have
> already been registered to a node in the minority partition?
>
> Can you clarify the above scenarios?
> Thanks in advance!
>
> --
> *Imesha Sudasingha*
> Undergraduate of Department of Computer Science and Engineering,
> University of Moratuwa.
> +94717086160
> View in Linkedin <https://lk.linkedin.com/in/imeshasudasingha>
>
RE: EXTERNAL: Zookeeper - Network partitioning behaviour
Posted by "Daly, Jeff" <je...@lmco.com>.
410-379-2411
-----Original Message-----
From: Imesha Sudasingha [mailto:imesha.13@cse.mrt.ac.lk]
Sent: Wednesday, October 05, 2016 9:10 AM
To: user@zookeeper.apache.org
Subject: EXTERNAL: Zookeeper - Network partitioning behaviour
Hi all,
When I was going through zookeeper, I noticed the following scenarios,
- Suppose we have a 5 member quorum operating as normal. In a network
partitioning where the leader and a member node goes into one partition and
other 3 nodes get into the other network partition. I known then the side
with the majority will elect a new leader and carry on to serve requests.
But will the partition with a minority will also continue to serve read
requests until a write request comes to the leader in the minority
partition? How does zookeeper handle this occasion?
- In the above scenario, what will happen to the "watches" that have
already been registered to a node in the minority partition?
Can you clarify the above scenarios?
Thanks in advance!
--
*Imesha Sudasingha*
Undergraduate of Department of Computer Science and Engineering, University of Moratuwa.
+94717086160
View in Linkedin <https://lk.linkedin.com/in/imeshasudasingha>