[GitHub] [pulsar] kjaggann edited a comment on issue #572: Don't depend on ZooKeeper for coordination

[GitBox <gi...@apache.org>]

kjaggann edited a comment on issue #572:
URL: https://github.com/apache/pulsar/issues/572#issuecomment-638395246


   My understanding zookeeper has an issue in terms of "quorum" and "split Brain"
   my finding correct me if I am wrong lets take we have 5 servers. The voting to elect the leader is based on the majority of the servers. the number of servers must accept specific server to become the leader, the other becomes the followers
   So there is this bad thing that may happen called "split brain". A split brain is simply this, as far as I understand: The cluster of 5 servers splits into two parts, or let's call it "server teams", with maybe one part of 2 and the other of 3 servers. This is really a bad situation as if both "server teams" must execute a specific order how would you decide which team should be preferred? So it is really important to know what "server team" is still relevant and which one can/should be ignored.
   
    When one server in {3,4,5} realizes the leader is gone, it triggers an election. Once a new leader is elected, it starts serving client requests normally. The other partitioned cluster {1,2} won't serve any client request and will be stuck in the election until the partition is resolved. So it waits till the issue resolved and the cluster runs normal again.
   
   Currently, pulsar uses  Apache zookeeper to store its metadata. Data such as the location of partitions and the configuration of topics are stored outside of pulsar itself, in a separate Zookeeper cluster. so is there a plan to break this dependency and bring metadata management into Pulsar itself.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org