[GitHub] [pulsar] SleeperSmith commented on issue #572: Don't depend on ZooKeeper for coordination

[GitBox <gi...@apache.org>]

SleeperSmith commented on issue #572:
URL: https://github.com/apache/pulsar/issues/572#issuecomment-829315109


   Moving raft into Pulsar imo is like asking Wordpress to move mariadb into its own code base. Strongly opposed to the idea as this is just wasting Pulsar dev's time rewriting the raft protocol when a mature implementation already exist. As well, the end result is the same operational burden people are facing are just going to be moved into Pulsar along with it. I'm assuming it's the same old membership/quorum management. Why do people think these are going to magically disappear when Zookeeper is removed but the same feature is required?
   
   I am strongly in favor of supporting etcd / consul in addition to zookeeper because etcd/consul has integration with AWS autoscaling group / k8s coordinator for dynamic membership. Consul has dynamic membership plugin for literally every platform it runs on. This means I can troubleshoot or perform updates and changes via replacing the nodes. That was a lot harder to accomplish with zookeeper. (Well back in the days. Nowadays ZK support dynamic membership too. Maybe used in conjunction with DNS membership discovery would make it easier to manage. Haven't looked into it yet).
   
   tldr, I hope Pulsar devs don't make the same mistake Kafka did. Kafka team are now going to have to be fixing every quirk/bug/edge case of their raft implementation when zk/consul/etcd have already dedicated years of effort into it. Then, we going to see people asking for every extra feature they like from zk/consul/etcd to be reimplemented too.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org