[jira] [Updated] (KAFKA-9733) Consider addition of leader quorum in replication model

["Richard Yu (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-9733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Richard Yu updated KAFKA-9733:
------------------------------
    Summary: Consider addition of leader quorum in replication model  (was: Consider addition of leader partition quorum)

> Consider addition of leader quorum in replication model
> -------------------------------------------------------
>
>                 Key: KAFKA-9733
>                 URL: https://issues.apache.org/jira/browse/KAFKA-9733
>             Project: Kafka
>          Issue Type: New Feature
>          Components: clients, core
>            Reporter: Richard Yu
>            Priority: Minor
>
> Note: Description still not finished. Still not sure if this is needed.
> Kafka's current replication model (with its single leader and several followers) is somewhat similar to the current consensus algorithms being used in databases (RAFT) with the major difference being the existence of the ISR. Consequently, Kafka suffers from the same fault tolerance issues as does other distributed systems which rely on RAFT: the leader tends to be the chokepoint for failures i.e. if it goes down, it will have a brief stop-the-world effect. 
> In contrast, giving all replicas the power to write and read to other replicas is also difficult to accomplish (as emphasized by the complexity of the Egalitarian Paxos algorithm), since consistency is so hard to maintain in such an algorithm, plus very little gain compared to the overhead. 
> Therefore, I propose that we have an intermediate step in between these two algorithms, and that is the leader partition quorum.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)