You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Kyle Kingsbury <ap...@jepsen.io> on 2024/11/12 16:04:05 UTC

Kafka transactions allow aborted reads, lost writes, and torn transactions

Hello all,

I've spent the last few months testing Bufstream, a Kafka-compatible 
system. In the course of that research, we discovered that the Kafka 
transaction protocol allows aborted reads, lost writes, and torn 
transactions:

https://jepsen.io/analyses/bufstream-0.1.0

In short, the protocol assumes that message delivery is ordered, but 
sends messages over different TCP connections, to different nodes, with 
automatic retries. When network or node hiccups (e.g. garbage 
collection) delay delivery of a commit or abort message, that message 
can commit or abort a different, later transaction. Committed 
transactions can actually be lost. Aborted transactions can actually 
succeed. Transactions can be torn into parts: some of their effects 
committed, others lost.

We've reproduced these problems in both Bufstream and Kafka itself, and 
we believe every Kafka-compatible system is most likely susceptible. 
KIP-890 may help. Client maintainers may also be able to defend against 
this problem by re-initializing producers on indefinite errors, like RPC 
timeouts.

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=235834631#KIP890:TransactionsServerSideDefense-BumpEpochonEachTransactionforNewClients(1).

Yours truly,

--Kyle