You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by "Jason Gustafson (Jira)" <ji...@apache.org> on 2020/11/09 18:36:00 UTC

[jira] [Created] (KAFKA-10702) Slow replication of empty transactions

Jason Gustafson created KAFKA-10702:
---------------------------------------

             Summary: Slow replication of empty transactions
                 Key: KAFKA-10702
                 URL: https://issues.apache.org/jira/browse/KAFKA-10702
             Project: Kafka
          Issue Type: Bug
            Reporter: Jason Gustafson
            Assignee: Jason Gustafson


We hit a case in which we had to re-replicate a compacted topic from the beginning of the log. Some portions of the log consisted mostly of transaction markers, which were extremely slow to replicate. The problem is that `ProducerStateManager` adds all of these empty transactions to its internal collection of `ongoingTxns` before immediately removing them. There could be tens of thousands of empty transactions in the worst case from a single `Fetch` response, so this can create a huge amount of pressure on the broker. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)