[jira] [Assigned] (KAFKA-9860) Transactional Producer could pre-add partitions

["Boyang Chen (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-9860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Boyang Chen reassigned KAFKA-9860:
----------------------------------

    Assignee:     (was: Boyang Chen)

> Transactional Producer could pre-add partitions
> -----------------------------------------------
>
>                 Key: KAFKA-9860
>                 URL: https://issues.apache.org/jira/browse/KAFKA-9860
>             Project: Kafka
>          Issue Type: Improvement
>          Components: producer 
>            Reporter: Boyang Chen
>            Priority: Major
>              Labels: need-kip
>
> As of today, the Producer transaction manager bookkeeps the partitions involved with current transaction. Each time it sees a new partition, it will try to send a request to add all the involved partitions to the broker, which results in multiple requests. If we could batch the work at the beginning of the transaction, we save unnecessary round trips.
> The idea is that most times the output partitions for a Producer is constant overtime, so we could leverage the last transactions affected partitions to do a batch `AddPartitionToTxn` first, and bump the EndTxn request with a field of partitions actually being added. The transaction coordinator will only send markers to the partitions included in the EndTxn. If the first batch is not a superset of affected partitions as we are producing data, we would still need a second AddPartition call. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)