[jira] [Commented] (KAFKA-9878) Block EndTxn call until the txn markers are committed

["Guozhang Wang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-9878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17085171#comment-17085171 ] 

Guozhang Wang commented on KAFKA-9878:
--------------------------------------

The original rationale is that if after committing the current txn there are some time gap before the producer send the next record for the new txn, then this time gap can be "pipelined". So theoretically I see both pros and cons of this proposal here. In practice, it is likely that the time gap we were thinking to be pipeline is small. Combining with KAFKA-9860, I think maybe we can instead make the beginTxn call a blocking call which would be sending the batched addPartitions request, and which could be blocked until the previous transaction completes --- so that the time before the previous commitTxn call and the next beginTxn call can still be overlapped.



> Block EndTxn call until the txn markers are committed
> -----------------------------------------------------
>
>                 Key: KAFKA-9878
>                 URL: https://issues.apache.org/jira/browse/KAFKA-9878
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Boyang Chen
>            Priority: Major
>
> Currently the EndTxn call from Producer will immediately return as the control record is written to the txn coordinator log. The ongoing transaction will be going to a pending state to wait for all txn markers to be propagated. In the meantime, producer client will start another new transaction but being rejected constantly until the pending state gets resolved, which is unnecessary round trips and more burden to the broker to handle repetitive requests.
> To avoid this situation, we should make the Producer client wait for txn marker completion instead. This will incur better performance overall, as no more back-off shall be triggered for a subsequent transaction to begin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)