[jira] [Created] (KAFKA-5826) Transactional Producer needs another way to control transactional expiration

["Canes Kelly (JIRA)" <ji...@apache.org>]

Canes Kelly created KAFKA-5826:
----------------------------------

             Summary: Transactional Producer needs another way to control transactional expiration
                 Key: KAFKA-5826
                 URL: https://issues.apache.org/jira/browse/KAFKA-5826
             Project: Kafka
          Issue Type: Improvement
          Components: clients, core
    Affects Versions: 0.11.0.0
            Reporter: Canes Kelly
            Priority: Minor


Since transactional producer can send message to several TopicPartitions, it's hard to customize transaction timeout by the user when starts a transaction.

In current version of kafka, users who want to send message with transaction should set the timeout configuration which indicates the max time interval between the beginning transaction time with the expiration time. But it's hard to say how long is the most appropriate time to wait for expiration. If timeout is too short, normal appending could be abort because of the large amount of TopicPartitions needed to append. If timeout is too long, consumer must wait for the LSO updated, which means the decline of performance. So the timeout is hard to be determined by users.

However, connection expiration time of producer and cluster is easier to determine.So I propose to try the other way to calculate the timeout of a transaction.
1. Use heartbeat timeout instead of transaction timeout. Like consumer, we can make producer to send heartbeat to prove it's alive. Then we can use the way of consumer heartbeat to calculate expiration.
2. update the timestamp of a transaction when producer sends a ProduceRequest to append message. Then we can make *latest timestamp + timeout < now * as the timout flag



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)