[jira] [Created] (KAFKA-3193) Wait for producer message acks in StreamTask.commit()

["Guozhang Wang (JIRA)" <ji...@apache.org>]

Guozhang Wang created KAFKA-3193:
------------------------------------

             Summary: Wait for producer message acks in StreamTask.commit()
                 Key: KAFKA-3193
                 URL: https://issues.apache.org/jira/browse/KAFKA-3193
             Project: Kafka
          Issue Type: Sub-task
            Reporter: Guozhang Wang


Today in StreamTask.commit(), we does the following:

1) flush local state storage if it is persistent (e.g. RocksDB).
2) flush produced records in the downstream and changelog Kafka topics.
3) commit consumed offsets if it is dirty already

However in step 2) above:

{code}
producer.flush();
{code}

does not actually block on all the acks, but only force draining all records in the buffer and send them. This actually result in a major defect of the commit() call. We'd better add the ability in producer to wait in flushing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)