You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by "Abhishek Mendhekar (JIRA)" <ji...@apache.org> on 2019/03/01 05:55:00 UTC

[jira] [Created] (KAFKA-8021) KafkaProducer.flush() can show unexpected behavior when a batch is split

Abhishek Mendhekar created KAFKA-8021:
-----------------------------------------

             Summary: KafkaProducer.flush() can show unexpected behavior when a batch is split
                 Key: KAFKA-8021
                 URL: https://issues.apache.org/jira/browse/KAFKA-8021
             Project: Kafka
          Issue Type: Bug
          Components: clients, producer 
    Affects Versions: 0.11.0.0
            Reporter: Abhishek Mendhekar
            Assignee: Abhishek Mendhekar


KafkaProducer.flush() marks the flush in progress and then waits for all incomplete batches to be completed (waits on the producer batch futures to finish).

The behavior is seen when a batch is split due to MESSAGE_TOO_LARGE exception.

The large batch is split into smaller batches (2 or more) but ProducerBatch.split() marks the large batch future as complete before adding the new batches in the incomplete list of batches. At this time if the KafkaProducer.flush() is called then it'll make a copy of existing incomplete list of batches and waits for them to complete while ignoring the large batch that was split into smaller batches.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)