[jira] [Comment Edited] (KAFKA-12776) Producer sends messages out-of-order inspite of enabling idempotence

["NEERAJ VAIDYA (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344517#comment-17344517 ] 

NEERAJ VAIDYA edited comment on KAFKA-12776 at 5/14/21, 12:51 PM:
------------------------------------------------------------------

Hi Tom

The topic has just 1 partition.

And just clarify, all messages have the same key. Yet they get reordered.


was (Author: neeraj.vaidya):
Hi Tom

The topic has just 1 partition.

> Producer sends messages out-of-order inspite of enabling idempotence
> --------------------------------------------------------------------
>
>                 Key: KAFKA-12776
>                 URL: https://issues.apache.org/jira/browse/KAFKA-12776
>             Project: Kafka
>          Issue Type: Bug
>          Components: producer 
>    Affects Versions: 2.6.0, 2.7.0
>         Environment: Linux RHEL 7.9 and Ubuntu 20.04
>            Reporter: NEERAJ VAIDYA
>            Priority: Major
>         Attachments: mocker.zip
>
>
> I have an Apache Kafka 2.6 Producer which writes to topic-A (TA). 
> My application is basically a Spring boot web-application which accepts JSON payloads via HTTP and then pushes each to a Kafka topic. I also use Spring Cloud Stream Kafka in the application to create and use a Producer.
> For one of my failure handling test cases, I shutdown the Kafka cluster while my applications are running. (Note : No messages have been published to the Kafka cluster before I stop the cluster)
> When the producer application tries to write messages to TA, it cannot because the cluster is down and hence (I assume) buffers the messages. Let's say it receives 4 messages m1,m2,m3,m4 in increasing time order. (i.e. m1 is first and m4 is last).
> When I bring the Kafka cluster back online, the producer sends the buffered messages to the topic, but they are not in order. I receive for example, m2 then m3 then m1 and then m4.
> Why is that ? Is it because the buffering in the producer is multi-threaded with each producing to the topic at the same time ?
> My project code is attached herewith.
> I can confirm that I have enabled idempotence. I have also tried with ```max.in.flight.requests=1```



--
This message was sent by Atlassian Jira
(v8.3.4#803005)