You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Uladzislau Blok (Jira)" <ji...@apache.org> on 2026/08/02 11:10:00 UTC

[jira] [Commented] (KAFKA-20547) Streams produces duplicates with EOS_v2

    [ https://issues.apache.org/jira/browse/KAFKA-20547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18101041#comment-18101041 ] 

Uladzislau Blok commented on KAFKA-20547:
-----------------------------------------

[~nikita-shupletsov] 

> were offsets 31316 and 31317 actually committed?

I believe so. Although I don't (and won't) have access to the broker logs, the outputs generated by those messages—where the Kafka Streams topology picked them up and mapped them to commands—were successfully processed downstream by Service C (which runs with {{{}EOS_v2{}}}).

 

> it looks like they may have been aborted due to the failure at the offset 39332. 
> and the picture also says at step3 that it picks up from the last committed offset.

> where were the duplicates? in the sink of  service B?

Yes, offsets 31316 and 31317 were processed twice (which is expected during a replay/rebalance).

However, if the first attempt was indeed aborted, that initial output should never have been visible to downstream EOS components (because of {{{}isolation.level = read_committed{}}}. ) The issue is that I observed the "aborted" transaction's output being read and processed by Service C anyway.

> Streams produces duplicates with EOS_v2
> ---------------------------------------
>
>                 Key: KAFKA-20547
>                 URL: https://issues.apache.org/jira/browse/KAFKA-20547
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 3.9.1
>            Reporter: Uladzislau Blok
>            Assignee: Nikita Shupletsov
>            Priority: Major
>         Attachments: Untitled-2026-04-28-1158.excalidraw-1.png
>
>
> Hello All,
> I saw an issue when Kafka Streams processed same input twice resulting in duplicated output, what breaks EOS
> See attached diagram describing what went wrong 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)