You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Mihai Lucian (Jira)" <ji...@apache.org> on 2025/07/08 09:24:00 UTC
[jira] [Created] (KAFKA-19479) at_least_once mode in Kafka Streams silently drops messages when the producer fails with MESSAGE_TOO_LARGE, violating delivery guarantees.
Mihai Lucian created KAFKA-19479:
------------------------------------
Summary: at_least_once mode in Kafka Streams silently drops messages when the producer fails with MESSAGE_TOO_LARGE, violating delivery guarantees.
Key: KAFKA-19479
URL: https://issues.apache.org/jira/browse/KAFKA-19479
Project: Kafka
Issue Type: Bug
Components: streams
Affects Versions: 4.0.0
Reporter: Mihai Lucian
Assignee: Mihai Lucian
*Description*
It appears there is a scenario where Kafka Streams running with {{processing.guarantee=at_least_once}} does {*}not uphold its delivery guarantees{*}, resulting in *message loss.*
*Reproduction Details*
*We run a simple Kafka Streams topology like the following:*
*kotlin```*
props[StreamsConfig.APPLICATION_ID_CONFIG] = "poc-at-least-once"
props[StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG] = Serdes.String().javaClass.name
props[StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG] = Serdes.String().javaClass.name
props[StreamsConfig.PROCESSING_GUARANTEE_CONFIG] = StreamsConfig.AT_LEAST_ONCE
// Large producer batch size to induce MESSAGE_TOO_LARGE
props[ProducerConfig.LINGER_MS_CONFIG] = "300000"
props[ProducerConfig.BATCH_SIZE_CONFIG] = "33554432"
// Custom handler registered (never triggered)
props[StreamsConfig.PRODUCTION_EXCEPTION_HANDLER_CLASS_CONFIG] = "poc.MyProductionExceptionHandler"
val stream = streamsBuilder.stream<String, String>("input.topic")
stream.peek \{ key, value -> println("$key:$value") }
.to("output.topic")
```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)