You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Matthias J. Sax (Jira)" <ji...@apache.org> on 2024/02/05 00:48:00 UTC
[jira] [Updated] (KAFKA-16221) IllegalStateException from Producer
[ https://issues.apache.org/jira/browse/KAFKA-16221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias J. Sax updated KAFKA-16221:
------------------------------------
Description:
https://issues.apache.org/jira/browse/KAFKA-14831 fixed a producer bug about internal TX state transition and the producer is now throwing an IllegalStateException in situations it did swallow an internal error before.
This change surfaces a bug in Kafka Streams: Kafka Streams calls `abortTransaction()` blindly when a task is closed dirty, even if the Producer is already in an internal fatal state. However, if the Producer is in a fatal state, Kafka Streams should skip `abortTransaction` and only `close()` the Producer when closing a task dirty.
The bug is surfaced after `commitTransaction()` did timeout and another error happens leading to the call to `abortTransaction()` – Kafka Streams does not track right now if a commit-TX is in progress.
If the Producer throws an IllegalStateException on `abortTransaction()` Kafka Streams treats this exception ("correctly") as fatal, and StreamsThread dies. However, Kafka Streams is actually in a state in which it can recover from, and thus should not let StreamThread die by carry forward (by not calling `abortTransaction()` and moving forward with the dirty close of the task).
was:
https://issues.apache.org/jira/browse/KAFKA-14831 fixed a producer bug about internal TX state transition and the producer is now throwing an IllegalStateException in situations it did swallow an internal error before.
This change surfaces a bug in Kafka Streams: Kafka Streams calls `abortTransaction()` blindly when a task is closed dirty, even if the Producer is already in an internal fatal state. However, if the Producer is in a fatal state, Kafka Streams should skip `abortTransaction` and only `close()` the Producer when closing a task dirty.
If the Producer throws an IllegalStateException on `abortTransaction()` Kafka Streams treats this exception ("correctly") as fatal, and StreamsThread dies. However, Kafka Streams is actually in a state in which it can recover from, and thus should not let StreamThread die by carry forward (by not calling `abortTransaction()` and moving forward with the dirty close of the task).
> IllegalStateException from Producer
> -----------------------------------
>
> Key: KAFKA-16221
> URL: https://issues.apache.org/jira/browse/KAFKA-16221
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 3.6.0
> Reporter: Matthias J. Sax
> Priority: Critical
>
> https://issues.apache.org/jira/browse/KAFKA-14831 fixed a producer bug about internal TX state transition and the producer is now throwing an IllegalStateException in situations it did swallow an internal error before.
> This change surfaces a bug in Kafka Streams: Kafka Streams calls `abortTransaction()` blindly when a task is closed dirty, even if the Producer is already in an internal fatal state. However, if the Producer is in a fatal state, Kafka Streams should skip `abortTransaction` and only `close()` the Producer when closing a task dirty.
> The bug is surfaced after `commitTransaction()` did timeout and another error happens leading to the call to `abortTransaction()` – Kafka Streams does not track right now if a commit-TX is in progress.
> If the Producer throws an IllegalStateException on `abortTransaction()` Kafka Streams treats this exception ("correctly") as fatal, and StreamsThread dies. However, Kafka Streams is actually in a state in which it can recover from, and thus should not let StreamThread die by carry forward (by not calling `abortTransaction()` and moving forward with the dirty close of the task).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)