You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Bill Bejeck (Jira)" <ji...@apache.org> on 2026/07/24 13:38:00 UTC
[jira] [Commented] (KAFKA-20827) StreamThread dies with IllegalStateException: Attempted to create an standby task that we already own during a standby/active recycle
[ https://issues.apache.org/jira/browse/KAFKA-20827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18098897#comment-18098897 ]
Bill Bejeck commented on KAFKA-20827:
-------------------------------------
I've been able to reproduce this under a local chaos soak (2 EOS-v2 instances, rapid rebalances via stream-thread shedding + wire-protocol fault injection)
It is not limited to num.standby.replicas >= 1. With num.standby.replicas=1 it fails on the standby insert; with num.standby.replicas=0 the same defect surfaces on the active insert:
{code:java}
Caused by: java.lang.IllegalStateException: Attempted to create an active task that we already own: 0_4 at org.apache.kafka.streams.processor.internals.Tasks.addActiveTask(Tasks.java:202) at org.apache.kafka.streams.processor.internals.Tasks.addTask(Tasks.java:191) at org.apache.kafka.streams.processor.internals.Tasks.addFailedTask(Tasks.java:234) at org.apache.kafka.streams.processor.internals.TaskManager.addTasksToStateUpdater(TaskManager.java:937)
at org.apache.kafka.streams.processor.internals.TaskManager.checkStateUpdater(TaskManager.java:848) {code}
Both variants take the identical path — only the final addActiveTask vs addStandbyTask differs — so this is role-agnostic bookkeeping, not standby-specific.
Mechanism: pendingTasksToInit is a Set<Task> keyed by object identity, so two distinct Task instances for the same id can coexist. When both are drained by addTasksToStateUpdater and both fail initialization, the first addFailedTask records the id in the owned map, and the second addFailedTask → addTask hits the single-owner invariant in addActiveTask/addStandbyTask and throws — escaping as a hard StreamThread fatal rather than per-task failure handling.
> StreamThread dies with IllegalStateException: Attempted to create an standby task that we already own during a standby/active recycle
> -------------------------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-20827
> URL: https://issues.apache.org/jira/browse/KAFKA-20827
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: Alan Lau
> Assignee: Alan Lau
> Priority: Critical
>
> With num.standby.replicas >= 1, a StreamThread can be killed by a fatal IllegalStateException from the task registry while a task recycles between active and standby:
> {code:java}
> java.lang.IllegalStateException: Attempted to create an standby task that we already own: 0_2
> at org.apache.kafka.streams.processor.internals.Tasks.addStandbyTask(Tasks.java:221)
> at org.apache.kafka.streams.processor.internals.Tasks.addTask(Tasks.java:193)
> at org.apache.kafka.streams.processor.internals.Tasks.addFailedTask(Tasks.java:234)
> at org.apache.kafka.streams.processor.internals.TaskManager.addTasksToStateUpdater(TaskManager.java:944)
> at org.apache.kafka.streams.processor.internals.TaskManager.checkStateUpdater(TaskManager.java:855)
> at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:1042)
> {code}
> A task whose initialization fails is left owned (registered in Tasks and flagged in failedTaskIds) to be reconciled via the corruption/failed-task path.
> handleAssignment's rectify-existing pass (handleTasks) iterates allNonFailedInitializedTasks(), which excludes failed tasks so the owned-but-failed task is not recognized as already-owned and is not removed from the ...ToCreate maps. createNewTasks then builds a second representation of it; that duplicate fails initialization again, and addTasksToStateUpdater -> Tasks.addFailedTask -> addStandbyTask/addActiveTask trips the single-owner invariant and kills the StreamThread. The throw also happens before the real init exception is recorded, masking it.
> The StreamThread should keep running and only the affected task briefly re-restores before resuming, with no impact on the other tasks on that thread. Instead, it throws a IllegalStateException that kills the entire StreamThread.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)