[jira] [Created] (KAFKA-14288) Processor topology in tasks is updated with internal intermediate topics without application ID prefix

["Bruno Cadonna (Jira)" <ji...@apache.org>]

Bruno Cadonna created KAFKA-14288:
-------------------------------------

             Summary: Processor topology in tasks is updated with internal intermediate topics without application ID prefix
                 Key: KAFKA-14288
                 URL: https://issues.apache.org/jira/browse/KAFKA-14288
             Project: Kafka
          Issue Type: Bug
          Components: streams
    Affects Versions: 3.3.1
            Reporter: Bruno Cadonna


Updating the input partitions of tasks during assignment handling also updates the mapping from source nodes to input topics in the processor topology within the task. The mapping is updated with the topics from the topology metadata. The topology metadata does not prefix internal intermediate topics with the application ID. Thus, if a standby task has input partitions from an internal intermediate topic the update of the mapping in the processor topology leads to an invalid topology exception during recycling of a standby task to an active task when the input queues are created. This is because the input topics in the processor topology and the input partitions of the task do not match because the former miss the application ID prefix.
The case were standby tasks have input partitions only from internal intermediate topics can be fixed by checking if the input partitions really changed before updating the input partitions (see PR https://github.com/apache/kafka/pull/12730). Unfortunately, a subtopology might have input partitions subscribed to with a regex additionally to internal intermediate topics which might still lead to an invalid topology exception during recycling irrespectively of the aforementioned verification.

This bug might also affect active tasks.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)