[jira] [Created] (FLINK-41088) DynamicKafkaSource loses or rejects splits whose formatted split ids collide

["Martijn Visser (Jira)" <ji...@apache.org>]

Martijn Visser created FLINK-41088:
--------------------------------------

             Summary: DynamicKafkaSource loses or rejects splits whose formatted split ids collide
                 Key: FLINK-41088
                 URL: https://issues.apache.org/jira/browse/FLINK-41088
             Project: Flink
          Issue Type: Bug
          Components: Connectors / Kafka
    Affects Versions: kafka-5.1.0
            Reporter: Martijn Visser
            Assignee: Martijn Visser


DynamicKafkaSourceSplit ids are {{<cluster id>-<topic>-<partition>}}, so cluster {{a}} with topic {{b-c}} and cluster {{a-b}} with topic {{c}} both give {{a-b-c-0}}. The enumerator's recovery path still keys reports by that id: {{ReaderRecoveryGate}} merges a re-registering reader's reports by {{splitId()}} and keeps only one of the two, and {{DynamicKafkaSourceEnumerator#reassignReportedSplits}} fails the restore with "Split a-b-c-0 was reported by both reader 0 and reader 1" when both are active. In bounded mode the dropped split is never read and the readers still get no-more-splits.

Both reproduce on main (e47cf425) and with FLINK-40710, which already keys ownership and the restarted-reader merge by cluster id plus topic partition. The fix is the same key in the gate and the initial reassignment.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)