You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "LvYanquan (Jira)" <ji...@apache.org> on 2024/05/24 03:58:00 UTC

[jira] [Created] (FLINK-35442) Kafka Pipeline Sink may distribute the same key to different partitions after modify parallelism

LvYanquan created FLINK-35442:
---------------------------------

             Summary: Kafka Pipeline Sink may distribute the same key to different partitions after modify parallelism
                 Key: FLINK-35442
                 URL: https://issues.apache.org/jira/browse/FLINK-35442
             Project: Flink
          Issue Type: Improvement
          Components: Flink CDC
    Affects Versions: cdc-3.2.0
            Reporter: LvYanquan
             Fix For: cdc-3.2.0


 Currently, Kafka Pipeline Sink partition events by 
parallelInstanceId % partitions.length，this is because the previous partition operator has already shuffled events by key.

However, If the parallelism of the task is modified and restarted, data with the same key before and after the task may be in different partitions.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)