You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Qingsheng Ren (Jira)" <ji...@apache.org> on 2024/08/09 09:31:00 UTC

[jira] [Assigned] (FLINK-35442) Kafka Pipeline Sink may distribute the same key to different partitions after modify parallelism

     [ https://issues.apache.org/jira/browse/FLINK-35442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Qingsheng Ren reassigned FLINK-35442:
-------------------------------------

    Assignee: LvYanquan

> Kafka Pipeline Sink may distribute the same key to different partitions after modify parallelism
> ------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-35442
>                 URL: https://issues.apache.org/jira/browse/FLINK-35442
>             Project: Flink
>          Issue Type: Improvement
>          Components: Flink CDC
>    Affects Versions: cdc-3.2.0
>            Reporter: LvYanquan
>            Assignee: LvYanquan
>            Priority: Critical
>              Labels: pull-request-available
>             Fix For: cdc-3.2.0
>
>
>  Currently, Kafka Pipeline Sink partition events by 
> parallelInstanceId % partitions.length，this is because the previous partition operator has already shuffled events by key.
> However, If the parallelism of the task is modified and restarted, data with the same key before and after the task may be in different partitions.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)