You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Flink CDC Issue Import (Jira)" <ji...@apache.org> on 2024/03/20 09:21:00 UTC

[jira] [Created] (FLINK-34782) Multiple source read binlogs of the same job are all assigned to SubTask-0, which may have performance problems.

Flink CDC Issue Import created FLINK-34782:
----------------------------------------------

             Summary: Multiple source read binlogs of the same job are all assigned to SubTask-0, which may have performance problems.
                 Key: FLINK-34782
                 URL: https://issues.apache.org/jira/browse/FLINK-34782
             Project: Flink
          Issue Type: Improvement
          Components: Flink CDC
            Reporter: Flink CDC Issue Import


### Search before asking

- [X] I searched in the [issues|https://github.com/ververica/flink-cdc-connectors/issues] and found nothing similar.


### Motivation

Multiple source read binlogs of the same job are all assigned to SubTask-0, which may have performance problems.
<img width="1242" alt="image" src="https://user-images.githubusercontent.com/5699014/226238858-bd3bbc63-d7b6-4e1f-b0ad-86c6d6df3f9b.png">
<img width="1200" alt="image" src="https://user-images.githubusercontent.com/5699014/226238965-f3ed29ed-4b21-4feb-8db1-9070a3af7a2f.png">
As shown in the above two figures, the jobs running in our production environment have several sources, and all sources are assigned to the same subtask. When there are many sources, performance problems may occur.

### Solution

When a job has multiple subtasks, different sources can be randomly assigned to different subtasks for execution to prevent all sources from being executed on one subtask.

### Alternatives

_No response_

### Anything else?

_No response_

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

---------------- Imported from GitHub ----------------
Url: https://github.com/apache/flink-cdc/issues/2027
Created by: [lintingbin|https://github.com/lintingbin]
Labels: enhancement, 
Created at: Mon Mar 20 11:26:04 CST 2023
State: open




--
This message was sent by Atlassian Jira
(v8.20.10#820010)