[jira] [Created] (FLINK-36813) MySQLCDC supports synchronization of specified fields

["Di Wu (Jira)" <ji...@apache.org>]

Di Wu created FLINK-36813:
-----------------------------

             Summary: MySQLCDC supports synchronization of specified fields
                 Key: FLINK-36813
                 URL: https://issues.apache.org/jira/browse/FLINK-36813
             Project: Flink
          Issue Type: Improvement
          Components: Flink CDC
    Affects Versions: cdc-3.2.1
            Reporter: Di Wu
             Fix For: cdc-3.3.0


*Background*
In some scenarios, MySQL synchronization only expects to synchronize specified fields instead of all fields in the table.
1. The user only has the permission for some fields in MySQL
2. The user has too many fields in a single table and only wants to synchronize some fields, for example, here https://github.com/apache/flink-cdc/discussions/3058

*Current situation*
For the incremental stage, you only need to configure the column.include.list property of debezium to support the synchronization of some fields in the incremental stage, refer to: https://debezium.io/documentation/reference/1.9/connectors/mysql.html#mysql-property-column-include-list

For the full snapshot stage, * is currently used in {_}MySqlSnapshotSplitReadTask{_}, refer to
{code:java}
if (isScanningData) {
return buildSelectWithRowLimits(
tableId, limitSize, "*", Optional.ofNullable(condition), Optional.empty()); {code}
 

*Solution*
The user configures {_}column.include.list{_}, and then captures the specific columns in MySqlSnapshotSplitReadTask, and splices them when constructing Scan SQL.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)