You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "sd4324530 (via GitHub)" <gi...@apache.org> on 2026/02/24 08:09:11 UTC
[PR] [FLINK-38244][hotfix] Fix the full snapshot phase, the field case is adjusted based on isTableIdCaseInsensitive [flink-cdc]
sd4324530 opened a new pull request, #4284:
URL: https://github.com/apache/flink-cdc/pull/4284
During the full snapshot phase, the field case is adjusted based on `isTableIdCaseInsensitive`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38244][hotfix] Fix the full snapshot phase, the field case is adjusted based on isTableIdCaseInsensitive [flink-cdc]
Posted by "yunhaoww (via GitHub)" <gi...@apache.org>.
yunhaoww commented on PR #4284:
URL: https://github.com/apache/flink-cdc/pull/4284#issuecomment-4386281180
@sd4324530
你好, 这次的 PR 引入了新的 BUG
--------
Flink CDC 3.6.0 在跑 MySQL → 下游的 pipeline 任务时,如果同时满足:
- 源 MySQL 的 lower_case_table_names != 0(即 1 或 2,macOS / Windows 默认;部分云上
RDS 也这么配)
- 源表 DDL 里有非全小写的列名(大写 / 驼峰均可)
- YAML pipeline 配了 transform 块,且 projection / primary-keys 按 DDL
原大小写引用了这些列
任务在 INITIAL snapshot 阶段就崩,0 行入库,错误堆栈是 Column 'XXX' not found in any
table,且故障 CreateTableEvent 的 schema 是空列 columns={} 但 primaryKeys
是有值的。任务进入崩溃循环,自动重启也不会自愈。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38244][hotfix] Fix the full snapshot phase, the field case is adjusted based on isTableIdCaseInsensitive [flink-cdc]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #4284:
URL: https://github.com/apache/flink-cdc/pull/4284#discussion_r2889640810
##########
flink-cdc-connect/flink-cdc-pipeline-connectors/flink-cdc-pipeline-connector-mysql/src/test/resources/ddl/inventory.sql:
##########
@@ -37,6 +37,26 @@ VALUES (default,"scooter","Small 2-wheel scooter",3.14),
(default,"jacket","water resistent black wind breaker",0.1),
(default,"spare tire","24 inch spare tire",22.2);
+-- Create a table where all fields are in uppercase.
+CREATE TABLE uppercase_products (
+ ID INTEGER NOT NULL AUTO_INCREMENT PRIMARY KEY,
+ NAME VARCHAR(255) NOT NULL DEFAULT 'flink',
+ DESCRIPTION VARCHAR(512),
+ WEIGHT FLOAT(6)
+);
+ALTER TABLE uppercase_products AUTO_INCREMENT = 101;
+
+INSERT INTO uppercase_products
+VALUES (default,"scooter","Small 2-wheel scooter",3.14),
+ (default,"car battery","12V car battery",8.1),
+ (default,"12-pack drill bits","12-pack of drill bits with sizes ranging from #40 to #3",0.8),
+ (default,"hammer","12oz carpenter's hammer",0.75),
+ (default,"hammer","14oz carpenter's hammer",0.875),
+ (default,"hammer","16oz carpenter's hammer",1.0),
+ (default,"rocks","box of assorted rocks",5.3),
+ (default,"jacket","water resistent black wind breaker",0.1),
Review Comment:
Typo in inserted test data: "water resistent" is misspelled and should be "water resistant" (even if this is only test DDL, keeping strings correct avoids propagating typos across fixtures).
##########
flink-cdc-connect/flink-cdc-pipeline-connectors/flink-cdc-pipeline-connector-mysql/src/main/java/org/apache/flink/cdc/connectors/mysql/source/reader/MySqlPipelineRecordEmitter.java:
##########
@@ -261,7 +266,10 @@ private Schema buildSchemaFromTable(Table table) {
for (int i = 0; i < columns.size(); i++) {
Column column = columns.get(i);
- String colName = column.name();
+ String colName =
+ this.isTableIdCaseInsensitive
+ ? column.name().toLowerCase(Locale.ROOT)
+ : column.name();
DataType dataType =
Review Comment:
The new case-normalization logic for column names/primary keys is gated by `isTableIdCaseInsensitive`, but there doesn't appear to be test coverage for the bounded snapshot (StartupOptions.snapshot()) path where schemas are fetched via SHOW CREATE TABLE/DESC. Consider adding an IT that runs with `StartupOptions.snapshot()` against a table with uppercase column names (and PK) to exercise this end-to-end.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38244][hotfix] Fix the full snapshot phase, the field case is adjusted based on isTableIdCaseInsensitive [flink-cdc]
Posted by "sd4324530 (via GitHub)" <gi...@apache.org>.
sd4324530 commented on PR #4284:
URL: https://github.com/apache/flink-cdc/pull/4284#issuecomment-4029438294
The CI failure should be unrelated to this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38244][hotfix] Fix the full snapshot phase, the field case is adjusted based on isTableIdCaseInsensitive [flink-cdc]
Posted by "lvyanquan (via GitHub)" <gi...@apache.org>.
lvyanquan commented on PR #4284:
URL: https://github.com/apache/flink-cdc/pull/4284#issuecomment-3997184569
MySQL field name case sensitivity should be independent of table name case sensitivity and should not be affected by
the [lower_case_table_names](https://dev.mysql.com/doc/refman/8.4/en/identifier-case-sensitivity.html) configuration parameter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38244][hotfix] Fix the full snapshot phase, the field case is adjusted based on isTableIdCaseInsensitive [flink-cdc]
Posted by "loserwang1024 (via GitHub)" <gi...@apache.org>.
loserwang1024 commented on PR #4284:
URL: https://github.com/apache/flink-cdc/pull/4284#issuecomment-4001722746
> the conclusion was to first handle the full phase with the same rules, and then consider a unified solution later, such as converting all fields to lowercase.
@lvyanquan , yes, we first make the behavior of https://github.com/apache/flink-cdc/pull/4095 same, and then can move forward in another pr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38244][hotfix] Fix the full snapshot phase, the field case is adjusted based on isTableIdCaseInsensitive [flink-cdc]
Posted by "lvyanquan (via GitHub)" <gi...@apache.org>.
lvyanquan merged PR #4284:
URL: https://github.com/apache/flink-cdc/pull/4284
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38244][hotfix] Fix the full snapshot phase, the field case is adjusted based on isTableIdCaseInsensitive [flink-cdc]
Posted by "sd4324530 (via GitHub)" <gi...@apache.org>.
sd4324530 commented on PR #4284:
URL: https://github.com/apache/flink-cdc/pull/4284#issuecomment-3997610724
> MySQL field name case sensitivity should be independent of table name case sensitivity and should not be affected by the [lower_case_table_names](https://dev.mysql.com/doc/refman/8.4/en/identifier-case-sensitivity.html) configuration parameter.
@lvyanquan You're right. This issue was discussed in the DingTalk group before. The `lower_case_table_names` configuration option is used to control the case of table names, not fields. But the previous [pr](https://github.com/apache/flink-cdc/pull/4095) modified the incremental phase, causing inconsistencies between the full and incremental phases when fields are uppercase. After offline communication, the conclusion was to first handle the full phase with the same rules, and then consider a unified solution later, such as converting all fields to lowercase. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38244][hotfix] Fix the full snapshot phase, the field case is adjusted based on isTableIdCaseInsensitive [flink-cdc]
Posted by "sd4324530 (via GitHub)" <gi...@apache.org>.
sd4324530 commented on PR #4284:
URL: https://github.com/apache/flink-cdc/pull/4284#issuecomment-3949958720
@leonardBang @loserwang1024 Please take a look when you have time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org