You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "yzeng1618 (via GitHub)" <gi...@apache.org> on 2026/01/30 08:32:26 UTC

[PR] [Fix][dbz] wrong column discovery due to getColumns tableNamePattern [seatunnel]

yzeng1618 opened a new pull request, #10422:
URL: https://github.com/apache/seatunnel/pull/10422

   <!--
   
   Thank you for contributing to SeaTunnel! Please make sure that your code changes
   are covered with tests. And in case of new features or big changes
   remember to adjust the documentation.
   
   Feel free to ping committers for the review!
   
   ## Contribution Checklist
     - Make sure that the pull request corresponds to a [GITHUB issue](https://github.com/apache/seatunnel/issues).
     - Name the pull request in the form "[Feature] [component] Title of the pull request", where *Feature* can be replaced by `Hotfix`, `Bug`, etc.
     - Minor fixes should be named following this pattern: `[hotfix] [docs] Fix typo in README.md doc`.
   -->
   
   ### Purpose of this pull request
   
   <!-- Describe the purpose of this pull request. For example: This pull request adds checkstyle plugin.-->
   
   Fixes #10410.
   
   Some JDBC drivers treat `DatabaseMetaData#getColumns`'s `tableNamePattern` as a SQL `LIKE` pattern. When the target table name is “pattern-like” (e.g. contains `_` / `%`), the metadata query may return columns from other tables (for example, `user_info` can also match `userAinfo`), which leads to wrong schema/column discovery.
   
   ### Does this PR introduce _any_ user-facing change?
   Yes. Previously, schema discovery could include extra columns from other matched tables; after this change, column discovery is restricted to the exact table/schema, producing the correct schema and avoiding downstream failures caused by wrong columns.
   <!--
   Note that it means *any* user-facing change including all aspects such as the documentation fix.
   If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
   If possible, please also clarify if this is a user-facing change compared to the released SeaTunnel versions or within the unreleased branches such as dev.
   If no, write 'No'.
   If you are adding/modifying connector documents, please follow our new specifications: https://github.com/apache/seatunnel/issues/4544.
   -->
   
   
   ### How was this patch tested?
   - Added unit test: `CatalogUtilsTest#testGetTableSchemaFiltersOutOtherMatchedTables`.
   <!--
   If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
   If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
   If tests were not added, please describe why they were not added and/or why it was difficult to add.
   If you are adding E2E test cases, maybe refer to https://github.com/apache/seatunnel/blob/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-mysql-e2e/src/test/resources/mysqlcdc_to_mysql.conf, here is a good example.
   -->
   
   
   ### Check list
   
   * [ ] If any new Jar binary package adding in your PR, please add License Notice according
     [New License Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/contribution/new-license.md)
   * [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
   * [ ] If necessary, please update `incompatible-changes.md` to describe the incompatibility caused by this PR.
   * [ ] If you are contributing the connector code, please check that the following files are updated:
     1. Update [plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties) and add new connector information in it
     2. Update the pom file of [seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
     3. Add ci label in [label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
     4. Add e2e testcase in [seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
     5. Update connector [plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-V2] Fix wrong column discovery when tableNamePattern is treated as LIKE [seatunnel]

Posted by "corgy-w (via GitHub)" <gi...@apache.org>.
corgy-w merged PR #10422:
URL: https://github.com/apache/seatunnel/pull/10422


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-V2] Fix wrong column discovery when tableNamePattern is treated as LIKE [seatunnel]

Posted by "Carl-Zhou-CN (via GitHub)" <gi...@apache.org>.
Carl-Zhou-CN commented on code in PR #10422:
URL: https://github.com/apache/seatunnel/pull/10422#discussion_r2750833779


##########
seatunnel-connectors-v2/connector-cdc/connector-cdc-sqlserver/src/main/java/io/debezium/connector/sqlserver/SqlServerConnection.java:
##########
@@ -705,6 +705,20 @@ public Table getTableSchemaFromTable(String databaseName, SqlServerChangeTable c
                         changeTable.getSourceTableId().table(),
                         null)) {
             while (rs.next()) {
+                // `tableNamePattern` is treated as a SQL LIKE pattern by many drivers, so filter

Review Comment:
   Why is only qlserver rather special?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-V2] Fix wrong column discovery when tableNamePattern is treated as LIKE [seatunnel]

Posted by "Carl-Zhou-CN (via GitHub)" <gi...@apache.org>.
Carl-Zhou-CN commented on code in PR #10422:
URL: https://github.com/apache/seatunnel/pull/10422#discussion_r2758383306


##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/utils/JdbcIdentifierUtils.java:
##########
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.utils;
+
+import java.sql.DatabaseMetaData;
+import java.sql.SQLException;
+import java.util.Locale;
+
+public final class JdbcIdentifierUtils {
+
+    private JdbcIdentifierUtils() {}
+
+    public enum IdentifierCaseStrategy {
+        CASE_SENSITIVE,
+        LOWER_CASE,
+        UPPER_CASE,
+        CASE_INSENSITIVE
+    }
+
+    /**
+     * Resolve case handling strategy for unquoted identifiers based on {@link DatabaseMetaData}.
+     *
+     * <p>Note: JDBC metadata APIs often treat {@code schemaPattern}/{@code tableNamePattern} as
+     * patterns (e.g. SQL LIKE), while identifier case sensitivity depends on the database. This
+     * method provides a best-effort strategy to compare identifiers returned by JDBC metadata APIs.
+     */
+    public static IdentifierCaseStrategy identifierCaseStrategy(DatabaseMetaData metadata)
+            throws SQLException {
+        if (metadata == null) {
+            return IdentifierCaseStrategy.CASE_INSENSITIVE;
+        }
+        if (metadata.supportsMixedCaseIdentifiers()) {

Review Comment:
   Could it be that he threw an exception



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-V2] Fix wrong column discovery when tableNamePattern is treated as LIKE [seatunnel]

Posted by "Carl-Zhou-CN (via GitHub)" <gi...@apache.org>.
Carl-Zhou-CN commented on code in PR #10422:
URL: https://github.com/apache/seatunnel/pull/10422#discussion_r2758383306


##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/utils/JdbcIdentifierUtils.java:
##########
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.utils;
+
+import java.sql.DatabaseMetaData;
+import java.sql.SQLException;
+import java.util.Locale;
+
+public final class JdbcIdentifierUtils {
+
+    private JdbcIdentifierUtils() {}
+
+    public enum IdentifierCaseStrategy {
+        CASE_SENSITIVE,
+        LOWER_CASE,
+        UPPER_CASE,
+        CASE_INSENSITIVE
+    }
+
+    /**
+     * Resolve case handling strategy for unquoted identifiers based on {@link DatabaseMetaData}.
+     *
+     * <p>Note: JDBC metadata APIs often treat {@code schemaPattern}/{@code tableNamePattern} as
+     * patterns (e.g. SQL LIKE), while identifier case sensitivity depends on the database. This
+     * method provides a best-effort strategy to compare identifiers returned by JDBC metadata APIs.
+     */
+    public static IdentifierCaseStrategy identifierCaseStrategy(DatabaseMetaData metadata)
+            throws SQLException {
+        if (metadata == null) {
+            return IdentifierCaseStrategy.CASE_INSENSITIVE;
+        }
+        if (metadata.supportsMixedCaseIdentifiers()) {

Review Comment:
   Could it be that he threw an exception



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-V2] Fix wrong column discovery when tableNamePattern is treated as LIKE [seatunnel]

Posted by "Carl-Zhou-CN (via GitHub)" <gi...@apache.org>.
Carl-Zhou-CN commented on code in PR #10422:
URL: https://github.com/apache/seatunnel/pull/10422#discussion_r2758386616


##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/utils/JdbcIdentifierUtils.java:
##########
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.utils;
+
+import java.sql.DatabaseMetaData;
+import java.sql.SQLException;
+import java.util.Locale;
+
+public final class JdbcIdentifierUtils {
+
+    private JdbcIdentifierUtils() {}
+
+    public enum IdentifierCaseStrategy {
+        CASE_SENSITIVE,
+        LOWER_CASE,
+        UPPER_CASE,
+        CASE_INSENSITIVE
+    }
+
+    /**
+     * Resolve case handling strategy for unquoted identifiers based on {@link DatabaseMetaData}.
+     *
+     * <p>Note: JDBC metadata APIs often treat {@code schemaPattern}/{@code tableNamePattern} as
+     * patterns (e.g. SQL LIKE), while identifier case sensitivity depends on the database. This
+     * method provides a best-effort strategy to compare identifiers returned by JDBC metadata APIs.
+     */
+    public static IdentifierCaseStrategy identifierCaseStrategy(DatabaseMetaData metadata)
+            throws SQLException {
+        if (metadata == null) {
+            return IdentifierCaseStrategy.CASE_INSENSITIVE;
+        }
+        if (metadata.supportsMixedCaseIdentifiers()) {
+            return IdentifierCaseStrategy.CASE_SENSITIVE;
+        }
+        if (metadata.storesLowerCaseIdentifiers()) {
+            return IdentifierCaseStrategy.LOWER_CASE;
+        }
+        if (metadata.storesUpperCaseIdentifiers()) {
+            return IdentifierCaseStrategy.UPPER_CASE;
+        }
+        return IdentifierCaseStrategy.CASE_INSENSITIVE;

Review Comment:
   Should we be the most tolerant?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-V2] Fix wrong column discovery when tableNamePattern is treated as LIKE [seatunnel]

Posted by "yzeng1618 (via GitHub)" <gi...@apache.org>.
yzeng1618 commented on code in PR #10422:
URL: https://github.com/apache/seatunnel/pull/10422#discussion_r2752861214


##########
seatunnel-connectors-v2/connector-cdc/connector-cdc-sqlserver/src/main/java/io/debezium/connector/sqlserver/SqlServerConnection.java:
##########
@@ -705,6 +705,20 @@ public Table getTableSchemaFromTable(String databaseName, SqlServerChangeTable c
                         changeTable.getSourceTableId().table(),
                         null)) {
             while (rs.next()) {
+                // `tableNamePattern` is treated as a SQL LIKE pattern by many drivers, so filter

Review Comment:
   This is not a SQL Server-specific case; it is caused by the pattern semantics of JDBC getColumns combined with differences in driver implementations.
   The reason SqlServerConnection.java appears in the PR is that SQL Server CDC follows a separate code path in Debezium and does not reuse SeaTunnel’s JDBC CatalogUtils or JdbcDialectTypeMapper. Therefore, the same protection logic must also be applied to the CDC code path.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-V2] Fix wrong column discovery when tableNamePattern is treated as LIKE [seatunnel]

Posted by "DanielCarter-stack (via GitHub)" <gi...@apache.org>.
DanielCarter-stack commented on PR #10422:
URL: https://github.com/apache/seatunnel/pull/10422#issuecomment-3831152467

   <!-- code-pr-reviewer -->
   <!-- cpr:pr_reply_v2_parts {"group": "apache/seatunnel#10422", "part": 1, "total": 1} -->
   ### Issue 1: Improper Case Sensitivity Handling
   
   **Location**: 
   - `JdbcDialectTypeMapper.java:97-110`
   - `JdbcColumnConverter.java:87-98`
   - `SqlServerConnection.java:710-721`
   
   **Issue Description**:
   Using `equalsIgnoreCase` for table name and schema name comparison, but in some databases (such as PostgreSQL under certain configurations, Oracle, H2, etc.), table names are **case-sensitive**. This could lead to:
   - User queries `UserInfo` (uppercase U)
   - Database returns columns of `userinfo` (lowercase)
   - `equalsIgnoreCase` considers them equal, incorrectly including that column
   
   **Related Context**:
   - Parent/interface: `DatabaseMetaData.getColumns()` JDBC specification states "whether identifiers are case-sensitive depends on the database"
   - Caller: `CatalogUtils.getTableSchema()` is called in 23 database connectors
   - Impact scope: All case-sensitive databases (PostgreSQL, Oracle, H2, DB2, etc.)
   
   **Potential Risks**:
   - Risk 1: On case-sensitive databases, columns from wrong tables may be included (though probability is low)
   - Risk 2: Tests do not cover case sensitivity scenarios
   
   **Impact Scope**:
   - Direct impact: Column discovery for case-sensitive databases like PostgreSQL, Oracle, H2, DB2, SAP HANA, etc.
   - Indirect impact: Type inference and data conversion based on column information
   - Affected area: Multiple Connectors (approximately 10+ case-sensitive databases)
   
   **Severity**: **MAJOR**
   
   **Improvement Suggestions**:
   ```java
   // Suggestion: Consider database case sensitivity
   // Solution 1: Get information from DatabaseMetaData.storesLowerCaseIdentifiers()
   boolean storesLowerCase = metadata.storesLowerCaseIdentifiers();
   boolean storesUpperCase = metadata.storesUpperCaseIdentifiers();
   boolean storesMixedCase = metadata.storesMixedCaseQuotedIdentifiers();
   
   // Solution 2: Use DatabaseMetaData.identifierEqualsString() (JDBC 4.3+)
   // But need to consider compatibility
   
   // Solution 3 (current best practice): Keep equalsIgnoreCase, but add warning log
   if (actualTableName == null || !actualTableName.equalsIgnoreCase(tableNamePattern)) {
       if (actualTableName != null && !actualTableName.equals(tableNamePattern)) {
           log.debug("Filtered column from table '{}' while looking for '{}', case-sensitive databases may require exact match", 
                     actualTableName, tableNamePattern);
       }
       continue;
   }
   ```
   
   **Rationale**:
   1. JDBC specification explicitly states that case sensitivity varies across different databases
   2. Using `equalsIgnoreCase` is correct on case-insensitive databases (such as MySQL, SQL Server)
   3. But may cause errors on case-sensitive databases
   4. Adding logs can help users diagnose issues
   
   ---
   
   ### Issue 2: Missing Handling and Warning for Empty Column Lists
   
   **Location**: 
   - `JdbcDialectTypeMapper.java:84-135`
   - `JdbcColumnConverter.java:73-121`
   
   **Issue Description**:
   If all returned rows are filtered out (for example, the JDBC driver returns completely mismatched tables), the code will **silently return an empty list**. This could lead to:
   1. Upstream callers assuming the table has no columns and continuing processing
   2. Failing at a later stage (such as missing required columns)
   3. Unclear error messages, making diagnosis difficult
   
   **Related Context**:
   - Caller 1: `CatalogUtils.getTableSchema()` (lines 223-245)
   - Caller 2: `SqlServerConnection.getTableSchemaFromTable()` (lines 696-739)
   - These callers expect non-empty column lists to be returned
   
   **Potential Risks**:
   - Risk 1: When table doesn't exist or configuration is wrong, errors are delayed to later stages
   - Risk 2: Users see "table has no columns" instead of "table doesn't exist or columns were filtered"
   - Risk 3: Difficult to debug
   
   **Impact Scope**:
   - Direct impact: All users using JDBC CDC and JDBC Source
   - Indirect impact: Task failures, data loss
   - Affected area: All Connectors
   
   **Severity**: **MINOR**
   
   **Improvement Suggestions**:
   ```java
   // In JdbcDialectTypeMapper.java
   List<Column> columns = new ArrayList<>();
   int filteredRows = 0; // Add counter
   try (ResultSet rs = metadata.getColumns(...)) {
       while (rs.next()) {
           if (tableNamePattern != null) {
               String actualTableName = rs.getString("TABLE_NAME");
               if (actualTableName == null || !actualTableName.equalsIgnoreCase(tableNamePattern)) {
                   filteredRows++; // Log filtered rows
                   continue;
               }
           }
           // ... rest of logic
           columns.add(mappingColumn(typeDefine));
       }
   }
   
   // Add: If all rows are filtered, log warning
   if (columns.isEmpty() && filteredRows > 0) {
       log.warn("No columns found for table '{}', filtered {} rows from other tables. " +
                "The table may not exist or the driver may be returning unexpected results.",
                tableNamePattern, filteredRows);
   }
   return columns;
   ```
   
   **Rationale**:
   1. Helps users quickly diagnose configuration issues
   2. Doesn't affect normal flow (just logging)
   3. Low cost, high value
   
   ---
   
   ### Issue 3: Insufficient Test Coverage
   
   **Location**: `CatalogUtilsTest.java:98-168`
   
   **Issue Description**:
   The newly added test case `testGetTableSchemaFiltersOutOtherMatchedTables` only covers one scenario:
   - Table name contains `_` wildcard
   - Returns 2 rows (1 matching + 1 non-matching)
   
   But the following scenarios are missing:
   1. Table name contains `%` wildcard (such as `user%info`)
   2. Schema name contains wildcards (such as `pub_lic`)
   3. Both table name and schema contain wildcards
   4. Returns multiple rows, with only partial matches
   5. Returns 0 rows (table doesn't exist)
   6. Case mismatch (`UserInfo` vs `userinfo`)
   7. SqlServerConnection modifications have no corresponding unit tests
   
   **Potential Risks**:
   - Risk 1: `%` wildcard scenarios not verified
   - Risk 2: Edge cases not covered
   - Risk 3: CDC path modifications have no test protection
   
   **Impact Scope**:
   - Direct impact: Code quality assurance
   - Indirect impact: Future regressions may be introduced
   - Affected area: Single module
   
   **Severity**: **MINOR**
   
   **Improvement Suggestions**:
   ```java
   // Add new test cases
   
   @Test
   void testGetTableSchemaFiltersOutPercentageWildcard() throws SQLException {
       // Test % wildcard
       TestDatabaseMetaData metadata = new TestDatabaseMetaData() {
           @Override
           public ResultSet getColumns(...) {
               List<Map<String, Object>> value = new ArrayList<>();
               value.add(Map.of(
                   "TABLE_NAME", "user%info",
                   "TABLE_SCHEM", "public",
                   "COLUMN_NAME", "id",
                   // ...
               ));
               value.add(Map.of(
                   "TABLE_NAME", "userXYZinfo", // Should be filtered
                   "TABLE_SCHEM", "public",
                   "COLUMN_NAME", "bad",
                   // ...
               ));
               return new TestResultSet(value);
           }
       };
       
       TablePath tablePath = TablePath.of("test_db", "public", "user%info");
       TableSchema tableSchema = CatalogUtils.getTableSchema(metadata, tablePath, typeMapper);
       
       Assertions.assertEquals(1, tableSchema.getColumns().size());
   }
   
   @Test
   void testGetTableSchemaFiltersOutSchemaWildcard() throws SQLException {
       // Test schema contains wildcard
       // ...
   }
   
   @Test
   void testGetTableSchemaEmptyWhenAllFiltered() throws SQLException {
       // Test all rows are filtered
       // ...
   }
   ```
   
   **Rationale**:
   1. Improve code coverage
   2. Prevent future modifications from introducing regressions
   3. `%` wildcards are equally common
   
   ---
   
   ### Issue 4: Missing Unit Tests for SqlServerConnection
   
   **Location**: `SqlServerConnection.java:708-721`
   
   **Issue Description**:
   Filtering logic was added in SqlServerConnection, but **there are no corresponding unit tests**. This is a CDC path modification, independent from the JDBC path, and requires separate verification.
   
   **Potential Risks**:
   - Risk 1: CDC path modifications have no test protection
   - Risk 2: Future modifications may break CDC functionality
   - Risk 3: SQL Server users may be affected
   
   **Impact Scope**:
   - Direct impact: SQL Server CDC Connector
   - Indirect impact: Data synchronization tasks using SQL Server CDC
   - Affected area: Single Connector
   
   **Severity**: **MINOR**
   
   **Improvement Suggestions**:
   ```java
   // Add in connector-cdc-sqlserver test directory
   // File: SqlServerConnectionTest.java
   
   @Test
   void testGetTableSchemaFiltersOutWildcardTables() throws SQLException {
       // Mock DatabaseMetaData to return multiple tables
       TestDatabaseMetaData metadata = new TestDatabaseMetaData() {
           @Override
           public ResultSet getColumns(...) {
               List<Map<String, Object>> value = new ArrayList<>();
               value.add(Map.of(
                   "TABLE_NAME", "user_info",
                   "TABLE_SCHEM", "dbo",
                   "COLUMN_NAME", "id",
                   // ...
               ));
               value.add(Map.of(
                   "TABLE_NAME", "userAinfo", // Should be filtered
                   "TABLE_SCHEM", "dbo",
                   "COLUMN_NAME", "bad",
                   // ...
               ));
               return new TestResultSet(value);
           }
       };
       
       SqlServerConnection connection = new SqlServerConnection(config);
       TableId tableId = TableId.parse("dbo", "user_info");
       SqlServerChangeTable changeTable = mock(SqlServerChangeTable.class);
       when(changeTable.getSourceTableId()).thenReturn(tableId);
       
       Table table = connection.getTableSchemaFromTable("database", changeTable);
       
       Assertions.assertEquals(1, table.columns().size());
       Assertions.assertEquals("id", table.columns().get(0).name());
   }
   ```
   
   **Rationale**:
   1. CDC path is independent from JDBC path
   2. SQL Server is an important data source
   3. Testing cost is not high (TestDatabaseMetaData can be reused)
   
   ---
   
   ### Issue 5: Documentation and Changelog Not Updated
   
   **Location**: PR metadata
   
   **Issue Description**:
   The PR description explicitly states "Does this PR introduce any user-facing change? Yes", but:
   1. No relevant documentation was updated
   2. No update to `incompatible-changes.md`
   3. No update to connector-specific documentation
   
   Although this is a bug fix, it changes observed behavior, and users need to be aware.
   
   **Potential Risks**:
   - Risk 1: Behavior changes after upgrade but no documentation explains it
   - Risk 2: Users may depend on the old incorrect behavior
   - Risk 3: Missing reference during troubleshooting
   
   **Impact Scope**:
   - Direct impact: All JDBC and CDC Connector users
   - Indirect impact: User experience, upgrade path
   - Affected area: All Connectors
   
   **Severity**: **MINOR**
   
   **Improvement Suggestions**:
   ```
   1. 在 docs/en/connector-v2/source/jdbc.md 中添加:
      ## Known Issues
      - Table names containing `_` or `%` may have caused incorrect column discovery. 
        This is fixed in version X.Y.Z. If you were affected, your schema discovery 
        should now work correctly.
   
   2. 在 incompatible-changes.md 中添加(如果适用):
      - JDBC Column Discovery: Previously, table names with SQL LIKE wildcards (`_`, `%`) 
        could return columns from other tables. This is now fixed to only return exact 
        matches.
   ```
   
   **Rationale**:
   1. This is a user-visible behavior change
   2. Helps users understand changes after upgrade
   3. Complies with Apache project documentation standards
   
   ---


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-V2] Fix wrong column discovery when tableNamePattern is treated as LIKE [seatunnel]

Posted by "yzeng1618 (via GitHub)" <gi...@apache.org>.
yzeng1618 commented on code in PR #10422:
URL: https://github.com/apache/seatunnel/pull/10422#discussion_r2761881562


##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/utils/JdbcIdentifierUtils.java:
##########
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.utils;
+
+import java.sql.DatabaseMetaData;
+import java.sql.SQLException;
+import java.util.Locale;
+
+public final class JdbcIdentifierUtils {
+
+    private JdbcIdentifierUtils() {}
+
+    public enum IdentifierCaseStrategy {
+        CASE_SENSITIVE,
+        LOWER_CASE,
+        UPPER_CASE,
+        CASE_INSENSITIVE
+    }
+
+    /**
+     * Resolve case handling strategy for unquoted identifiers based on {@link DatabaseMetaData}.
+     *
+     * <p>Note: JDBC metadata APIs often treat {@code schemaPattern}/{@code tableNamePattern} as
+     * patterns (e.g. SQL LIKE), while identifier case sensitivity depends on the database. This
+     * method provides a best-effort strategy to compare identifiers returned by JDBC metadata APIs.
+     */
+    public static IdentifierCaseStrategy identifierCaseStrategy(DatabaseMetaData metadata)
+            throws SQLException {
+        if (metadata == null) {
+            return IdentifierCaseStrategy.CASE_INSENSITIVE;
+        }
+        if (metadata.supportsMixedCaseIdentifiers()) {
+            return IdentifierCaseStrategy.CASE_SENSITIVE;
+        }
+        if (metadata.storesLowerCaseIdentifiers()) {
+            return IdentifierCaseStrategy.LOWER_CASE;
+        }
+        if (metadata.storesUpperCaseIdentifiers()) {
+            return IdentifierCaseStrategy.UPPER_CASE;
+        }
+        return IdentifierCaseStrategy.CASE_INSENSITIVE;

Review Comment:
   The main issue is that JDBC sources include both case-sensitive and case-insensitive data sources. Therefore, we use the supportsMixedCaseIdentifiers() function as a switch to treat unquoted identifiers containing mixed case as case-sensitive.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org