[GitHub] [incubator-paimon] yuzelin commented on a diff in pull request #2016: [cdc] kafka cdc uses hive catalog to support field conversion to lowercase

["yuzelin (via GitHub)" <gi...@apache.org>]

yuzelin commented on code in PR #2016:
URL: https://github.com/apache/incubator-paimon/pull/2016#discussion_r1330873796


##########
paimon-flink/paimon-flink-cdc/src/test/java/org/apache/paimon/flink/action/cdc/kafka/KafkaCanalSyncDatabaseActionITCase.java:
##########
@@ -571,4 +576,47 @@ public void testCatalogAndTableConfig() {
         assertThat(action.tableConfig())
                 .containsExactlyEntriesOf(Collections.singletonMap("table-key", "table-value"));
     }
+
+    @Test
+    @Timeout(60)
+    public void testCaseInsensitive() throws Exception {
+        catalog =
+                new TestCaseInsensitiveCatalogFactory()
+                        .createCatalog(CatalogContext.create(new Path(warehouse)));
+
+        final String topic = "case-insensitive";
+        createTestTopic(topic, 1, 1);
+
+        // ---------- Write the Canal json into Kafka -------------------
+        writeRecordsToKafka(
+                topic, readLines("kafka/canal/database/case-insensitive/canal-data-1.txt"));
+
+        Map<String, String> kafkaConfig = getBasicKafkaConfig();
+        kafkaConfig.put("value.format", "canal-json");
+        kafkaConfig.put("topic", topic);
+
+        KafkaSyncDatabaseAction action =
+                syncDatabaseActionBuilder(kafkaConfig)
+                        .withTableConfig(getBasicTableConfig())
+                        .build();
+        Field catalogField = ActionBase.class.getDeclaredField("catalog");
+        catalogField.setAccessible(true);
+        Object newCatalog = catalog;
+        catalogField.set(action, newCatalog);

Review Comment:
   use catalog config to set `CatalogOptions.METASTORE.key() = test-case-insensitive`



##########
paimon-flink/paimon-flink-cdc/src/test/java/org/apache/paimon/flink/action/cdc/kafka/KafkaOggSyncDatabaseActionITCase.java:
##########
@@ -462,4 +467,51 @@ private void includingAndExcludingTablesImpl(
         waitingTables(existedTables);
         assertTableNotExists(notExistedTables);
     }
+
+    @Test
+    @Timeout(60)
+    public void testCaseInsensitive() throws Exception {
+        catalog =
+                new TestCaseInsensitiveCatalogFactory()
+                        .createCatalog(CatalogContext.create(new Path(warehouse)));
+
+        final String topic = "case-insensitive";
+        createTestTopic(topic, 1, 1);
+
+        // ---------- Write the ogg json into Kafka -------------------
+
+        writeRecordsToKafka(topic, readLines("kafka/ogg/database/case-insensitive/ogg-data-1.txt"));
+
+        Map<String, String> kafkaConfig = getBasicKafkaConfig();
+        kafkaConfig.put("value.format", "ogg-json");
+        kafkaConfig.put("topic", topic);
+
+        KafkaSyncDatabaseAction action =
+                syncDatabaseActionBuilder(kafkaConfig)
+                        .withTableConfig(getBasicTableConfig())
+                        .build();
+        Field catalogField = ActionBase.class.getDeclaredField("catalog");
+        catalogField.setAccessible(true);
+        Object newCatalog = catalog;
+        catalogField.set(action, newCatalog);

Review Comment:
   Use catalog-conf, too



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org