Re: [PR] feat: Allow non-registry schema provider in debezium source [hudi]

["yihua (via GitHub)" <gi...@apache.org>]

yihua commented on code in PR #17725:
URL: https://github.com/apache/hudi/pull/17725#discussion_r2795557113


##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/debezium/DebeziumSource.java:
##########
@@ -124,7 +122,7 @@ protected Pair<Option<Dataset<Row>>, Checkpoint> fetchNextBatch(Option<Checkpoin
     LOG.info("About to read " + totalNewMsgs + " from Kafka for topic :" + offsetGen.getTopicName());
 
     try {
-      String schemaStr = schemaRegistryProvider.fetchSchemaFromRegistry(getStringWithAltKeys(props, HoodieSchemaProviderConfig.SRC_SCHEMA_REGISTRY_URL));
+      String schemaStr = schemaProvider.getSourceHoodieSchema().getAvroSchema().toString();

Review Comment:
   `getSourceHoodieSchema()` can return null (when the underlying `getSourceSchema()` returns null), which would cause an NPE here. Could you add a null check, or at least call it once and verify it's non-null before chaining `.getAvroSchema().toString()`?



##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/debezium/DebeziumSource.java:
##########
@@ -100,15 +99,14 @@ public DebeziumSource(TypedProperties props, JavaSparkContext sparkContext,
       throw new HoodieReadFromSourceException(error, e);
     }
 
-    // Currently, debezium source requires Confluent/Kafka schema-registry to fetch the latest schema.
-    if (schemaProvider == null || !(schemaProvider instanceof SchemaRegistryProvider)) {
-      schemaRegistryProvider = new SchemaRegistryProvider(props, sparkContext);
+    if (schemaProvider == null) {
+      this.schemaProvider = new SchemaRegistryProvider(props, sparkContext);
     } else {
-      schemaRegistryProvider = (SchemaRegistryProvider) schemaProvider;
+      this.schemaProvider = schemaProvider;

Review Comment:
   Add a test case around this change?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org