Re: [PR] Kafka Connect: Preserve explicit null values instead of replacing them with schema defaults [iceberg]

["moggaa (via GitHub)" <gi...@apache.org>]

moggaa commented on code in PR #17653:
URL: https://github.com/apache/iceberg/pull/17653#discussion_r4241808381


##########
kafka-connect/kafka-connect-transforms/src/main/java/org/apache/iceberg/connect/transforms/CopyValue.java:
##########
@@ -90,9 +90,10 @@ private R applyWithSchema(R record) {
     Struct updatedValue = new Struct(updatedSchema);
 
     for (Field field : value.schema().fields()) {
-      updatedValue.put(field.name(), value.get(field));
+      // getWithoutDefault so an explicit null is not replaced by the schema default value
+      updatedValue.put(field.name(), value.getWithoutDefault(field.name()));

Review Comment:
   The floor holds — `Struct.getWithoutDefault` is not a 3.5 addition, it's already in connect-api 2.5.0 ([Struct.java#L98](https://github.com/apache/kafka/blob/2.5.0/connect/api/src/main/java/org/apache/kafka/connect/data/Struct.java#L98)). What [KAFKA-8713](https://issues.apache.org/jira/browse/KAFKA-8713) added in 3.5 is the JsonConverter `replace.null.with.default` config, not the Struct accessor.
   
   To double-check the worker scenario, I compiled the call against connect-api 3.9.2 and ran it with the 2.5.0 jars on the classpath — `getWithoutDefault` works there as expected, returning the stored null. The sink `false` path calls the same method, so the documented minimum can stay at 2.5 with no doc note needed for either.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org