[GitHub] [iceberg] rzhang10 commented on a diff in pull request #6004: Data: Support reading default values from generic Avro readers

[GitBox <gi...@apache.org>]

rzhang10 commented on code in PR #6004:
URL: https://github.com/apache/iceberg/pull/6004#discussion_r1005296940


##########
core/src/main/java/org/apache/iceberg/avro/BuildAvroProjection.java:
##########
@@ -102,9 +102,17 @@ public Schema record(Schema record, List<String> names, Iterable<Schema.Field> s
 
       } else {
         Preconditions.checkArgument(
-            field.isOptional() || MetadataColumns.metadataFieldIds().contains(field.fieldId()),
-            "Missing required field: %s",
+            (field.isRequired() && field.initialDefault() != null)
+                || field.isOptional()
+                || MetadataColumns.metadataFieldIds().contains(field.fieldId()),
+            "Missing required field that doesn't have a default value: %s",
             field.name());
+        // If the field from Iceberg schema has initial default value, we just pass and don't
+        // project it to the avro file read schema with the generated _r field name,
+        // the default value will be directly read from the Iceberg layer
+        if (field.initialDefault() != null) {

Review Comment:
   I see, I originally thought using `continue` can just be an implementation trick, since although the output avro schema of the buildavroprojection process is lacking the field, it will be taken care of when they are being compared again in the `ValueReader`.
   
   I think I can make the code logic more consistent by still creating a fake field with some other suffix like "_d" instead of "_r". The "_d" field will not be projected in the real file either, and the default value read is still taken care of by `ValueReader`. WDYT?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org