[GitHub] [hudi] minihippo commented on a diff in pull request #5629: [WIP][HUDI-3384][HUDI-3385] Spark specific file reader/writer.

[GitBox <gi...@apache.org>]

minihippo commented on code in PR #5629:
URL: https://github.com/apache/hudi/pull/5629#discussion_r903317660


##########
hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java:
##########
@@ -436,14 +444,18 @@ public static GenericRecord removeFields(GenericRecord record, List<String> fiel
 
   private static void copyOldValueOrSetDefault(GenericRecord oldRecord, GenericRecord newRecord, Schema.Field field) {
     Schema oldSchema = oldRecord.getSchema();
-    Object fieldValue = oldSchema.getField(field.name()) == null ? null : oldRecord.get(field.name());
+    Field oldSchemaField = oldSchema.getField(field.name());
+    Object fieldValue = oldSchemaField == null ? null : oldRecord.get(field.name());
 
     if (fieldValue != null) {
       // In case field's value is a nested record, we have to rewrite it as well
       Object newFieldValue;
       if (fieldValue instanceof GenericRecord) {
         GenericRecord record = (GenericRecord) fieldValue;
-        newFieldValue = rewriteRecord(record, resolveUnionSchema(field.schema(), record.getSchema().getFullName()));
+        // May return null when use rewrite
+        String recordFullName = record.getSchema().getFullName();
+        String fullName = recordFullName != null ? recordFullName : oldSchemaField.name();
+        newFieldValue = rewriteRecord(record, resolveUnionSchema(field.schema(), fullName));

Review Comment:
   `fullName ` shouldn't be null. I suppose the changes is caused by `EmptyRecord`, right?



##########
hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java:
##########
@@ -1037,4 +1049,25 @@ public GenericRecord next() {
       }
     };
   }
+
+  public static HoodieRecord createHoodieRecordFromAvro(IndexedRecord data, Map<String, Object> mapperConfig) {

Review Comment:
   mapperconfig -> props may be better? Align with the current config passing way



##########
hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java:
##########
@@ -436,14 +444,18 @@ public static GenericRecord removeFields(GenericRecord record, List<String> fiel
 
   private static void copyOldValueOrSetDefault(GenericRecord oldRecord, GenericRecord newRecord, Schema.Field field) {
     Schema oldSchema = oldRecord.getSchema();
-    Object fieldValue = oldSchema.getField(field.name()) == null ? null : oldRecord.get(field.name());
+    Field oldSchemaField = oldSchema.getField(field.name());
+    Object fieldValue = oldSchemaField == null ? null : oldRecord.get(field.name());

Review Comment:
   Can field.name be null? 



##########
hudi-common/src/main/java/org/apache/hudi/common/config/HoodieConfig.java:
##########
@@ -218,4 +240,15 @@ public <T> String getStringOrThrow(ConfigProperty<T> configProperty, String erro
       throw new HoodieException(errorMessage);
     }
   }
+
+  public HoodieRecordType getRecordType() {

Review Comment:
   It's not a basic function, so it's inappropriate to put it here



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org