[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #207: Add external schema mappings for files written with name-based schemas #40

[GitBox <gi...@apache.org>]

rdblue commented on a change in pull request #207: Add external schema mappings for files written with name-based schemas #40
URL: https://github.com/apache/incubator-iceberg/pull/207#discussion_r330275738
 
 

 ##########
 File path: core/src/main/java/org/apache/iceberg/avro/BuildAvroProjection.java
 ##########
 @@ -165,13 +165,14 @@ public Schema array(Schema array, Supplier<Schema> element) {
       try {
         Schema keyValueSchema = array.getElementType();
         Schema.Field keyField = keyValueSchema.getFields().get(0);
+        Schema.Field keyProjection = element.get().getField("key");
         Schema.Field valueField = keyValueSchema.getFields().get(1);
         Schema.Field valueProjection = element.get().getField("value");
 
         // element was changed, create a new array
-        if (valueProjection.schema() != valueField.schema()) {
+        if (keyProjection.schema() != keyField.schema() || valueProjection.schema() != valueField.schema()) {
           return AvroSchemaUtil.createProjectionMap(keyValueSchema.getFullName(),
-              AvroSchemaUtil.getFieldId(keyField), keyField.name(), keyField.schema(),
+              AvroSchemaUtil.getFieldId(keyField), keyField.name(), keyProjection.schema(),
 
 Review comment:
   The previous version didn't use `keyProjection` because the entire key must be projected. If the key is a struct of multiple columns, then projecting a subset of those columns can easily introduce key collisions that aren't in the original data.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org