[GitHub] [iceberg] lxynov commented on a change in pull request #1321: Avro: Fix pruning columns when a logical-map array's value type is nested

[GitBox <gi...@apache.org>]

lxynov commented on a change in pull request #1321:
URL: https://github.com/apache/iceberg/pull/1321#discussion_r469001831



##########
File path: core/src/main/java/org/apache/iceberg/avro/PruneColumns.java
##########
@@ -144,16 +144,18 @@ public Schema array(Schema array, Schema element) {
       if (selectedIds.contains(keyId) || selectedIds.contains(valueId)) {
         return complexMapWithIds(array, keyId, valueId);
       } else if (element != null) {
-        Schema keyProjection = element.getField("key").schema();
+        Schema.Field keyProjectionField = element.getField("key");

Review comment:
       Discussing with @rdsr offline, also posting here for @rdblue 's visibility
   
   I tried to always project the `keyField` but encountered two issues:
    - `TestGenericReadProjection` in Parquet fails somehow. (I haven't spent time looking into how to resolve it)
    - We'll still need to change `PruneColumns` because its current logic is `// if either key or value is selected, the whole map must be projected`.
   
   @rdsr @rdblue what are your thoughts? I think we can make it to always project `keyField` but it may need some efforts




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org