[GitHub] [iceberg] pvary commented on a change in pull request #1417: Hive: Add column projection

[GitBox <gi...@apache.org>]

pvary commented on a change in pull request #1417:
URL: https://github.com/apache/iceberg/pull/1417#discussion_r483007545



##########
File path: mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergInputFormat.java
##########
@@ -61,6 +62,9 @@
       }
     }
 
+    String[] readColumns = ColumnProjectionUtils.getReadColumnNames(job);

Review comment:
       I have seen this in Hive code as a comment in ColumnProjectionUtils.getReadColumnIDs:
   ```
         // NOTE: some code uses this list to correlate with column names, and yet these lists may
         //       contain duplicates, which this call will remove and the other won't. As far as I can
         //       tell, no code will actually use these two methods together; all is good if the code
         //       gets the ID list without relying on this method. Or maybe it just works by magic.
   ```
   Not sure if this comment is still true or it is already fixed. If we might get duplicates I am not sure how Iceberg will behave in this case. Might worth to check. Maybe with some query with different UDFs around the same column?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org