[GitHub] [iceberg] marton-bod commented on a change in pull request #1920: Hive: Serialize metadata location so split generation does not need to load the table

[GitBox <gi...@apache.org>]

marton-bod commented on a change in pull request #1920:
URL: https://github.com/apache/iceberg/pull/1920#discussion_r541548984



##########
File path: mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java
##########
@@ -205,6 +210,8 @@ static void put(Configuration config, Table table) {
     config.set(InputFormatConfig.TABLE_LOCATION, table.location());
     config.set(InputFormatConfig.TABLE_SCHEMA, SchemaParser.toJson(table.schema()));
     config.set(InputFormatConfig.PARTITION_SPEC, PartitionSpecParser.toJson(table.spec()));
+    config.set(InputFormatConfig.METADATA_LOCATION,
+        ((HasTableOperations) table).operations().current().metadataFileLocation());

Review comment:
       Same as above. Wondering if the `Table` instance that's passed in can be something other than `BaseTable`, such as `PartitionsTable`, `HistoryTable`, etc.

##########
File path: mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java
##########
@@ -85,7 +86,10 @@ public void configureInputJobProperties(TableDesc tableDesc, Map<String, String>
     map.put(InputFormatConfig.TABLE_IDENTIFIER, props.getProperty(Catalogs.NAME));
     map.put(InputFormatConfig.TABLE_LOCATION, table.location());
     map.put(InputFormatConfig.TABLE_SCHEMA, schemaJson);
+    map.put(InputFormatConfig.METADATA_LOCATION,
+        ((HasTableOperations) table).operations().current().metadataFileLocation());

Review comment:
       can we get a `Table` instance here which does not implement `HasTableOperarions` and therefore would fail? Only `BaseTable` does so, wondering about the other `Table` implementations, if they can make it here?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org