[GitHub] [iceberg] jackye1995 commented on a change in pull request #3752: Hive: add config for disabling FileIO hadoop configuration serialization

[GitBox <gi...@apache.org>]

jackye1995 commented on a change in pull request #3752:
URL: https://github.com/apache/iceberg/pull/3752#discussion_r773355451



##########
File path: mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java
##########
@@ -165,12 +167,47 @@ public DecomposedPredicate decomposePredicate(JobConf jobConf, Deserializer dese
 
   /**
    * Returns the Table serialized to the configuration based on the table name.
+   * If configuration is missing from the FileIO of the table, it will be populated with the input config.
+   *
    * @param config The configuration used to get the data from
    * @param name The name of the table we need as returned by TableDesc.getTableName()
    * @return The Table
    */
   public static Table table(Configuration config, String name) {
-    return SerializationUtil.deserializeFromBase64(config.get(InputFormatConfig.SERIALIZED_TABLE_PREFIX + name));
+    Table table = SerializationUtil.deserializeFromBase64(config.get(InputFormatConfig.SERIALIZED_TABLE_PREFIX + name));
+    checkAndSetIoConfig(config, table);
+    return table;
+  }
+
+  /**
+   * If enabled, it populates the FileIO's hadoop configuration with the input config object.
+   * This might be necessary when the table object was serialized without the FileIO config.
+   *
+   * @param config Configuration to set for FileIO, if enabled
+   * @param table The Iceberg table object
+   */
+  public static void checkAndSetIoConfig(Configuration config, Table table) {
+    if (config.getBoolean(InputFormatConfig.CONFIG_SERIALIZATION_DISABLED, false) &&

Review comment:
       nit: false should be moved to be `InputFormatConfig.CONFIG_SERIALIZATION_DISABLED_DEFAULT`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org