[GitHub] [iceberg] maswin commented on pull request #7802: Add support for Tez Execution engine in Hive-3.x

["maswin (via GitHub)" <gi...@apache.org>]

maswin commented on PR #7802:
URL: https://github.com/apache/iceberg/pull/7802#issuecomment-1593755776

   > I am not sure this will compile and run with Hive2
   
   [DefaultHiveMetaHook](https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/DefaultHiveMetaHook.java) was introduced in Hive-2.X , so there shouldn't be any issue. 
   
   The problem I could think of is backward compatibility. Without the new changes in Hive, the commit information like jobId and number of tasks won't be stored in conf. Atleast for number of tasks it can revert to the old logic, but I couldn't find a way to get jobId from HiveMetaHook. 
   
   So users with older version Hive without this [PR](https://github.com/apache/hive/pull/4398) it will throw error. 
   
   Thinking if we should have a setting like `hive.tez.commit.job.am` set from Hive, if set to true to commit from HiveIcebergMetaHook else from commitJob call by the AM using HiveIcebergOutputCommitter.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org