[GitHub] [hive] ayushtkn commented on a diff in pull request #4165: HIVE-27185: Iceberg: Cache iceberg table while loading for stats.

["ayushtkn (via GitHub)" <gi...@apache.org>]

ayushtkn commented on code in PR #4165:
URL: https://github.com/apache/hive/pull/4165#discussion_r1153050938


##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/IcebergTableUtil.java:
##########
@@ -52,12 +52,13 @@ private IcebergTableUtil() {
    * Constructs the table properties needed for the Iceberg table loading by retrieving the information from the
    * hmsTable. It then calls {@link IcebergTableUtil#getTable(Configuration, Properties)} with these properties.
    * @param configuration a Hadoop configuration
-   * @param hmsTable the HMS table
-   * @param skipCache if set to true there won't be an attempt to retrieve the table from SessionState
+   * @param hmsTable      the HMS table
+   * @param skipCache     if set to true there won't be an attempt to retrieve the table from SessionState
+   * @param suffix        the suffix to use for cache.
    * @return the Iceberg table
    */
   static Table getTable(Configuration configuration, org.apache.hadoop.hive.metastore.api.Table hmsTable,
-      boolean skipCache) {
+      boolean skipCache, String suffix) {

Review Comment:
   We want to cache the table for Stats only, if you fetch once for Stats, then use that till end, if you fetch for something else, we don't use that.
   
   In most cases that won't be a problem, but the way we do CTAS and MV's it did creates problem. It first creates an iceberg table and then does the load into that, so in that cases caching would have stale table. Got a test failure as well for those MV cases in the last run



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org