You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2023/05/22 03:32:00 UTC
[jira] [Updated] (HIVE-27355) Iceberg: Create table can be slow due to file listing for stats
[ https://issues.apache.org/jira/browse/HIVE-27355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated HIVE-27355:
----------------------------------
Labels: pull-request-available (was: )
> Iceberg: Create table can be slow due to file listing for stats
> ---------------------------------------------------------------
>
> Key: HIVE-27355
> URL: https://issues.apache.org/jira/browse/HIVE-27355
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
> Reporter: Rajesh Balamohan
> Priority: Major
> Labels: pull-request-available
>
> Stacktrace can be different for hive master branch. But issue is, stats need not be populated for iceberg tables and currently it is doing recursive calls causing delays during table creation (e.g CTAS).
>
> {noformat}
> at org.apache.hadoop.hive.common.FileUtils.listStatusRecursively(FileUtils.java:329)
> at org.apache.hadoop.hive.common.FileUtils.listStatusRecursively(FileUtils.java:330)
> at org.apache.hadoop.hive.common.FileUtils.listStatusRecursively(FileUtils.java:330)
> at org.apache.hadoop.hive.common.HiveStatsUtils.getFileStatusRecurse(HiveStatsUtils.java:61)
> at org.apache.hadoop.hive.metastore.Warehouse.getFileStatusesForUnpartitionedTable(Warehouse.java:581)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.updateTableStatsFast(MetaStoreUtils.java:201)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.updateTableStatsFast(MetaStoreUtils.java:194)
> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1445)
> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1502)
> at sun.reflect.GeneratedMethodAccessor118.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> at com.sun.proxy.$Proxy49.create_table_with_environment_context(Unknown Source)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.create_table_with_environment_context(HiveMetaStoreClient.java:2419)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:755)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:743)
> at sun.reflect.GeneratedMethodAccessor117.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)