You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Shohei Okumiya (Jira)" <ji...@apache.org> on 2025/06/14 15:54:00 UTC

[jira] [Commented] (HIVE-29016) Disable caching on the Iceberg REST Catalog

    [ https://issues.apache.org/jira/browse/HIVE-29016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17975999#comment-17975999 ] 

Shohei Okumiya commented on HIVE-29016:
---------------------------------------

The reference implementation of the server doesn't cache anything.

[https://github.com/apache/iceberg/blob/apache-iceberg-1.9.1/core/src/test/java/org/apache/iceberg/rest/RESTCatalogAdapter.java]

 

CachingCatalog is used in SparkCatalog or FlinkCatalog, it's a implementation of a client-side catalog.

> Disable caching on the Iceberg REST Catalog
> -------------------------------------------
>
>                 Key: HIVE-29016
>                 URL: https://issues.apache.org/jira/browse/HIVE-29016
>             Project: Hive
>          Issue Type: Improvement
>          Components: Iceberg integration, Standalone Metastore
>            Reporter: Shohei Okumiya
>            Priority: Major
>              Labels: hive-4.1.0-must
>
> The current implementation caches entries based on the property `
> metastore.iceberg.catalog.cache.expiry`.
> [https://github.com/apache/hive/blob/4193a6db6243bfd8a5a2a3d142bef5acdb02e389/standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/HMSCachingCatalog.java]
>  
> I observed that this made CREATE TABLE from Trino non-functional. With CachingCatalog, [a transaction is created as a CommitCallbackTransaction|https://github.com/apache/iceberg/blob/62d9ff5d043a5571efe020b9177998ae763a41a0/core/src/main/java/org/apache/iceberg/CachingCatalog.java#L286-L303]. However, CatalogHandler can't process the class, as CommitCallbackTransaction is not a sub-class of BaseTransaction. Our implementation is probably wrong.
>  * [https://github.com/apache/hive/blob/4193a6db6243bfd8a5a2a3d142bef5acdb02e389/standalone-metastore/metastore-rest-catalog/src/main/java/org/apache/iceberg/rest/HMSCatalogAdapter.java#L420]
>  * [https://github.com/apache/iceberg/blob/62d9ff5d043a5571efe020b9177998ae763a41a0/core/src/main/java/org/apache/iceberg/rest/CatalogHandlers.java#L350-L356]
>  
> Essentially, I believe that caching Iceberg objects is a form of premature optimization. What if a user accesses HMS via Iceberg REST API and another accesses the same table via Thrift API? What if Hive Metastore is configured with HA? It is likely causing a issue.
> In this ticket, I propse to remove the caching mechanism once.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)