You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2022/11/15 01:10:00 UTC
[jira] [Commented] (IMPALA-11723) Use separate cache in CatalogdMetaProvider for caching Iceberg table objects
[ https://issues.apache.org/jira/browse/IMPALA-11723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17634099#comment-17634099 ]
Quanlong Huang commented on IMPALA-11723:
-----------------------------------------
If they are larger than other objects, aren't they evicted first?
[https://github.com/apache/impala/blob/522ee1fcc09d47074c75440ba4fc3d258e1c95b3/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java#L366]
[https://github.com/apache/impala/blob/522ee1fcc09d47074c75440ba4fc3d258e1c95b3/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java#L2013-L2019]
Or do we just want to invalidate the stale objects in time? Note that we already do so for stale partitions:
https://github.com/apache/impala/blob/522ee1fcc09d47074c75440ba4fc3d258e1c95b3/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java#L1470-L1473
> Use separate cache in CatalogdMetaProvider for caching Iceberg table objects
> ----------------------------------------------------------------------------
>
> Key: IMPALA-11723
> URL: https://issues.apache.org/jira/browse/IMPALA-11723
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog, Frontend
> Reporter: Zoltán Borók-Nagy
> Priority: Major
> Labels: impala-iceberg, ramp-up
>
> Iceberg table objects can be very large. Currently they are being stored in the meta provider's cache with other metadata objects. We load them by the
> MetaProvider.loadIcebergApiTable() method.
> Probably we should evict them more often, especially if the table is updated in CatalogD.
> It would make sense to store them in a separate cache.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org