[Impala-ASF-CR] IMPALA-10737: Optimize the number of Iceberg API Metadata requests

["Tamas Mate (Code Review)" <ge...@cloudera.org>]

Tamas Mate has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/18226 )

Change subject: IMPALA-10737: Optimize the number of Iceberg API Metadata requests
......................................................................

IMPALA-10737: Optimize the number of Iceberg API Metadata requests

Iceberg stores the table metadata next to the data files, when this is
accessed through the Iceberg API a filesystem call is executed (HDFS,
S3, ADLS). These calls were used in various places during query
processing and this patch unifies the Iceberg metadata request in the
CatalogD similar to other metadata requests:
 - CatalogD loads and caches the org.apache.iceberg.BaseTable object.
 - ImpalaDs requests the org.apache.iceberg.BaseTable from the
CatalogD and caches it as well.

As a result REFRESH/INVALIDATE METADATA is required to reload any
Iceberg metadata changes and the metadata load time is improved.
This improvement is more significant for smaller queries, where the
metadata request has larger impact on the query execution time.

Testing:
 - Passed Iceberg E2E tests.

Change-Id: I9e62a1fb9753ea1b022c7763047d9ccfd1d27d62
---
M common/thrift/CatalogObjects.thrift
M common/thrift/CatalogService.thrift
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
A fe/src/main/java/org/apache/impala/catalog/IcebergTableLoadingException.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java
M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java
M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalTable.java
M fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/util/IcebergUtil.java
M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-insert.test
17 files changed, 229 insertions(+), 175 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/18226/8
-- 
To view, visit http://gerrit.cloudera.org:8080/18226
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9e62a1fb9753ea1b022c7763047d9ccfd1d27d62
Gerrit-Change-Number: 18226
Gerrit-PatchSet: 8
Gerrit-Owner: Tamas Mate <tm...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>