You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Tamas Mate (Code Review)" <ge...@cloudera.org> on 2022/03/10 09:06:36 UTC
[Impala-ASF-CR] Revert "IMPALA-10737: Optimize the number of Iceberg API Metadata requests"
Tamas Mate has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18305
Change subject: Revert "IMPALA-10737: Optimize the number of Iceberg API Metadata requests"
......................................................................
Revert "IMPALA-10737: Optimize the number of Iceberg API Metadata requests"
This reverts commit cd10acdbb15c74eae21fa630bcb174104283560a.
This commit has been reverted, because it blocks upgrading the Iceberg
version to 0.13.
In the newer Iceberg version the BaseTable serialization has been
changed, it serializes the BaseTable to a SerializableTable sibiling
class. This is a lightweigth Table class which does not have the
necessary metadata that could be cached and reused by the ImpalaDs.
SerializableTable utilization has to be further considered.
Change-Id: I21e65cb3ab38d9e683223fb100d7ced90caa6edd
---
M common/thrift/CatalogObjects.thrift
M common/thrift/CatalogService.thrift
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
D fe/src/main/java/org/apache/impala/catalog/IcebergTableLoadingException.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java
M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java
M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalTable.java
M fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java
M fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/util/IcebergUtil.java
M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-insert.test
18 files changed, 182 insertions(+), 234 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/05/18305/1
--
To view, visit http://gerrit.cloudera.org:8080/18305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I21e65cb3ab38d9e683223fb100d7ced90caa6edd
Gerrit-Change-Number: 18305
Gerrit-PatchSet: 1
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] Revert "IMPALA-10737: Optimize the number of Iceberg API Metadata requests"
Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.
Tamas Mate has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18305 )
Change subject: Revert "IMPALA-10737: Optimize the number of Iceberg API Metadata requests"
......................................................................
Revert "IMPALA-10737: Optimize the number of Iceberg API Metadata requests"
This reverts commit cd10acdbb15c74eae21fa630bcb174104283560a.
This commit has been reverted, because it blocks upgrading the Iceberg
version to 0.13.
In the newer Iceberg version the BaseTable serialization has been
changed, it serializes the BaseTable to a SerializableTable sibiling
class. This is a lightweigth Table class which does not have the
necessary metadata that could be cached and reused by the ImpalaDs.
SerializableTable utilization has to be further considered.
Change-Id: I21e65cb3ab38d9e683223fb100d7ced90caa6edd
Reviewed-on: http://gerrit.cloudera.org:8080/18305
Reviewed-by: Zoltan Borok-Nagy <bo...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M common/thrift/CatalogObjects.thrift
M common/thrift/CatalogService.thrift
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
D fe/src/main/java/org/apache/impala/catalog/IcebergTableLoadingException.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java
M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java
M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalTable.java
M fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java
M fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/util/IcebergUtil.java
M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-insert.test
18 files changed, 182 insertions(+), 234 deletions(-)
Approvals:
Zoltan Borok-Nagy: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/18305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I21e65cb3ab38d9e683223fb100d7ced90caa6edd
Gerrit-Change-Number: 18305
Gerrit-PatchSet: 2
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] Revert "IMPALA-10737: Optimize the number of Iceberg API Metadata requests"
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18305 )
Change subject: Revert "IMPALA-10737: Optimize the number of Iceberg API Metadata requests"
......................................................................
Patch Set 1:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7924/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/18305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I21e65cb3ab38d9e683223fb100d7ced90caa6edd
Gerrit-Change-Number: 18305
Gerrit-PatchSet: 1
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 10 Mar 2022 10:30:37 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-10737: Optimize the number of Iceberg API Metadata requests"
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18305 )
Change subject: Revert "IMPALA-10737: Optimize the number of Iceberg API Metadata requests"
......................................................................
Patch Set 1: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/18305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I21e65cb3ab38d9e683223fb100d7ced90caa6edd
Gerrit-Change-Number: 18305
Gerrit-PatchSet: 1
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 10 Mar 2022 15:28:53 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-10737: Optimize the number of Iceberg API Metadata requests"
Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/18305 )
Change subject: Revert "IMPALA-10737: Optimize the number of Iceberg API Metadata requests"
......................................................................
Patch Set 2:
Thank you for the review Zoltan!
--
To view, visit http://gerrit.cloudera.org:8080/18305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I21e65cb3ab38d9e683223fb100d7ced90caa6edd
Gerrit-Change-Number: 18305
Gerrit-PatchSet: 2
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 10 Mar 2022 15:35:38 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-10737: Optimize the number of Iceberg API Metadata requests"
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18305 )
Change subject: Revert "IMPALA-10737: Optimize the number of Iceberg API Metadata requests"
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/10276/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I21e65cb3ab38d9e683223fb100d7ced90caa6edd
Gerrit-Change-Number: 18305
Gerrit-PatchSet: 1
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 10 Mar 2022 09:26:20 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-10737: Optimize the number of Iceberg API Metadata requests"
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/18305 )
Change subject: Revert "IMPALA-10737: Optimize the number of Iceberg API Metadata requests"
......................................................................
Patch Set 1: Code-Review+2
Thanks Tamas for fixing this!
--
To view, visit http://gerrit.cloudera.org:8080/18305
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I21e65cb3ab38d9e683223fb100d7ced90caa6edd
Gerrit-Change-Number: 18305
Gerrit-PatchSet: 1
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 10 Mar 2022 10:38:21 +0000
Gerrit-HasComments: No