You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org> on 2025/02/24 17:24:57 UTC
[Impala-ASF-CR] IMPALA-13785: test catalog tables stats legacy catalog and test catalog tables stats local catalog seem to be flaky in the exhaustive build
Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/22531
Change subject: IMPALA-13785: test_catalog_tables_stats_legacy_catalog and test_catalog_tables_stats_local_catalog seem to be flaky in the exhaustive build
......................................................................
IMPALA-13785: test_catalog_tables_stats_legacy_catalog and test_catalog_tables_stats_local_catalog seem to be flaky in the exhaustive build
IMPALA-13737 changed how we load file metadata for Iceberg tables,
as we don't rely on the underlying HdfsTable anymore. As a result,
metrics were not correctly updated. This patch adds code to update
metrics correctly.
Testing:
* executed the tests in the title in exhaustive mode
Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
---
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
3 files changed, 14 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/22531/1
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-13785: Deflake test catalog tables stats in exhaustive build
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22531 )
Change subject: IMPALA-13785: Deflake test_catalog_tables_stats in exhaustive build
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/18364/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Noemi Pap-Takacs <np...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Comment-Date: Tue, 25 Feb 2025 13:57:56 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13785: Deflake test catalog tables stats in exhaustive build
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/22531 )
Change subject: IMPALA-13785: Deflake test_catalog_tables_stats in exhaustive build
......................................................................
Patch Set 2: Code-Review+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/22531/2/tests/custom_cluster/test_web_pages.py
File tests/custom_cluster/test_web_pages.py:
http://gerrit.cloudera.org:8080/#/c/22531/2/tests/custom_cluster/test_web_pages.py@45
PS2, Line 45: if cls.exploration_strategy() != 'exhaustive':
: pytest.skip('runs only in exhaustive')
Some tests in this class run shortly:
test_webserver_hide_logs_link 4.5 sec Passed
test_cdp_header_disabled 4.5 sec Passed
test_query_log_size_in_bytes 4.7 sec Passed
test_query_stmt_without_truncate 4.8 sec Passed
test_query_stmt_with_custom_length 4.8 sec Passed
test_catalog_operations_limit 5.6 sec Passed
test_max_post_length 6.3 sec Passed
test_transient_topic_size 7.4 sec Passed
test_catalog_operations_with_rpc_retry 7.9 sec Passed
test_varz_hidden_variables 8 sec Passed
test_catalog_tables_stats_legacy_catalog 8.3 sec Failed
test_catalog_tables_stats_local_catalog 8.6 sec Failed
test_webserver_url_not_exposed 8.7 sec Passed
test_webserver_interface 10 sec Passed
test_event_processor_error_message 17 sec Passed
test_event_processor_status 24 sec Passed
test_catalog_operations_negative_limit 31 sec Passed
It'd be better to move some of them to a new class and run them in CORE builds, at least the two that failed this time.
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Noemi Pap-Takacs <np...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Feb 2025 08:10:11 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-13785: test catalog tables stats legacy catalog and test catalog tables stats local catalog seem to be flaky in the exhaustive build
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/22531 )
Change subject: IMPALA-13785: test_catalog_tables_stats_legacy_catalog and test_catalog_tables_stats_local_catalog seem to be flaky in the exhaustive build
......................................................................
Patch Set 1: Code-Review+2
(2 comments)
http://gerrit.cloudera.org:8080/#/c/22531/1/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
File fe/src/main/java/org/apache/impala/catalog/IcebergTable.java:
http://gerrit.cloudera.org:8080/#/c/22531/1/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java@478
PS1, Line 478: long memUsageEstimate = stats.numFiles * PER_FD_MEM_USAGE_BYTES +
: stats.numBlocks * PER_BLOCK_MEM_USAGE_BYTES;
I think we should improve this in the future to account for cached Iceberg objects.
http://gerrit.cloudera.org:8080/#/c/22531/1/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java@481
PS1, Line 481: setNumFiles(stats.numFiles);
It's not revealed by the test. But the total number of blocks and total file size are also missing. We can get them in catalogd WebUI -> /catalog, then click any item in the Metrics column. For HDFS tables, more metrics are shown there.
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Noemi Pap-Takacs <np...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Comment-Date: Tue, 25 Feb 2025 02:29:44 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-13785: Deflake test catalog tables stats in exhaustive build
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/22531 )
Change subject: IMPALA-13785: Deflake test_catalog_tables_stats in exhaustive build
......................................................................
Patch Set 2: Code-Review+2
Code looks good.
I see no major objection from other reviewers.
I'm giving my +2 vote so we can stabilize the flaky tests.
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Noemi Pap-Takacs <np...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Feb 2025 22:05:30 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13785: test catalog tables stats legacy catalog and test catalog tables stats local catalog seem to be flaky in the exhaustive build
Posted by "Xuebin Su (Code Review)" <ge...@cloudera.org>.
Xuebin Su has posted comments on this change. ( http://gerrit.cloudera.org:8080/22531 )
Change subject: IMPALA-13785: test_catalog_tables_stats_legacy_catalog and test_catalog_tables_stats_local_catalog seem to be flaky in the exhaustive build
......................................................................
Patch Set 1: Code-Review+1
Thanks!
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Noemi Pap-Takacs <np...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Comment-Date: Tue, 25 Feb 2025 02:40:52 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13785: Deflake test catalog tables stats in exhaustive build
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/22531 )
Change subject: IMPALA-13785: Deflake test_catalog_tables_stats in exhaustive build
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/22531/2/tests/custom_cluster/test_web_pages.py
File tests/custom_cluster/test_web_pages.py:
http://gerrit.cloudera.org:8080/#/c/22531/2/tests/custom_cluster/test_web_pages.py@45
PS2, Line 45: if cls.exploration_strategy() != 'exhaustive':
: pytest.skip('runs only in exhaustive')
> Some tests in this class run shortly:
I think it's a good idea, but it's probably better to do it under a separate Jira ticket. I filed IMPALA-13808 for this.
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Noemi Pap-Takacs <np...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Feb 2025 16:02:02 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-13785: Deflake test catalog tables stats in exhaustive build
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22531 )
Change subject: IMPALA-13785: Deflake test_catalog_tables_stats in exhaustive build
......................................................................
Patch Set 2:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/11513/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Noemi Pap-Takacs <np...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Comment-Date: Wed, 26 Feb 2025 09:34:11 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13785: Deflake test catalog tables stats in exhaustive build
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Riza Suminto, Xuebin Su, Noemi Pap-Takacs, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/22531
to look at the new patch set (#2).
Change subject: IMPALA-13785: Deflake test_catalog_tables_stats in exhaustive build
......................................................................
IMPALA-13785: Deflake test_catalog_tables_stats in exhaustive build
IMPALA-13737 changed how we load file metadata for Iceberg tables,
as we don't rely on the underlying HdfsTable anymore. As a result,
metrics were not correctly updated. This patch adds code to update
metrics correctly.
Testing:
* executed the tests in the title in exhaustive mode
* added Iceberg-specific tests to test_web_pages
Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
---
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/FileMetadataLoader.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergFileMetadataLoader.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
M tests/webserver/test_web_pages.py
7 files changed, 127 insertions(+), 72 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/22531/2
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Noemi Pap-Takacs <np...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
[Impala-ASF-CR] IMPALA-13785: Deflake test catalog tables stats in exhaustive build
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22531 )
Change subject: IMPALA-13785: Deflake test_catalog_tables_stats in exhaustive build
......................................................................
Patch Set 2: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Noemi Pap-Takacs <np...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Feb 2025 21:28:07 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13785: Deflake test catalog tables stats in exhaustive build
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22531 )
Change subject: IMPALA-13785: Deflake test_catalog_tables_stats in exhaustive build
......................................................................
Patch Set 2:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/11528/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Noemi Pap-Takacs <np...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Feb 2025 16:02:20 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13785: Deflake test catalog tables stats in exhaustive build
Posted by "Noemi Pap-Takacs (Code Review)" <ge...@cloudera.org>.
Noemi Pap-Takacs has posted comments on this change. ( http://gerrit.cloudera.org:8080/22531 )
Change subject: IMPALA-13785: Deflake test_catalog_tables_stats in exhaustive build
......................................................................
Patch Set 2: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Noemi Pap-Takacs <np...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 27 Feb 2025 14:49:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13785: Deflake test catalog tables stats in exhaustive build
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/22531 )
Change subject: IMPALA-13785: Deflake test_catalog_tables_stats in exhaustive build
......................................................................
IMPALA-13785: Deflake test_catalog_tables_stats in exhaustive build
IMPALA-13737 changed how we load file metadata for Iceberg tables,
as we don't rely on the underlying HdfsTable anymore. As a result,
metrics were not correctly updated. This patch adds code to update
metrics correctly.
Testing:
* executed the tests in the title in exhaustive mode
* added Iceberg-specific tests to test_web_pages
Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Reviewed-on: http://gerrit.cloudera.org:8080/22531
Reviewed-by: Quanlong Huang <hu...@gmail.com>
Reviewed-by: Noemi Pap-Takacs <np...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Riza Suminto <ri...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/FileMetadataLoader.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergFileMetadataLoader.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
M tests/webserver/test_web_pages.py
7 files changed, 127 insertions(+), 70 deletions(-)
Approvals:
Quanlong Huang: Looks good to me, but someone else must approve
Noemi Pap-Takacs: Looks good to me, but someone else must approve
Impala Public Jenkins: Verified
Riza Suminto: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Noemi Pap-Takacs <np...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-13785: test catalog tables stats legacy catalog and test catalog tables stats local catalog seem to be flaky in the exhaustive build
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/22531 )
Change subject: IMPALA-13785: test_catalog_tables_stats_legacy_catalog and test_catalog_tables_stats_local_catalog seem to be flaky in the exhaustive build
......................................................................
Patch Set 1: Code-Review+1
(2 comments)
http://gerrit.cloudera.org:8080/#/c/22531/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/22531/1//COMMIT_MSG@7
PS1, Line 7: test_catalog_tables_stats_legacy_catalog and test_catalog_tables_stats_local_catalog seem to be flaky in the exhaustive build
nit: Title can be shorter:
IMPALA-13785: Deflake test_catalog_tables_stats in exhaustive build.
http://gerrit.cloudera.org:8080/#/c/22531/1//COMMIT_MSG@15
PS1, Line 15: * executed the tests in the title in exhaustive mode
I tested this patch locally and it works.
impala-py.test --exploration=exhaustive -k test_catalog_tables_stats custom_cluster/test_web_pages.py
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Noemi Pap-Takacs <np...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Comment-Date: Mon, 24 Feb 2025 20:03:47 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-13785: Deflake test catalog tables stats in exhaustive build
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22531 )
Change subject: IMPALA-13785: Deflake test_catalog_tables_stats in exhaustive build
......................................................................
Patch Set 2: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/11513/
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Noemi Pap-Takacs <np...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Comment-Date: Wed, 26 Feb 2025 14:56:44 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13785: Deflake test catalog tables stats in exhaustive build
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/22531 )
Change subject: IMPALA-13785: Deflake test_catalog_tables_stats in exhaustive build
......................................................................
Patch Set 2:
(4 comments)
Thanks for the comments!
http://gerrit.cloudera.org:8080/#/c/22531/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/22531/1//COMMIT_MSG@7
PS1, Line 7: Deflake test_catalog_tables_stats in exhaustive build
> nit: Title can be shorter:
Done
http://gerrit.cloudera.org:8080/#/c/22531/1//COMMIT_MSG@15
PS1, Line 15: * executed the tests in the title in exhaustive mode
> I tested this patch locally and it works.
Done
http://gerrit.cloudera.org:8080/#/c/22531/1/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
File fe/src/main/java/org/apache/impala/catalog/IcebergTable.java:
http://gerrit.cloudera.org:8080/#/c/22531/1/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java@478
PS1, Line 478: getHostIndex(), Preconditions.checkNotNull(icebergFiles),
: Utils.requiresDataFilesInTableLocation(t
> I think we should improve this in the future to account for cached Iceberg
I agree, opened IMPALA-13794 to track this.
http://gerrit.cloudera.org:8080/#/c/22531/1/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java@481
PS1, Line 481: fileStore_ = new IcebergContentFileStore(
> It's not revealed by the test. But the total number of blocks and total fil
Added these metrics for Iceberg tables, also added tests for them.
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Noemi Pap-Takacs <np...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 26 Feb 2025 15:32:28 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-13785: test catalog tables stats legacy catalog and test catalog tables stats local catalog seem to be flaky in the exhaustive build
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/22531 )
Change subject: IMPALA-13785: test_catalog_tables_stats_legacy_catalog and test_catalog_tables_stats_local_catalog seem to be flaky in the exhaustive build
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/18347/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/22531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I686e0e32b36ace4c5a8c9eeb715148c98521252a
Gerrit-Change-Number: 22531
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Noemi Pap-Takacs <np...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Comment-Date: Mon, 24 Feb 2025 17:51:10 +0000
Gerrit-HasComments: No