You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org> on 2026/04/27 09:27:22 UTC
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Mihaly Szjatinya has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24243
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
This patch extends the skipping mechanism from IMPALA-6671 for Iceberg
tables. It uses the Iceberg table's own tableLock, and the
lastVersionSeenByTopicUpdate mechanism of the underlying HDFS table.
Testing:
Added TestTopicUpdateFrequency::test_topic_updates_*_iceberg in 4
variants
Fixed the original HDFS tests:
1. test_topic_updates_unblock
- By default it was running in Local Catalog mode, which has no effect
for fast DML/DQL queries. Added variants for both Legacy and Local
Catalog mode to demonstrate the case.
- In the blocking scenario there was a missing assert (!), making the
test always pass.
- Fast queries are much faster than stated, which doesn't seem to
matter however for the nature of the test.
2. test_topic_updates_advance
- The test claimed to test catalog_max_lock_skipped_topic_updates but
experimentally I could see no counter blockings triggered at all
under any configuration. Added more parallel threads to reliably
trigger the blockings.
- Query execution time expectations largely incorrect. Perhaps for
some reason it changed over time.
- Assert expects the max query time to be no more than a predictable
value. Which conceptually makes sense for SYNC_DDL, but I wasn't
able to reliably reproduce the case yet.
- Hence for now at least checking the blockings occurring in catalog
logs.
3. Removed test_topic_lock_timeout_disabled, it is now covered by one of
the test_topic_updates_unblock variants
Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
---
M be/src/catalog/catalog-server.cc
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M tests/custom_cluster/test_topic_update_frequency.py
4 files changed, 191 insertions(+), 81 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/24243/1
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 1
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
This patch extends the skipping mechanism from IMPALA-6671 for Iceberg
tables. It uses the Iceberg table's own tableLock, and the
lastVersionSeenByTopicUpdate mechanism of the underlying HDFS table.
Testing:
Added TestTopicUpdateFrequency::test_topic_updates_*_iceberg in 4
variants
Fixed the original HDFS tests:
1. test_topic_updates_unblock
- By default it was running in Local Catalog mode, which has no effect
for fast DML/DQL queries. Added variants for both Legacy and Local
Catalog mode to demonstrate the case.
- In the blocking scenario there was a missing assert (!), making the
test always pass.
- Fast queries are much faster than stated, which doesn't seem to
matter however for the nature of the test.
2. test_topic_updates_advance
- The test claimed to test catalog_max_lock_skipped_topic_updates but
experimentally I could see no counter blockings triggered at all
under any configuration. Added more parallel threads to reliably
trigger the blockings.
- Query execution time expectations largely incorrect. Perhaps for
some reason it changed over time.
- Assert expects the max query time to be no more than a predictable
value. Which conceptually makes sense for SYNC_DDL, but I wasn't
able to reliably reproduce the case yet.
- Hence for now at least checking the blockings occurring in catalog
logs.
3. Removed test_topic_lock_timeout_disabled, it is now covered by one of
the test_topic_updates_unblock variants
Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
---
M be/src/catalog/catalog-server.cc
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M tests/custom_cluster/test_topic_update_frequency.py
4 files changed, 199 insertions(+), 81 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/24243/2
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 2
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 8:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
File fe/src/main/java/org/apache/impala/catalog/IcebergTable.java:
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java@331
PS7, Line 331: // HdfsTable and get back the effective version in a single synchronized block
: long effectiveVersion = hdfsTable_.setCatalogVersionAndGet(newVersion);
: super.setCatalogVersion(effectiveVersion);
> I am not sure if there is an actual issue, but it seems confusing.
The way GatherThread works:
1. First it reads tblVersion without lock in 'addTableToCatalogDelta' to take a snapshot and do the coarse filtering. This is racy by design.
2. Then after obtaining the lock it re-checks tblVersion again already under the lock in 'addTableToCatalogDeltaHelper' and does the proper (fromVersion, toVersion] filtering. This can happen only query is finished and tableLock released.
In the lock-timeout path, GatherThread may call 'updatePendingVersion' using the earlier expected version.
If that expected version is stale because we are between the two lines, 'updatePendingVersion 'simply returns false and GatherThread retries lock once. But this case is conceptually no different from a query just still running on an earlier stage.
And the 'pendingVersionLock_' is specifically for pendingVersion, not the catalog version itself.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 8
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 May 2026 19:09:18 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 8:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/22329/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 8
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 May 2026 12:44:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 8:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
File fe/src/main/java/org/apache/impala/catalog/IcebergTable.java:
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java@331
PS7, Line 331: // HdfsTable and get back the effective version in a single synchronized block
: long effectiveVersion = hdfsTable_.setCatalogVersionAndGet(newVersion);
: super.setCatalogVersion(effectiveVersion);
> Isn't it a problem that this can go out of sync in case of parallel calls?
Good point. What about this?
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 8
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 May 2026 12:22:56 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 7:
Fixed the problem in IcebergTable.java. Pls take a look.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 7
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 May 2026 10:56:24 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 8:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
File fe/src/main/java/org/apache/impala/catalog/IcebergTable.java:
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java@331
PS7, Line 331: // HdfsTable and get back the effective version in a single synchronized block
: long effectiveVersion = hdfsTable_.setCatalogVersionAndGet(newVersion);
: super.setCatalogVersion(effectiveVersion);
> The way GatherThread works:
>And the 'pendingVersionLock_' is specifically for pendingVersion, not the catalog version itself.
I disagree with this point. pendingVersionLock_ guards all cases when catalogVersion_ is modified. This is why I think that this is a reasonable expectation for IcebergTable too.
(+ modifying of catalogVersion_ is also protected by tableLock_)
Reads of catalogVersion_ is a different thing, that is allowed intentionally without locking.
>But this case is conceptually no different from a query just still running on an earlier stage.
I agree, I also didn't see difference between the 2 cases. My argument for locking is to make it easier to understand by not having to think through the IcebergTable/HdfsTable version out of sync case.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 8
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 May 2026 19:56:33 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/22259/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 3
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 28 Apr 2026 08:38:09 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 3: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 3
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 29 Apr 2026 06:46:02 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: [WIP] Catalog topic update creation can't skip Iceberg tables
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: [WIP] Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 4:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/22287/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 4
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Apr 2026 21:40:00 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
This patch extends the skipping mechanism from IMPALA-6671 for Iceberg
tables. It uses the Iceberg table's own tableLock, and the
lastVersionSeenByTopicUpdate mechanism of the underlying HDFS table.
Testing:
Added TestTopicUpdateFrequency::test_topic_updates_*_iceberg in 4
variants
Fixed the original HDFS tests:
1. test_topic_updates_unblock
- By default it was running in Local Catalog mode, which has no effect
for fast DML/DQL queries. Added variants for both Legacy and Local
Catalog mode to demonstrate the case.
- In the blocking scenario there was a missing assert (!), making the
test always pass.
- Fast queries are much faster than stated, which doesn't seem to
matter however for the nature of the test.
- Reduced query delay times
2. test_topic_updates_advance
- The test claimed to test catalog_max_lock_skipped_topic_updates but
experimentally I could see no counter blockings triggered at all
under any configuration. Added more parallel threads to reliably
trigger the blockings.
- Query execution time expectations largely incorrect. Perhaps for
some reason it changed over time.
- Assert expects the max query time to be no more than a predictable
value. Which conceptually makes sense for SYNC_DDL, but I wasn't
able to reliably reproduce the case yet.
- Hence for now at least checking the blockings occurring in catalog
logs.
- Reduced query delay times
3. Removed test_topic_lock_timeout_disabled, it is now covered by one of
the test_topic_updates_unblock variants
Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
---
M be/src/catalog/catalog-server.cc
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
M tests/custom_cluster/test_topic_update_frequency.py
5 files changed, 348 insertions(+), 106 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/24243/8
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 8
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 9:
(4 comments)
Answered the thread. Also deflaked 'test_topic_updates_advance'.
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
File fe/src/main/java/org/apache/impala/catalog/IcebergTable.java:
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java@331
PS7, Line 331: // HdfsTable and get back the effective version in a single synchronized block
: long effectiveVersion = hdfsTable_.setCatalogVersionAndGet(newVersion);
: super.setCatalogVersion(effectiveVersion);
> This looks correct. But is there a plan to totally get rid of HdfsTable in
Yeah, that does seem relevant.
http://gerrit.cloudera.org:8080/#/c/24243/8/tests/custom_cluster/test_topic_update_frequency.py
File tests/custom_cluster/test_topic_update_frequency.py:
http://gerrit.cloudera.org:8080/#/c/24243/8/tests/custom_cluster/test_topic_update_frequency.py@148
PS8, Line 148: logic
> nit: shouldn't we use "__run_topic_update_block_sync_ddl_create_table" sinc
This inner function shouln't have 'unblock' at all in its name, as well as the others. Blocking/skipping comes from the caller test functions.
http://gerrit.cloudera.org:8080/#/c/24243/8/tests/custom_cluster/test_topic_update_frequency.py@149
PS8, Line 149: ble_name = "non_blocking_ddl_
> nit: it seems we don't need this parameter since the callers all have this
Ack
http://gerrit.cloudera.org:8080/#/c/24243/8/tests/custom_cluster/test_topic_update_frequency.py@451
PS8, Line 451:
> nit: this method is quite similar to __run_topic_update_unblock_test_sync_d
Absolutely
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 9
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 06 May 2026 13:01:04 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
This patch extends the skipping mechanism from IMPALA-6671 for Iceberg
tables. It uses the Iceberg table's own tableLock, and the
lastVersionSeenByTopicUpdate mechanism of the underlying HDFS table.
Testing:
Added TestTopicUpdateFrequency::test_topic_updates_*_iceberg in 4
variants
Fixed the original HDFS tests:
1. test_topic_updates_unblock
- By default it was running in Local Catalog mode, which has no effect
for fast DML/DQL queries. Added variants for both Legacy and Local
Catalog mode to demonstrate the case.
- In the blocking scenario there was a missing assert (!), making the
test always pass.
- Fast queries are much faster than stated, which doesn't seem to
matter however for the nature of the test.
- Reduced query delay times
2. test_topic_updates_advance
- The test claimed to test catalog_max_lock_skipped_topic_updates but
experimentally I could see no counter blockings triggered at all
under any configuration. Added more parallel threads to reliably
trigger the blockings.
- Query execution time expectations largely incorrect. Perhaps for
some reason it changed over time.
- Assert expects the max query time to be no more than a predictable
value. Which conceptually makes sense for SYNC_DDL, but I wasn't
able to reliably reproduce the case yet.
- Hence for now at least checking the blockings occurring in catalog
logs.
- Reduced query delay times
3. Removed test_topic_lock_timeout_disabled, it is now covered by one of
the test_topic_updates_unblock variants
Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
---
M be/src/catalog/catalog-server.cc
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
M tests/custom_cluster/test_topic_update_frequency.py
5 files changed, 313 insertions(+), 109 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/24243/10
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 10
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
This patch extends the skipping mechanism from IMPALA-6671 for Iceberg
tables. It uses the Iceberg table's own tableLock, and the
lastVersionSeenByTopicUpdate mechanism of the underlying HDFS table.
Testing:
Added TestTopicUpdateFrequency::test_topic_updates_*_iceberg in 4
variants
Fixed the original HDFS tests:
1. test_topic_updates_unblock
- By default it was running in Local Catalog mode, which has no effect
for fast DML/DQL queries. Added variants for both Legacy and Local
Catalog mode to demonstrate the case.
- In the blocking scenario there was a missing assert (!), making the
test always pass.
- Fast queries are much faster than stated, which doesn't seem to
matter however for the nature of the test.
2. test_topic_updates_advance
- The test claimed to test catalog_max_lock_skipped_topic_updates but
experimentally I could see no counter blockings triggered at all
under any configuration. Added more parallel threads to reliably
trigger the blockings.
- Query execution time expectations largely incorrect. Perhaps for
some reason it changed over time.
- Assert expects the max query time to be no more than a predictable
value. Which conceptually makes sense for SYNC_DDL, but I wasn't
able to reliably reproduce the case yet.
- Hence for now at least checking the blockings occurring in catalog
logs.
3. Removed test_topic_lock_timeout_disabled, it is now covered by one of
the test_topic_updates_unblock variants
Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
---
M be/src/catalog/catalog-server.cc
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M tests/custom_cluster/test_topic_update_frequency.py
4 files changed, 203 insertions(+), 84 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/24243/3
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 3
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 2: Code-Review+1
Thanks for updating these old tests!
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 2
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 27 Apr 2026 15:21:30 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 12: Verified+1
The failed tests is related to a know flaky maven artifact https://jenkins.impala.io/job/all-build-options-ub2004/4226/
Real tests have passes, so I am merging this.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 12
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 11 May 2026 14:19:44 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 11: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 11
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 May 2026 11:48:40 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Balazs Hevele (Code Review)" <ge...@cloudera.org>.
Balazs Hevele has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24243/2/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java:
http://gerrit.cloudera.org:8080/#/c/24243/2/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@1801
PS2, Line 1801: if the table is a {@link HdfsTable}
Doesn't this comment need to be updated?
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 2
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 28 Apr 2026 08:01:09 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has uploaded a new patch set (#11). ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
This patch extends the skipping mechanism from IMPALA-6671 for Iceberg
tables. It uses the Iceberg table's own tableLock, and the
lastVersionSeenByTopicUpdate mechanism of the underlying HDFS table.
Testing:
Added TestTopicUpdateFrequency::test_topic_updates_*_iceberg in 4
variants
Fixed the original HDFS tests:
1. test_topic_updates_unblock
- By default it was running in Local Catalog mode, which has no effect
for fast DML/DQL queries. Added variants for both Legacy and Local
Catalog mode to demonstrate the case.
- In the blocking scenario there was a missing assert (!), making the
test always pass.
- Fast queries are much faster than stated, which doesn't seem to
matter however for the nature of the test.
- Reduced query delay times
2. test_topic_updates_advance
- The test claimed to test catalog_max_lock_skipped_topic_updates but
experimentally I could see no counter blockings triggered at all
under any configuration.
- Added more parallel threads and reduced
catalog_max_lock_skipped_topic_updates to 2 to reliably trigger the
blockings. Ran 20 times locally to verify.
- Query execution time expectations largely incorrect. Perhaps for
some reason it changed over time.
- Assert expects the max query time to be no more than a predictable
value. Which conceptually makes sense for SYNC_DDL, but I wasn't
able to reliably reproduce the case yet.
- Hence for now at least checking the blockings occurring in catalog
logs.
- Reduced query delay times
3. Removed test_topic_lock_timeout_disabled, it is now covered by one of
the test_topic_updates_unblock variants
Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
---
M be/src/catalog/catalog-server.cc
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
M tests/custom_cluster/test_topic_update_frequency.py
5 files changed, 314 insertions(+), 109 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/24243/11
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 11
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 11:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24243/10/tests/custom_cluster/test_topic_update_frequency.py
File tests/custom_cluster/test_topic_update_frequency.py:
http://gerrit.cloudera.org:8080/#/c/24243/10/tests/custom_cluster/test_topic_update_frequency.py@281
PS10, Line 281: # TODO: find better way to test catalog_max_lock_skipped_topic_updates
: self.assert_catalogd_log_contains(
: "INFO",
> Running the test twice without restart would make this pass automatically,
Ack
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 11
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 May 2026 08:50:48 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 11:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/22362/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 11
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 May 2026 09:12:04 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 3:
(6 comments)
Thanks for updating the old tests! I also take this chance to review them again.
The code changes LGTM. Just left some comments about tests.
http://gerrit.cloudera.org:8080/#/c/24243/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/24243/3//COMMIT_MSG@24
PS3, Line 24: - Fast queries are much faster than stated, which doesn't seem to
They take >2s in the legacy catalog mode since the default statestore_update_frequency_ms is 2s. But in the tests we use statestore_update_frequency_ms=50 so they run faster than stated.
https://github.com/apache/impala/blob/10772982f204c06deb68a44ffcd66eebd86ba615/tests/common/custom_cluster_test_suite.py#L89
http://gerrit.cloudera.org:8080/#/c/24243/3/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java:
http://gerrit.cloudera.org:8080/#/c/24243/3/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@1801
PS3, Line 1801: Link
nit: "L" should be lowercase -> "link"
http://gerrit.cloudera.org:8080/#/c/24243/3/tests/custom_cluster/test_topic_update_frequency.py
File tests/custom_cluster/test_topic_update_frequency.py:
http://gerrit.cloudera.org:8080/#/c/24243/3/tests/custom_cluster/test_topic_update_frequency.py@57
PS3, Line 57: via RPC and don't depend on topic updates."""
DDL/DMLs with sync_ddl=true still depend on topic updates to make sure other coordinators have receive the updates. Both catalog modes have the same behavior here:
https://github.com/apache/impala/blob/10772982f204c06deb68a44ffcd66eebd86ba615/be/src/service/impala-server.cc#L2480
Since we are testing topic updates and DDL/DMLs with sync_ddl=true are important consumers of it, it'd be better to add test on them. E.g. add a simple createTable statement in fast_queries. We might need to distinguish DDL/DMLs with sync_ddl=true and use longer wait time.
http://gerrit.cloudera.org:8080/#/c/24243/3/tests/custom_cluster/test_topic_update_frequency.py@112
PS3, Line 112: # blocking query is without sync_ddl
: blocking_query_options["sync_ddl"] = "false"
: self.__run_topic_update_test(blocking_query,
: non_blocking_queries, init_queries, blocking_query_options=blocking_query_options,
: expect_topic_updates_to_block=expect_topic_updates_to_block)
: # blocking query is with sync_ddl
: blocking_query_options["sync_ddl"] = "true"
: self.__run_topic_update_test(blocking_query,
: non_blocking_queries, init_queries, blocking_query_options=blocking_query_options,
: expect_topic_updates_to_block=expect_topic_updates_to_block)
:
: non_blocking_query_options = {
: "sync_ddl": "true",
: }
: self.__run_topic_update_test(blocking_query,
: non_blocking_queries, init_queries, blocking_query_options=blocking_query_options,
: non_blocking_query_options=non_blocking_query_options,
: expect_topic_updates_to_block=expect_topic_updates_to_block)
: blocking_query_options["sync_ddl"] = "false"
: self.__run_topic_update_test(blocking_query,
: non_blocking_queries, init_queries, blocking_query_options=blocking_query_options,
: non_blocking_query_options=non_blocking_query_options,
: expect_topic_updates_to_block=expect_topic_updates_to_block)
nit: these can be simplified
for blocking_sync_ddl, non_blocking_sync_ddl in (
("false", "false"),
("true", "false"),
("true", "true"),
("false", "true"),
):
blocking_query_options["sync_ddl"] = blocking_sync_ddl
non_blocking_query_options = {"sync_ddl": non_blocking_sync_ddl}
self.__run_topic_update_test(blocking_query,
non_blocking_queries, init_queries,
blocking_query_options=blocking_query_options,
non_blocking_query_options=non_blocking_query_options,
expect_topic_updates_to_block=expect_topic_updates_to_block)
http://gerrit.cloudera.org:8080/#/c/24243/3/tests/custom_cluster/test_topic_update_frequency.py@153
PS3, Line 153: len(slow_blocking_query)
This seems wrong. 'slow_blocking_query' is a string, not a list. We can simply use 1 here.
http://gerrit.cloudera.org:8080/#/c/24243/3/tests/custom_cluster/test_topic_update_frequency.py@360
PS3, Line 360: "sync_ddl": "false"
Can we also test sync_ddl=true in blocking queries and DDLs/DMLs (if we add) in non_blocking_queries?
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 3
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Apr 2026 04:50:49 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 1:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/24243/1/tests/custom_cluster/test_topic_update_frequency.py
File tests/custom_cluster/test_topic_update_frequency.py:
http://gerrit.cloudera.org:8080/#/c/24243/1/tests/custom_cluster/test_topic_update_frequency.py@348
PS1, Line 348: catalogd_update_stats_delay
just a side note:
ideally impala would detect that the table didn't change at all (same snapshot), and if this optimizatino would be implemented, then we wouldn't execute this debug action
as you have fixed the test to check that Impala actually blocks, we would detect this issue, so it is not a real problem
http://gerrit.cloudera.org:8080/#/c/24243/1/tests/custom_cluster/test_topic_update_frequency.py@354
PS1, Line 354: iceberg_partitioned
I would prefer to create new table in unique_database to avoid mutating shared tables.
The old test also mutates tpcds.store_sales and functional.alltypes, which is also not great, but I don't mind if this change doesn't change that old logic.
http://gerrit.cloudera.org:8080/#/c/24243/1/tests/custom_cluster/test_topic_update_frequency.py@359
PS1, Line 359: blocking_query_options["sync_ddl"] = "false"
This is already set in line 351
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 1
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 27 Apr 2026 13:29:50 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/24243/1/tests/custom_cluster/test_topic_update_frequency.py
File tests/custom_cluster/test_topic_update_frequency.py:
http://gerrit.cloudera.org:8080/#/c/24243/1/tests/custom_cluster/test_topic_update_frequency.py@348
PS1, Line 348: ute("create table {0} (id i
> just a side note:
Ack
http://gerrit.cloudera.org:8080/#/c/24243/1/tests/custom_cluster/test_topic_update_frequency.py@354
PS1, Line 354: operations on the I
> I would prefer to create new table in unique_database to avoid mutating sha
Ack
http://gerrit.cloudera.org:8080/#/c/24243/1/tests/custom_cluster/test_topic_update_frequency.py@359
PS1, Line 359: "debug_action": debug_action,
> This is already set in line 351
Ack
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 2
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 27 Apr 2026 14:39:55 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 11:
> Patch Set 10: Code-Review+1
>
> > Also deflaked 'test_topic_updates_advance'.
>
> To make sure the test is not flaky, it'd be nice to iterate this test in a loop for multiple times (e.g. 20) in your local env. You can also mention this in the commit message, e.g., "ran test_topic_updates_advance 20 times locally"
>
> The current patch set LGTM. We can bump to +2 if no more comments from other reviewers.
Done.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 11
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 May 2026 08:51:04 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 12: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 12
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 11 May 2026 06:57:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 10: Code-Review+1
> Also deflaked 'test_topic_updates_advance'.
To make sure the test is not flaky, it'd be nice to iterate this test in a loop for multiple times (e.g. 20) in your local env. You can also mention this in the commit message, e.g., "ran test_topic_updates_advance 20 times locally"
The current patch set LGTM. We can bump to +2 if no more comments from other reviewers.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 10
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 07 May 2026 14:55:05 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 10:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/22337/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 10
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 06 May 2026 14:10:29 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24243/2/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java:
http://gerrit.cloudera.org:8080/#/c/24243/2/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@1801
PS2, Line 1801: if the table is one of {@link HdfsT
> Doesn't this comment need to be updated?
Ack
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 3
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 28 Apr 2026 08:15:44 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: [WIP] Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: [WIP] Catalog topic update creation can't skip Iceberg tables
......................................................................
IMPALA-14801: [WIP] Catalog topic update creation can't skip Iceberg tables
This patch extends the skipping mechanism from IMPALA-6671 for Iceberg
tables. It uses the Iceberg table's own tableLock, and the
lastVersionSeenByTopicUpdate mechanism of the underlying HDFS table.
Testing:
Added TestTopicUpdateFrequency::test_topic_updates_*_iceberg in 4
variants
Fixed the original HDFS tests:
1. test_topic_updates_unblock
- By default it was running in Local Catalog mode, which has no effect
for fast DML/DQL queries. Added variants for both Legacy and Local
Catalog mode to demonstrate the case.
- In the blocking scenario there was a missing assert (!), making the
test always pass.
- Fast queries are much faster than stated, which doesn't seem to
matter however for the nature of the test.
- Reduced query delay times
2. test_topic_updates_advance
- The test claimed to test catalog_max_lock_skipped_topic_updates but
experimentally I could see no counter blockings triggered at all
under any configuration. Added more parallel threads to reliably
trigger the blockings.
- Query execution time expectations largely incorrect. Perhaps for
some reason it changed over time.
- Assert expects the max query time to be no more than a predictable
value. Which conceptually makes sense for SYNC_DDL, but I wasn't
able to reliably reproduce the case yet.
- Hence for now at least checking the blockings occurring in catalog
logs.
- Reduced query delay times
3. Removed test_topic_lock_timeout_disabled, it is now covered by one of
the test_topic_updates_unblock variants
Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
---
M be/src/catalog/catalog-server.cc
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M tests/custom_cluster/test_topic_update_frequency.py
4 files changed, 337 insertions(+), 104 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/24243/5
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 5
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
This patch extends the skipping mechanism from IMPALA-6671 for Iceberg
tables. It uses the Iceberg table's own tableLock, and the
lastVersionSeenByTopicUpdate mechanism of the underlying HDFS table.
Testing:
Added TestTopicUpdateFrequency::test_topic_updates_*_iceberg in 4
variants
Fixed the original HDFS tests:
1. test_topic_updates_unblock
- By default it was running in Local Catalog mode, which has no effect
for fast DML/DQL queries. Added variants for both Legacy and Local
Catalog mode to demonstrate the case.
- In the blocking scenario there was a missing assert (!), making the
test always pass.
- Fast queries are much faster than stated, which doesn't seem to
matter however for the nature of the test.
- Reduced query delay times
2. test_topic_updates_advance
- The test claimed to test catalog_max_lock_skipped_topic_updates but
experimentally I could see no counter blockings triggered at all
under any configuration. Added more parallel threads to reliably
trigger the blockings.
- Query execution time expectations largely incorrect. Perhaps for
some reason it changed over time.
- Assert expects the max query time to be no more than a predictable
value. Which conceptually makes sense for SYNC_DDL, but I wasn't
able to reliably reproduce the case yet.
- Hence for now at least checking the blockings occurring in catalog
logs.
- Reduced query delay times
3. Removed test_topic_lock_timeout_disabled, it is now covered by one of
the test_topic_updates_unblock variants
Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
---
M be/src/catalog/catalog-server.cc
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
M tests/custom_cluster/test_topic_update_frequency.py
5 files changed, 315 insertions(+), 109 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/24243/9
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 9
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-14801: [WIP] Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: [WIP] Catalog topic update creation can't skip Iceberg tables
......................................................................
IMPALA-14801: [WIP] Catalog topic update creation can't skip Iceberg tables
This patch extends the skipping mechanism from IMPALA-6671 for Iceberg
tables. It uses the Iceberg table's own tableLock, and the
lastVersionSeenByTopicUpdate mechanism of the underlying HDFS table.
Testing:
Added TestTopicUpdateFrequency::test_topic_updates_*_iceberg in 4
variants
Fixed the original HDFS tests:
1. test_topic_updates_unblock
- By default it was running in Local Catalog mode, which has no effect
for fast DML/DQL queries. Added variants for both Legacy and Local
Catalog mode to demonstrate the case.
- In the blocking scenario there was a missing assert (!), making the
test always pass.
- Fast queries are much faster than stated, which doesn't seem to
matter however for the nature of the test.
- Reduced query delay times
2. test_topic_updates_advance
- The test claimed to test catalog_max_lock_skipped_topic_updates but
experimentally I could see no counter blockings triggered at all
under any configuration. Added more parallel threads to reliably
trigger the blockings.
- Query execution time expectations largely incorrect. Perhaps for
some reason it changed over time.
- Assert expects the max query time to be no more than a predictable
value. Which conceptually makes sense for SYNC_DDL, but I wasn't
able to reliably reproduce the case yet.
- Hence for now at least checking the blockings occurring in catalog
logs.
- Reduced query delay times
3. Removed test_topic_lock_timeout_disabled, it is now covered by one of
the test_topic_updates_unblock variants
Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
---
M be/src/catalog/catalog-server.cc
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
M tests/custom_cluster/test_topic_update_frequency.py
5 files changed, 342 insertions(+), 105 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/24243/6
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 6
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
This patch extends the skipping mechanism from IMPALA-6671 for Iceberg
tables. It uses the Iceberg table's own tableLock, and the
lastVersionSeenByTopicUpdate mechanism of the underlying HDFS table.
Testing:
Added TestTopicUpdateFrequency::test_topic_updates_*_iceberg in 4
variants
Fixed the original HDFS tests:
1. test_topic_updates_unblock
- By default it was running in Local Catalog mode, which has no effect
for fast DML/DQL queries. Added variants for both Legacy and Local
Catalog mode to demonstrate the case.
- In the blocking scenario there was a missing assert (!), making the
test always pass.
- Fast queries are much faster than stated, which doesn't seem to
matter however for the nature of the test.
- Reduced query delay times
2. test_topic_updates_advance
- The test claimed to test catalog_max_lock_skipped_topic_updates but
experimentally I could see no counter blockings triggered at all
under any configuration. Added more parallel threads to reliably
trigger the blockings.
- Query execution time expectations largely incorrect. Perhaps for
some reason it changed over time.
- Assert expects the max query time to be no more than a predictable
value. Which conceptually makes sense for SYNC_DDL, but I wasn't
able to reliably reproduce the case yet.
- Hence for now at least checking the blockings occurring in catalog
logs.
- Reduced query delay times
3. Removed test_topic_lock_timeout_disabled, it is now covered by one of
the test_topic_updates_unblock variants
Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
---
M be/src/catalog/catalog-server.cc
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
M tests/custom_cluster/test_topic_update_frequency.py
5 files changed, 342 insertions(+), 105 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/24243/7
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 7
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 7:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/22326/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 7
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 May 2026 11:09:50 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
This patch extends the skipping mechanism from IMPALA-6671 for Iceberg
tables. It uses the Iceberg table's own tableLock, and the
lastVersionSeenByTopicUpdate mechanism of the underlying HDFS table.
Testing:
Added TestTopicUpdateFrequency::test_topic_updates_*_iceberg in 4
variants
Fixed the original HDFS tests:
1. test_topic_updates_unblock
- By default it was running in Local Catalog mode, which has no effect
for fast DML/DQL queries. Added variants for both Legacy and Local
Catalog mode to demonstrate the case.
- In the blocking scenario there was a missing assert (!), making the
test always pass.
- Fast queries are much faster than stated, which doesn't seem to
matter however for the nature of the test.
- Reduced query delay times
2. test_topic_updates_advance
- The test claimed to test catalog_max_lock_skipped_topic_updates but
experimentally I could see no counter blockings triggered at all
under any configuration. Added more parallel threads to reliably
trigger the blockings.
- Query execution time expectations largely incorrect. Perhaps for
some reason it changed over time.
- Assert expects the max query time to be no more than a predictable
value. Which conceptually makes sense for SYNC_DDL, but I wasn't
able to reliably reproduce the case yet.
- Hence for now at least checking the blockings occurring in catalog
logs.
- Reduced query delay times
3. Removed test_topic_lock_timeout_disabled, it is now covered by one of
the test_topic_updates_unblock variants
Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
---
M be/src/catalog/catalog-server.cc
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M tests/custom_cluster/test_topic_update_frequency.py
4 files changed, 337 insertions(+), 104 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/24243/4
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 4
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 12: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/13443/
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 12
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 11 May 2026 11:21:41 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has removed a vote on this change.
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Removed Verified-1 by Impala Public Jenkins <im...@cloudera.com>
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 12
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-14801: [WIP] Catalog topic update creation can't skip Iceberg tables
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: [WIP] Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/22288/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 5
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Apr 2026 21:53:35 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 3:
(6 comments)
> Patch Set 3:
>
> (6 comments)
>
> Thanks for updating the old tests! I also take this chance to review them again.
> The code changes LGTM. Just left some comments about tests.
Thanks for the details. I've fixed the suggestions.
Also, reduced query delay times to make the tests faster overall.
Currently I'm still working on:
1. Blocking query SYNC_DDL for Iceberg with skipping hanging
2. test_topic_updates_advance caught as flaky in current state
3. refactor to reduce duplication
http://gerrit.cloudera.org:8080/#/c/24243/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/24243/3//COMMIT_MSG@24
PS3, Line 24: - Fast queries are much faster than stated, which doesn't seem to
> They take >2s in the legacy catalog mode since the default statestore_updat
Got it
http://gerrit.cloudera.org:8080/#/c/24243/3/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java:
http://gerrit.cloudera.org:8080/#/c/24243/3/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@1801
PS3, Line 1801: Link
> nit: "L" should be lowercase -> "link"
Ack
http://gerrit.cloudera.org:8080/#/c/24243/3/tests/custom_cluster/test_topic_update_frequency.py
File tests/custom_cluster/test_topic_update_frequency.py:
http://gerrit.cloudera.org:8080/#/c/24243/3/tests/custom_cluster/test_topic_update_frequency.py@57
PS3, Line 57: via RPC and don't depend on topic updates."""
> DDL/DMLs with sync_ddl=true still depend on topic updates to make sure othe
Added separate pair of tests for DDL (create table), which demonstrates that with SYNC_DDL those block regardless of the Catalog mode.
http://gerrit.cloudera.org:8080/#/c/24243/3/tests/custom_cluster/test_topic_update_frequency.py@112
PS3, Line 112: # blocking query is without sync_ddl
: blocking_query_options["sync_ddl"] = "false"
: self.__run_topic_update_test(blocking_query,
: non_blocking_queries, init_queries, blocking_query_options=blocking_query_options,
: expect_topic_updates_to_block=expect_topic_updates_to_block)
: # blocking query is with sync_ddl
: blocking_query_options["sync_ddl"] = "true"
: self.__run_topic_update_test(blocking_query,
: non_blocking_queries, init_queries, blocking_query_options=blocking_query_options,
: expect_topic_updates_to_block=expect_topic_updates_to_block)
:
: non_blocking_query_options = {
: "sync_ddl": "true",
: }
: self.__run_topic_update_test(blocking_query,
: non_blocking_queries, init_queries, blocking_query_options=blocking_query_options,
: non_blocking_query_options=non_blocking_query_options,
: expect_topic_updates_to_block=expect_topic_updates_to_block)
: blocking_query_options["sync_ddl"] = "false"
: self.__run_topic_update_test(blocking_query,
: non_blocking_queries, init_queries, blocking_query_options=blocking_query_options,
: non_blocking_query_options=non_blocking_query_options,
: expect_topic_updates_to_block=expect_topic_updates_to_block)
> nit: these can be simplified
Yeah, much better
http://gerrit.cloudera.org:8080/#/c/24243/3/tests/custom_cluster/test_topic_update_frequency.py@153
PS3, Line 153: len(slow_blocking_query)
> This seems wrong. 'slow_blocking_query' is a string, not a list. We can sim
Ack
http://gerrit.cloudera.org:8080/#/c/24243/3/tests/custom_cluster/test_topic_update_frequency.py@360
PS3, Line 360: "sync_ddl": "false"
> sync_ddl=true in blocking queries
Good that you brought this up: this causes iceberg test to hang. Investigating...
> DDLs/DMLs (if we add) in non_blocking_queries
Done, similarly as for non-Iceberg
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 3
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Apr 2026 21:25:28 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 8:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
File fe/src/main/java/org/apache/impala/catalog/IcebergTable.java:
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java@331
PS7, Line 331: // HdfsTable and get back the effective version in a single synchronized block
: long effectiveVersion = hdfsTable_.setCatalogVersionAndGet(newVersion);
: super.setCatalogVersion(effectiveVersion);
> Good point. What about this?
I am not sure if there is an actual issue, but it seems confusing.
Imagine the following scenario:
1. setCatalogVersion starts and finishes line 332 in another thread (note that only 1 thread can call due to table lock)
2. catalog topic update threads get version in
addTableToCatalogDelta - this is the IcebergTable's and different than hdfsTable_'s at this point
Is this a problem? My guess is that it is not worse than if the whole setCatalogVersion happens after getting version at 2., but it is pretty confusing, so my preference would be to keep the two versions synced using some lock.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 8
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 May 2026 14:16:36 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 7:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
File fe/src/main/java/org/apache/impala/catalog/IcebergTable.java:
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java@331
PS7, Line 331: // be propagated to Iceberg tables as well
: hdfsTable_.setCatalogVersion(newVersion);
: super.setCatalogVersion(hdfsTable_.getCatalogVersion());
Isn't it a problem that this can go out of sync in case of parallel calls? HDFS table's setCatalogVersion has a synchronized block, but there is no guarantee that parallel threads call 332 in the same order as 333.
A solution could be to expose pendingVersionLock_ in HdfsTable and use it in a synchronized block.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 7
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 May 2026 11:20:07 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 9:
(1 comment)
gerrit-auto-critic failed. You can reproduce it locally using command:
python3 bin/jenkins/critique-gerrit-review.py --dryrun
To run it, you might need a virtual env with Python3's venv installed.
http://gerrit.cloudera.org:8080/#/c/24243/9/tests/custom_cluster/test_topic_update_frequency.py
File tests/custom_cluster/test_topic_update_frequency.py:
http://gerrit.cloudera.org:8080/#/c/24243/9/tests/custom_cluster/test_topic_update_frequency.py@450
PS9, Line 450:
flake8: W391 blank line at end of file
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 9
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 06 May 2026 13:02:45 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 11: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 11
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 May 2026 10:32:21 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
This patch extends the skipping mechanism from IMPALA-6671 for Iceberg
tables. It uses the Iceberg table's own tableLock, and the
lastVersionSeenByTopicUpdate mechanism of the underlying HDFS table.
Testing:
Added TestTopicUpdateFrequency::test_topic_updates_*_iceberg in 4
variants
Fixed the original HDFS tests:
1. test_topic_updates_unblock
- By default it was running in Local Catalog mode, which has no effect
for fast DML/DQL queries. Added variants for both Legacy and Local
Catalog mode to demonstrate the case.
- In the blocking scenario there was a missing assert (!), making the
test always pass.
- Fast queries are much faster than stated, which doesn't seem to
matter however for the nature of the test.
- Reduced query delay times
2. test_topic_updates_advance
- The test claimed to test catalog_max_lock_skipped_topic_updates but
experimentally I could see no counter blockings triggered at all
under any configuration.
- Added more parallel threads and reduced
catalog_max_lock_skipped_topic_updates to 2 to reliably trigger the
blockings. Ran 20 times locally to verify.
- Query execution time expectations largely incorrect. Perhaps for
some reason it changed over time.
- Assert expects the max query time to be no more than a predictable
value. Which conceptually makes sense for SYNC_DDL, but I wasn't
able to reliably reproduce the case yet.
- Hence for now at least checking the blockings occurring in catalog
logs.
- Reduced query delay times
3. Removed test_topic_lock_timeout_disabled, it is now covered by one of
the test_topic_updates_unblock variants
Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Reviewed-on: http://gerrit.cloudera.org:8080/24243
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Csaba Ringhofer <cs...@cloudera.com>
---
M be/src/catalog/catalog-server.cc
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
M tests/custom_cluster/test_topic_update_frequency.py
5 files changed, 314 insertions(+), 109 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved
Csaba Ringhofer: Verified
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 13
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Mihaly Szjatinya (Code Review)" <ge...@cloudera.org>.
Mihaly Szjatinya has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 7:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
File fe/src/main/java/org/apache/impala/catalog/IcebergTable.java:
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java@331
PS7, Line 331: // be propagated to Iceberg tables as well
: hdfsTable_.setCatalogVersion(newVersion);
: super.setCatalogVersion(hdfsTable_.getCatalogVersion());
> >And the 'pendingVersionLock_' is specifically for pendingVersion, not the
I see. If only for that matter, let's do it separately. Since it was unsynced before, we only changed the order here.
Maybe in a broader context of better syncing Iceberg table and underlying Hdfs table in general. I suspect this design backfires elsewhere too.
The 'setCatalogVersionAndGet' still have some sense though.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 7
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 May 2026 20:05:48 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 8:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
File fe/src/main/java/org/apache/impala/catalog/IcebergTable.java:
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java@331
PS7, Line 331: // HdfsTable and get back the effective version in a single synchronized block
: long effectiveVersion = hdfsTable_.setCatalogVersionAndGet(newVersion);
: super.setCatalogVersion(effectiveVersion);
> I see. If only for that matter, let's do it separately. Since it was unsync
This looks correct. But is there a plan to totally get rid of HdfsTable in IcebergTable so we can make these simpler? Maybe in IMPALA-13738?
http://gerrit.cloudera.org:8080/#/c/24243/8/tests/custom_cluster/test_topic_update_frequency.py
File tests/custom_cluster/test_topic_update_frequency.py:
http://gerrit.cloudera.org:8080/#/c/24243/8/tests/custom_cluster/test_topic_update_frequency.py@148
PS8, Line 148: unblock
nit: shouldn't we use "__run_topic_update_block_sync_ddl_create_table" since topic update actually blocks the CreateTable with sync_ddl=true?
http://gerrit.cloudera.org:8080/#/c/24243/8/tests/custom_cluster/test_topic_update_frequency.py@149
PS8, Line 149: expect_topic_updates_to_block
nit: it seems we don't need this parameter since the callers all have this set to True.
http://gerrit.cloudera.org:8080/#/c/24243/8/tests/custom_cluster/test_topic_update_frequency.py@451
PS8, Line 451: def __run_topic_update_unblock_test_sync_ddl_create_table_iceberg(self,
nit: this method is quite similar to __run_topic_update_unblock_test_sync_ddl_create_table. Just the create/drop statements are different. Maybe we can extract the common code into a common method.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 8
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 06 May 2026 08:14:15 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 9:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/22336/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 9
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 06 May 2026 13:12:28 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 10: Code-Review+1
(2 comments)
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
File fe/src/main/java/org/apache/impala/catalog/IcebergTable.java:
http://gerrit.cloudera.org:8080/#/c/24243/7/fe/src/main/java/org/apache/impala/catalog/IcebergTable.java@331
PS7, Line 331: // HdfsTable and get back the effective version in a single synchronized block
: long effectiveVersion = hdfsTable_.setCatalogVersionAndGet(newVersion);
: super.setCatalogVersion(effectiveVersion);
> Yeah, that does seem relevant.
AFAIK no one is working on IMPALA-13738. I hope that non-local catalog mode can be removed at one point and then catalog's class hierarchy can be simplified/improved.
http://gerrit.cloudera.org:8080/#/c/24243/10/tests/custom_cluster/test_topic_update_frequency.py
File tests/custom_cluster/test_topic_update_frequency.py:
http://gerrit.cloudera.org:8080/#/c/24243/10/tests/custom_cluster/test_topic_update_frequency.py@281
PS10, Line 281: self.assert_catalogd_log_contains(
: "INFO",
: r"Topic update thread blocking until lock is acquired for table",
Running the test twice without restart would make this pass automatically, so test_topic_updates_advance should have force_restart=True
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 10
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 07 May 2026 17:44:24 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 6:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/22325/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 6
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 May 2026 11:08:05 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 1:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/22247/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 1
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 27 Apr 2026 09:49:25 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/22249/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 2
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 27 Apr 2026 15:04:52 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24243 )
Change subject: IMPALA-14801: Catalog topic update creation can't skip Iceberg tables
......................................................................
Patch Set 12:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/13443/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/24243
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I51e46820aaa096f3eb69f4dcf580e49a69d6603d
Gerrit-Change-Number: 24243
Gerrit-PatchSet: 12
Gerrit-Owner: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Balazs Hevele <bh...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Mihaly Szjatinya <ms...@pm.me>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 11 May 2026 06:57:52 +0000
Gerrit-HasComments: No