You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org> on 2026/10/28 13:41:10 UTC
[Impala-ASF-CR] IMPALA-15638: Don't fail Iceberg scans on invalid UTF-8 bounds
Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/25195
Change subject: IMPALA-15638: Don't fail Iceberg scans on invalid UTF-8 bounds
......................................................................
IMPALA-15638: Don't fail Iceberg scans on invalid UTF-8 bounds
Parquet files written without the UTF8 annotation store strings as
plain BINARY. When such files are migrated to Iceberg, their bounds are
truncated to 16 bytes, possibly in the middle of a multi-byte
character. When a predicate on such a column was pushed down, Iceberg
failed to decode the bound, and the query failed with
"RuntimeIOException: Failed to decode value as UTF-8".
If planFiles() fails this way, IcebergScanPlanner now plans the scan
again without pushing down the predicates.
The Calcite planner decides to use the count star optimization before
that, assuming that the predicates are pushed down. A precondition
check makes such scans fail instead of returning a wrong count if the
predicates can't be pushed down.
Testing:
- Added an EE test with a migrated table.
Change-Id: I94cd94633d009d5c8d6130930dc8f86390e96a34
Assisted-by: Claude Opus 5.5 (Claude Code)
---
M fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java
M java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaHdfsScanRel.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-migrated-tables.test
3 files changed, 77 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/25195/1
--
To view, visit http://gerrit.cloudera.org:8080/25195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I94cd94633d009d5c8d6130930dc8f86390e96a34
Gerrit-Change-Number: 25195
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Arnab Karmakar <ar...@gmail.com>
[Impala-ASF-CR] IMPALA-15638: Don't fail Iceberg scans on invalid UTF-8 bounds
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/25195 )
Change subject: IMPALA-15638: Don't fail Iceberg scans on invalid UTF-8 bounds
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/24717/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/25195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I94cd94633d009d5c8d6130930dc8f86390e96a34
Gerrit-Change-Number: 25195
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Arnab Karmakar <ar...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 28 Oct 2026 14:08:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-15638: Don't fail Iceberg scans on invalid UTF-8 bounds
Posted by "Arnab Karmakar (Code Review)" <ge...@cloudera.org>.
Arnab Karmakar has posted comments on this change. ( http://gerrit.cloudera.org:8080/25195 )
Change subject: IMPALA-15638: Don't fail Iceberg scans on invalid UTF-8 bounds
......................................................................
Patch Set 1:
(3 comments)
Thanks for working on this!
http://gerrit.cloudera.org:8080/#/c/25195/1/fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java
File fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java:
http://gerrit.cloudera.org:8080/#/c/25195/1/fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java@242
PS1, Line 242: !(e.getCause() instanceof CharacterCodingException))
Is it okay to check only the direct cause as there could be CharacterCodingException nested deeper in the chain?
http://gerrit.cloudera.org:8080/#/c/25195/1/testdata/workloads/functional-query/queries/QueryTest/iceberg-migrated-tables.test
File testdata/workloads/functional-query/queries/QueryTest/iceberg-migrated-tables.test:
http://gerrit.cloudera.org:8080/#/c/25195/1/testdata/workloads/functional-query/queries/QueryTest/iceberg-migrated-tables.test@247
PS1, Line 247: ---- QUERY
I think we could cover some more cases:
1. Case that hits the Calcite precondition.
2. Tables with delete files which would exercise the state reset.
3. Case with impala.iceberg.push_down_hint.
http://gerrit.cloudera.org:8080/#/c/25195/1/testdata/workloads/functional-query/queries/QueryTest/iceberg-migrated-tables.test@252
PS1, Line 252: false
Should this be reset to true to prevent issues if any tests are appended later?
--
To view, visit http://gerrit.cloudera.org:8080/25195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I94cd94633d009d5c8d6130930dc8f86390e96a34
Gerrit-Change-Number: 25195
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Arnab Karmakar <ar...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 28 Oct 2026 18:15:55 +0000
Gerrit-HasComments: Yes