You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2026/10/05 10:30:00 UTC
[jira] [Commented] (IMPALA-15365) Add UUID partition transforms support
[ https://issues.apache.org/jira/browse/IMPALA-15365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18123317#comment-18123317 ]
ASF subversion and git services commented on IMPALA-15365:
----------------------------------------------------------
Commit 2e8fb1d8b7ec3ecbfe8878a03b6dcf2af942e785 in impala's branch refs/heads/master from Arnab Karmakar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=2e8fb1d8b ]
IMPALA-15365: Iceberg UUID partition transform support
Adds partition transform support for the Iceberg UUID type. The Iceberg
spec only allows the identity, bucket(N, col) and void partition
transforms on UUID columns. Impala now rejects the other transforms
(year, month, day, hour and truncate) at analysis time for CREATE TABLE
and ALTER TABLE ... SET PARTITION SPEC, instead of failing later in
Iceberg.
The Iceberg Java library compares UUIDs with the signed
java.util.UUID.compareTo(), while Parquet writes UUID min/max bounds in
unsigned byte order. This is a bug in the Java library, not in the
Iceberg spec, which stores UUIDs as 16 big-endian bytes. Because of it,
Iceberg's metrics evaluation can skip data files that contain matching
rows. UUID value predicates are therefore not pushed to Iceberg and are
evaluated by the scan. IS NULL and IS NOT NULL are still pushed.
DROP PARTITION, SHOW FILES ... PARTITION and SHOW PARTITIONS ... WHERE
accept =, !=, IN and NOT IN on UUID columns that are identity
partitioned in every partition spec. Iceberg matches these values
exactly against the partition values, so byte order does not matter.
For = and IN, Iceberg also prunes manifests by their partition summary
bounds. This assumes the summaries were written by the Iceberg Java
library (Impala, Trino, Spark), which builds them with the same signed
comparison. Range comparisons on UUID columns are rejected in these
statements.
Testing:
- FE tests: identity, bucket and void are accepted on UUID columns and
the other transforms are rejected, for CREATE TABLE and SET PARTITION
SPEC.
- e2e tests on the new iceberg_uuid_test_part fixture for reads, scan
predicates, EXPLAIN, DROP PARTITION, SHOW FILES and SHOW PARTITIONS,
including =, !=, IN and NOT IN.
- Trino interop tests for partition evolution from identity to bucket
and for DROP PARTITION on Trino-written identity partitions.
- Added the iceberg_uuid_test_part fixture (Iceberg V3,
identity(uuid_identity), bucket(4, uuid_bucket)), generated with the
Iceberg Java API (version 1.11.0). One data file holds values on both
sides of the sign bit. All data files are added in a single append,
so one manifest's uuid_identity summary spans the sign bit.
Change-Id: I1dc3af4418285d6ea1d489912419be470e668f29
Assisted-by: Claude Opus 5.5 (Cursor)
Reviewed-on: http://gerrit.cloudera.org:8080/24973
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Add UUID partition transforms support
> -------------------------------------
>
> Key: IMPALA-15365
> URL: https://issues.apache.org/jira/browse/IMPALA-15365
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend, Frontend
> Reporter: Arnab Karmakar
> Assignee: Arnab Karmakar
> Priority: Major
>
> Iceberg UUID bucket partition read and pruning.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org