You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Csaba Ringhofer (Jira)" <ji...@apache.org> on 2026/07/08 17:41:00 UTC
[jira] [Updated] (IMPALA-14593) Iceberg V3 Geometry support
[ https://issues.apache.org/jira/browse/IMPALA-14593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Csaba Ringhofer updated IMPALA-14593:
-------------------------------------
Description:
Add support for Iceberg/Parquet GEOMETRY type. This are WKB byte arrays, so this change involves switching to WKB as internal serialization format from the current one used by Hive ESRI functions.
The old mode is not removed at this point, but allow switching with flag geospatail_libary (current: ESRI_HIVE new: WKB_EXPERIMENTAL).
The main goal of keeping the old mode is to be able to help testing by checking if both modes return the same result.
The plan is to do this in several steps:
1. allow Java functions to use WKB and switch to this with WKB_EXPERIMENTAL (only java, even where c++ implementation exists for ESRI_HIVE mode)
2. and GEOMETRY type that is used in WKB_EXPERIMENTAL mode, and switch Java functions to use it instead of BINARY
3. add read/write support for Iceberg/Parquet GEOMETRY columns (no bounding rect statistics yet)
4. add basic c++ support for WKB parsing
5. add bounding rect support for Parquet (read/write) and Iceberg (only write due to gap in Icebeg lib)
6. implement critical functions in c++ - until this step the performance in worse in WKB_EXPERIMENTAL mode due to having 0 c++ functions and the extra conversion in Java)
The c++ changes are based on https://gerrit.cloudera.org/#/c/20602/ , but use WKB.
was:Add support for Iceberg geospatial types and functions.
> Iceberg V3 Geometry support
> ---------------------------
>
> Key: IMPALA-14593
> URL: https://issues.apache.org/jira/browse/IMPALA-14593
> Project: IMPALA
> Issue Type: New Feature
> Components: Backend, Frontend
> Reporter: Zoltán Borók-Nagy
> Assignee: Csaba Ringhofer
> Priority: Major
> Labels: impala-iceberg
>
> Add support for Iceberg/Parquet GEOMETRY type. This are WKB byte arrays, so this change involves switching to WKB as internal serialization format from the current one used by Hive ESRI functions.
> The old mode is not removed at this point, but allow switching with flag geospatail_libary (current: ESRI_HIVE new: WKB_EXPERIMENTAL).
> The main goal of keeping the old mode is to be able to help testing by checking if both modes return the same result.
> The plan is to do this in several steps:
> 1. allow Java functions to use WKB and switch to this with WKB_EXPERIMENTAL (only java, even where c++ implementation exists for ESRI_HIVE mode)
> 2. and GEOMETRY type that is used in WKB_EXPERIMENTAL mode, and switch Java functions to use it instead of BINARY
> 3. add read/write support for Iceberg/Parquet GEOMETRY columns (no bounding rect statistics yet)
> 4. add basic c++ support for WKB parsing
> 5. add bounding rect support for Parquet (read/write) and Iceberg (only write due to gap in Icebeg lib)
> 6. implement critical functions in c++ - until this step the performance in worse in WKB_EXPERIMENTAL mode due to having 0 c++ functions and the extra conversion in Java)
> The c++ changes are based on https://gerrit.cloudera.org/#/c/20602/ , but use WKB.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org