You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/09/08 22:12:00 UTC
[jira] [Commented] (IMPALA-10610) Support multiple file formats in a single Iceberg Table
[ https://issues.apache.org/jira/browse/IMPALA-10610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17602025#comment-17602025 ]
ASF subversion and git services commented on IMPALA-10610:
----------------------------------------------------------
Commit f598b2ad68f59c2deda3d8cdf0974d8e9a55111f in impala's branch refs/heads/master from Gergely Fürnstáhl
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=f598b2ad6 ]
IMPALA-10610: Support multiple file formats in a single Iceberg Table
Added support for multiple file formats. Previously Impala created a
Scanner class based on the partitions file format, now in case of an
Iceberg table it will read out the file format from the file level
metadata instead.
IcebergScanNode will aggregate file formats as well instead of relying
on partitions, so it can be used for plannig.
Testing:
Created a mixed file format table with hive and added a test for it.
Change-Id: Ifc816595724e8fd2c885c6664f790af61ddf5c07
Reviewed-on: http://gerrit.cloudera.org:8080/18935
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Support multiple file formats in a single Iceberg Table
> -------------------------------------------------------
>
> Key: IMPALA-10610
> URL: https://issues.apache.org/jira/browse/IMPALA-10610
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend, Frontend
> Reporter: Zoltán Borók-Nagy
> Assignee: Gergely Fürnstáhl
> Priority: Major
> Labels: impala-iceberg
>
> Iceberg allows having different file formats in a single table. It stores the file format information for each data file.
> Impala only allows a single file format per partition. Iceberg tables are handled as non-partitioned HMS tables (Iceberg partitioning is more or less hidden from Impala). Therefore currently Impala don't allow having different file formats in a single table.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org