You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/01/10 00:08:00 UTC
[jira] [Commented] (IMPALA-10610) Support multiple file formats in a single Iceberg Table
[ https://issues.apache.org/jira/browse/IMPALA-10610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656345#comment-17656345 ]
ASF subversion and git services commented on IMPALA-10610:
----------------------------------------------------------
Commit 0549d9562bce876d93d9bfe188e828fa29c056f6 in impala's branch refs/heads/master from noemi
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=0549d9562 ]
IMPALA-11620: Enable setting 'write.format.default'
Enable setting 'write.format.default' to a different file format
than what the table already contains.
Before IMPALA-10610 Iceberg tables with mixed-format data files were not
supported.
We used 'write.format.default' to determine the file format of the
table, which was only a temporary workaround. Because of this we did not
allow changing this table property if the table already contained
different table formats. E.g. we did not allow modifying
'write.format.default' to PARQUET if the table already contained ORC
files, because it would have made the table unreadable for Impala.
Since IMPALA-10610 'write.format.default' is not used to determine the
Iceberg table's format anymore, so we can allow changing it.
This table property change is not synchronized between HMS and Iceberg
metadata files in case of true external Hive Catalog tables.
See IMPALA-11710.
Testing:
- E2E test in iceberg-alter.test
Change-Id: I22d0a8a18fce99015fcfe1fd15cb4a4d4c2deaec
Reviewed-on: http://gerrit.cloudera.org:8080/19221
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Support multiple file formats in a single Iceberg Table
> -------------------------------------------------------
>
> Key: IMPALA-10610
> URL: https://issues.apache.org/jira/browse/IMPALA-10610
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend, Frontend
> Reporter: Zoltán Borók-Nagy
> Assignee: Gergely Fürnstáhl
> Priority: Major
> Labels: impala-iceberg
> Fix For: Impala 4.2.0
>
>
> Iceberg allows having different file formats in a single table. It stores the file format information for each data file.
> Impala only allows a single file format per partition. Iceberg tables are handled as non-partitioned HMS tables (Iceberg partitioning is more or less hidden from Impala). Therefore currently Impala don't allow having different file formats in a single table.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org