You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/01/10 00:08:00 UTC
[jira] [Commented] (IMPALA-11620) Enable setting 'write.format.default'
[ https://issues.apache.org/jira/browse/IMPALA-11620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656344#comment-17656344 ]
ASF subversion and git services commented on IMPALA-11620:
----------------------------------------------------------
Commit 0549d9562bce876d93d9bfe188e828fa29c056f6 in impala's branch refs/heads/master from noemi
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=0549d9562 ]
IMPALA-11620: Enable setting 'write.format.default'
Enable setting 'write.format.default' to a different file format
than what the table already contains.
Before IMPALA-10610 Iceberg tables with mixed-format data files were not
supported.
We used 'write.format.default' to determine the file format of the
table, which was only a temporary workaround. Because of this we did not
allow changing this table property if the table already contained
different table formats. E.g. we did not allow modifying
'write.format.default' to PARQUET if the table already contained ORC
files, because it would have made the table unreadable for Impala.
Since IMPALA-10610 'write.format.default' is not used to determine the
Iceberg table's format anymore, so we can allow changing it.
This table property change is not synchronized between HMS and Iceberg
metadata files in case of true external Hive Catalog tables.
See IMPALA-11710.
Testing:
- E2E test in iceberg-alter.test
Change-Id: I22d0a8a18fce99015fcfe1fd15cb4a4d4c2deaec
Reviewed-on: http://gerrit.cloudera.org:8080/19221
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Enable setting 'write.format.default'
> -------------------------------------
>
> Key: IMPALA-11620
> URL: https://issues.apache.org/jira/browse/IMPALA-11620
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Zoltán Borók-Nagy
> Assignee: Noemi Pap-Takacs
> Priority: Major
> Labels: impala-iceberg, ramp-up
>
> Before IMPALA-10610 we didn't support Iceberg tables with mixed-format data files.
> And we used 'write.format.default' to determine the file format of the table, which was only a temporary workaround. Because of this we didn't allow changing this table property if the table already contained different table formats. E.g. we didn't allow modifying 'write.format.default' to PARQUET if the table already contained ORC files, because it would have made the table unreadable for Impala.
> Since IMPALA-10610 'write.format.default' is not used to determine the Iceberg table's format anymore, so we can allow changing it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org