You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/03/30 01:31:00 UTC
[jira] [Commented] (IMPALA-10597) Enable setting
'iceberg.file_format'
[ https://issues.apache.org/jira/browse/IMPALA-10597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17311063#comment-17311063 ]
ASF subversion and git services commented on IMPALA-10597:
----------------------------------------------------------
Commit dbc2fc14d86f84b25670ee9af7cedbbc3cd18b9f in impala's branch refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=dbc2fc1 ]
IMPALA-10597: Enable setting 'iceberg.file_format'
Currently we prohibit setting the following properties:
* iceberg.catalog
* iceberg.catalog_location
* iceberg.file_format
* iceberg.table_identifier
This patch enables setting 'iceberg.file_format', therefore if
a table was created by another engine, but using HiveCatalog,
we'll be able to set the data file format to the proper value
and make the table readable by Impala. Setting the other
properties are not needed for HiveCatalog tables.
If the table wasn't created by HiveCatalog, then we cannot load the
table, therefore we cannot invoke any ALTER TABLE statement at all.
In that case we need to create an external table.
If the table already contains data files, then Impala checks if
all of them have the proper file format. If not, the ALTER TABLE
statement fails.
Before this patch a CREATE TABLE statement accepted any string
for 'iceberg.file_format', and in case of invalid file formats the
frontend silently used Parquet. This patch also adds a check to only
allow valid file formats.
Testing:
* added e2e test
Change-Id: I4b3506be4562a1ace3e6435867aadb3bdde7a8e2
Reviewed-on: http://gerrit.cloudera.org:8080/17207
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Enable setting 'iceberg.file_format'
> ------------------------------------
>
> Key: IMPALA-10597
> URL: https://issues.apache.org/jira/browse/IMPALA-10597
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: impala-iceberg
>
> Currently we prohibit setting the following properties:
> * iceberg.catalog
> * iceberg.catalog_location
> * iceberg.file_format
> * icceberg.table_identifier
> Impala needs these properties to be able to correctly use the table. However, if the table was created by an other engine. e.g. by Hive, then the table won't have these properties.
> We need to allow setting at least 'iceberg.file_format'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org