You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Noemi Pap-Takacs (Jira)" <ji...@apache.org> on 2023/01/17 13:11:00 UTC
[jira] [Commented] (IMPALA-11839) impala fails to read iceberg table stored as avro which has decimal(15,2) datatype created by hive
[ https://issues.apache.org/jira/browse/IMPALA-11839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17677754#comment-17677754 ]
Noemi Pap-Takacs commented on IMPALA-11839:
-------------------------------------------
{{'Decimal' }}is a _logical_ type in AVRO. It can be represented in AVRO '{{{}bytes'{}}} or '{{{}fixed'{}}} types.
According to the Iceberg specification, the implementations and the Iceberg library can read both, but write '{{{}decimal' {}}}with '{{{}fixed'{}}} type, and so does Hive, when we create and insert into an Iceberg table. However, Impala can only read '{{{}bytes{}}}'. {{'Fixed'}} type is currently not supported, that is why the error message says:
{code:java}
Avro enum, array, map, union, and fixed types are not supported. Got type: fixed{code}
> impala fails to read iceberg table stored as avro which has decimal(15,2) datatype created by hive
> --------------------------------------------------------------------------------------------------
>
> Key: IMPALA-11839
> URL: https://issues.apache.org/jira/browse/IMPALA-11839
> Project: IMPALA
> Issue Type: Bug
> Reporter: Riddhi jain
> Assignee: Noemi Pap-Takacs
> Priority: Major
> Labels: iceberg
>
> Create table using Hive
> CREATE EXTERNAL TABLE decimal_table_txt (decimal_col decimal(15,2)) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' stored as TEXTFILE LOCATION 's3a://dwx-testdata/tmp/tpch/data/customer';CREATE TABLE decimal_table STORED BY iceberg STORED AS avro AS SELECT * FROM decimal_table_txt;
> When trying to read iceberg table (stored as avro) using impala with decimal(15,2) as datatype it fails to return the rows.
> {code:java}
> select * from decimal_table;{code}
> Output:
> {code:java}
> Query 88420c9d3ee0e2f1:4ef29b2a00000000 100% Complete (2 out of 2)
> Avro enum, array, map, union, and fixed types are not supported. Got type: fixed
> Avro enum, array, map, union, and fixed types are not supported. Got type: fixed
> Query 88420c9d3ee0e2f1:4ef29b2a00000000 100% Complete (2 out of 2)
> Avro enum, array, map, union, and fixed types are not supported. Got type: fixed
> Avro enum, array, map, union, and fixed types are not supported. Got type: fixed
> Query 88420c9d3ee0e2f1:4ef29b2a00000000 100% Complete (2 out of 2)
> Avro enum, array, map, union, and fixed types are not supported. Got type: fixed
> Avro enum, array, map, union, and fixed types are not supported. Got type: fixed {code}
> {code:java}
> Done. 0 results. 88420c9d3ee0e2f1:4ef29b2a00000000{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org