[jira] [Resolved] (IMPALA-10948) Impala shouldn't require DECIMAL scale for Parquet files

["Gergely Fürnstáhl (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-10948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gergely Fürnstáhl resolved IMPALA-10948.
----------------------------------------
    Fix Version/s: Impala 4.1.0
       Resolution: Fixed

> Impala shouldn't require DECIMAL scale for Parquet files
> --------------------------------------------------------
>
>                 Key: IMPALA-10948
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10948
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Backend
>            Reporter: Zoltán Borók-Nagy
>            Assignee: Gergely Fürnstáhl
>            Priority: Major
>              Labels: ramp-up
>             Fix For: Impala 4.1.0
>
>
> Impala requires the 'scale' to be set for decimal columns: https://github.com/apache/impala/blob/1a61a8025c87c37921a1bba4c49f754d8bd10bcc/be/src/exec/parquet/parquet-metadata-utils.cc#L332
> But it is only an optional field in Parquet's [SchemaElement|https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L392] and the [docs|https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#decimal] says that if the scale is unspecified then it should be considered to be 0.
> Then there's the new logical type annotation [DecimalType|https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L253], but Impala doesn't use it during scans.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)