[jira] [Resolved] (IMPALA-2494) Impala Unable to scan a Decimal column stored as Bytes

["Bikramjeet Vig (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bikramjeet Vig resolved IMPALA-2494.
------------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.11.0

https://github.com/apache/incubator-impala/commit/94236ff2ff6e3d6d25a80150c98d4275914dc8c2


Extendes parquet column reader and associated classes to allow for more
than one possible physical type for a given logical type. This patch
only adds support for variable sized byte array encoded decimals and
more will be added in upcoming commits.
Also, column level metadata verification which was currently being
done per row group will now only be done once per column per file.

Testing:
Added backend test for verifying newly added decimal types are decoded
correctly.
Added Query test that decodes both plain and dictionary-encoded
decimals using binary encoding.

Performance:
Initial perf testing using tpcds_1000 shows no regression.

Change-Id: I2c0e881045109f337fecba53fec21f9cfb9e619e
Reviewed-on: http://gerrit.cloudera.org:8080/7822
Reviewed-by: Bikramjeet Vig <bi...@cloudera.com>
Tested-by: Impala Public Jenkins

> Impala Unable to scan a Decimal column stored as Bytes
> ------------------------------------------------------
>
>                 Key: IMPALA-2494
>                 URL: https://issues.apache.org/jira/browse/IMPALA-2494
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Backend
>    Affects Versions: Impala 2.3.0
>            Reporter: Taras Bobrovytsky
>            Assignee: Bikramjeet Vig
>             Fix For: Impala 2.11.0
>
>         Attachments: table20.parquet
>
>
> The CREATE TABLE LIKE PARQUET statement works on the file attached. However, executing {code}
> Select * from table20
> {code}
> causes the following error:
> {code}
> File 'hdfs://localhost:20500/taras/table20.parquet' has an incompatible Parquet schema for column 'taras_tmp.table20.col2'. Column type: DECIMAL(9, 2), Parquet schema:
> required byte_array col2 [i:1 d:0 r:0]
> {code}
> This is supported according to the Parquet spec: https://github.com/Parquet/parquet-format/blob/master/LogicalTypes.md#decimal



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)