[jira] [Assigned] (IMPALA-9074) Add support for zstd in ORC

["Quanlong Huang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-9074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Quanlong Huang reassigned IMPALA-9074:
--------------------------------------

    Assignee: Quanlong Huang

> Add support for zstd in ORC
> ---------------------------
>
>                 Key: IMPALA-9074
>                 URL: https://issues.apache.org/jira/browse/IMPALA-9074
>             Project: IMPALA
>          Issue Type: New Feature
>            Reporter: Quanlong Huang
>            Assignee: Quanlong Huang
>            Priority: Major
>         Attachments: id_name_zstd.orc
>
>
> The ORC lib already supports reading/writing to zstd compressed ORC files. However, I failed in a quick try in Impala:
> {code:sql}
> hive> create table orc_zstd (id int, name string) stored as orc;
> $ hdfs dfs -put id_name_zstd.orc hdfs://localhost:20500/test-warehouse/orc_zstd
> impala-shell> invalidate metadata orc_zstd;
> impala-shell> select * from orc_zstd;
> ERROR: Encountered parse error in tail of ORC file hdfs://localhost:20500/test-warehouse/orc_zstd/id_name_zstd.orc: Unknown compression codec 5
> {code}
> The ORC file is generated by the csv-import tool: https://github.com/apache/orc/blob/rel/release-1.6.0/tools/src/CSVFileImport.cc
> (Manually changing the compression from ZLIB to ZSTD in it)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org