[jira] [Created] (IMPALA-9074) Add support for zstd in ORC

["Quanlong Huang (Jira)" <ji...@apache.org>]

Quanlong Huang created IMPALA-9074:
--------------------------------------

             Summary: Add support for zstd in ORC
                 Key: IMPALA-9074
                 URL: https://issues.apache.org/jira/browse/IMPALA-9074
             Project: IMPALA
          Issue Type: New Feature
            Reporter: Quanlong Huang


The ORC lib already supports reading/writing to zstd compressed ORC files. However, I failed in a quick try in Impala:

{code:sql}
hive> create table orc_zstd (id int, name string) stored as orc;
$ hdfs dfs -put id_name_zstd.orc hdfs://localhost:20500/test-warehouse/orc_zstd
impala-shell> invalidate metadata orc_zstd;
impala-shell> select * from orc_zstd;
ERROR: Encountered parse error in tail of ORC file hdfs://localhost:20500/test-warehouse/orc_zstd/id_name_zstd.orc: Unknown compression codec 5
{code}

The ORC file is generated by the csv-import tool: https://github.com/apache/orc/blob/rel/release-1.6.0/tools/src/CSVFileImport.cc
(Manually changing the compression from ZLIB to ZSTD in it)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)