You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Bikramjeet Vig (JIRA)" <ji...@apache.org> on 2019/06/05 18:31:00 UTC
[jira] [Assigned] (IMPALA-8549) Add support for scanning DEFLATE
text files
[ https://issues.apache.org/jira/browse/IMPALA-8549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bikramjeet Vig reassigned IMPALA-8549:
--------------------------------------
Assignee: Ethan
> Add support for scanning DEFLATE text files
> -------------------------------------------
>
> Key: IMPALA-8549
> URL: https://issues.apache.org/jira/browse/IMPALA-8549
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Sahil Takiar
> Assignee: Ethan
> Priority: Minor
> Labels: ramp-up
>
> Several Hadoop tools (e.g. Hive, MapReduce, etc.) support reading and writing text files stored using zlib / deflate (results in files such as {{000000_0.deflate}}). Impala currently does not support reading {{.deflate}} files and returns errors such as: {{ERROR: Scanner plugin 'DEFLATE' is not one of the enabled plugins: 'LZO'}}.
> Moreover, the default compression codec in Hadoop is zlib / deflate (see {{o.a.h.io.compress.DefaultCodec}}). So when writing to a text table in Hive, if users set {{hive.exec.compress.output}} to true, then {{.deflate}} files will be written by default.
> Impala does support zlib / deflate with other file formats though: Avro, RCFiles, SequenceFiles (see https://impala.apache.org/docs/build/html/topics/impala_file_formats.html).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org