You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tim Armstrong (Jira)" <ji...@apache.org> on 2020/12/24 00:22:01 UTC
[jira] [Updated] (IMPALA-3073) Verify if compressed data in avro,
sequence file format could be multistream
[ https://issues.apache.org/jira/browse/IMPALA-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong updated IMPALA-3073:
----------------------------------
Priority: Minor (was: Major)
> Verify if compressed data in avro, sequence file format could be multistream
> ----------------------------------------------------------------------------
>
> Key: IMPALA-3073
> URL: https://issues.apache.org/jira/browse/IMPALA-3073
> Project: IMPALA
> Issue Type: New Feature
> Components: Backend
> Affects Versions: Impala 2.0
> Reporter: Juan Yu
> Priority: Minor
>
> When generating compressed text file, certain tools (like pbzip2) could parallel compression and create multistream compressed data. We need to verify if this applies to other file formats like parquet, avro, sequence that use those codec. If yes, Codec::ProcessBlock() should support multistream compressd block data. decompression should not stop when reach the end of compressed stream, e.g. Z_STREAM_END, BZ_STREAM_END.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org