You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by "Wes McKinney (JIRA)" <ji...@apache.org> on 2016/09/21 20:29:20 UTC
[jira] [Created] (ARROW-300) [Format] Add buffer compression option
to IPC file format
Wes McKinney created ARROW-300:
----------------------------------
Summary: [Format] Add buffer compression option to IPC file format
Key: ARROW-300
URL: https://issues.apache.org/jira/browse/ARROW-300
Project: Apache Arrow
Issue Type: New Feature
Components: Format
Reporter: Wes McKinney
It may be useful if data is to be sent over the wire to compress the data buffers themselves as their being written in the file layout.
I would propose that we keep this extremely simple with a global buffer compression setting in the file Footer. Probably only two compressors worth supporting out of the box would be zlib (higher compression ratios) and lz4 (better performance).
What does everyone think?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)