You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by "Xinli Shang (Jira)" <ji...@apache.org> on 2020/06/12 03:42:00 UTC

[jira] [Created] (PARQUET-1872) Add TransCompression command

Xinli Shang created PARQUET-1872:
------------------------------------

             Summary: Add TransCompression command 
                 Key: PARQUET-1872
                 URL: https://issues.apache.org/jira/browse/PARQUET-1872
             Project: Parquet
          Issue Type: Improvement
          Components: parquet-mr
    Affects Versions: 1.12.0
            Reporter: Xinli Shang
            Assignee: Xinli Shang


When ZSTD becomes more popular, there is a need to translate existing data ZSTD compressed which can achieve a higher compression ratio. It would be useful if we can have a tool to convert a Parquet file directly by just decompressing/compressing each page without decoding/encoding or assembling the record because it is much faster. The initial result shows it is ~5 times faster. 





--
This message was sent by Atlassian Jira
(v8.3.4#803005)