You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@parquet.apache.org by "Xinli Shang (Jira)" <ji...@apache.org> on 2021/12/03 23:08:00 UTC

[jira] [Updated] (PARQUET-1872) Add TransCompression Feature

     [ https://issues.apache.org/jira/browse/PARQUET-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xinli Shang updated PARQUET-1872:
---------------------------------
    Summary: Add TransCompression Feature   (was: Add TransCompression command )

> Add TransCompression Feature 
> -----------------------------
>
>                 Key: PARQUET-1872
>                 URL: https://issues.apache.org/jira/browse/PARQUET-1872
>             Project: Parquet
>          Issue Type: Improvement
>          Components: parquet-mr
>    Affects Versions: 1.12.0
>            Reporter: Xinli Shang
>            Assignee: Xinli Shang
>            Priority: Major
>
> When ZSTD becomes more popular, there is a need to translate existing data to ZSTD compressed which can achieve a higher compression ratio. It would be useful if we can have a tool to convert a Parquet file directly by just decompressing/compressing each page without decoding/encoding or assembling the record because it is much faster. The initial result shows it is ~5 times faster. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)