You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@parquet.apache.org by "Antoine Pitrou (Jira)" <ji...@apache.org> on 2021/04/19 14:47:00 UTC

[jira] [Commented] (PARQUET-2032) Deprecate LZ4, introduce new LZ4_RAW

    [ https://issues.apache.org/jira/browse/PARQUET-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325095#comment-17325095 ] 

Antoine Pitrou commented on PARQUET-2032:
-----------------------------------------

cc [~emkornfield]

> Deprecate LZ4, introduce new LZ4_RAW
> ------------------------------------
>
>                 Key: PARQUET-2032
>                 URL: https://issues.apache.org/jira/browse/PARQUET-2032
>             Project: Parquet
>          Issue Type: Improvement
>            Reporter: Gabor Szadovszky
>            Priority: Major
>
> The currently implemented LZ4 compression is based on the hadoop codec which is now deprecated (see details at PARQUET-1996). Also, a new, properly specified LZ4 compression (LZ4_RAW) has been introduced in the format.
> The idea is to use the new LZ4_RAW compression for all cases when we currently use LZ4 and introduce a new configuration where the user would be able to switch to the deprecated behavior in case of the selected codec is LZ4.
> For LZ4_RAW we will need to add the proper libs that will provide the officially specified LZ4 raw format.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)