[jira] [Assigned] (HADOOP-13578) Add Codec for ZStandard Compression

["Jason Lowe (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-13578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Lowe reassigned HADOOP-13578:
-----------------------------------

    Assignee: churro morales

Thanks for posting a patch!  Some comments after a quick look, I'll try to get a more thorough look later this week:

The BUILDING.txt addition must have been copied from snappy, since a lot of it still refers to snappy. ;-)

There's a TODO in the code that should get sorted out:
{code}
   // TODO:RG Figure out the overhead, right now keeping it similar to zlib
{code}

Shouldn't this come from IO_COMPRESSION_CODEC_ZSTD_LEVEL_DEFAULT that's already in the config keys?
{code}
@VisibleForTesting static final int DEFAULT_COMPRESSION_LEVEL = 2;
{code}

Speaking of default compression level, could you elaborate on why level 2 was chosen?  The zstd CLI defaults to level 3.


> Add Codec for ZStandard Compression
> -----------------------------------
>
>                 Key: HADOOP-13578
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13578
>             Project: Hadoop Common
>          Issue Type: New Feature
>            Reporter: churro morales
>            Assignee: churro morales
>         Attachments: HADOOP-13578.patch
>
>
> ZStandard: https://github.com/facebook/zstd has been used in production for 6 months by facebook now.  v1.0 was recently released.  Create a codec for this library.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org