You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by "wenweijian (Jira)" <ji...@apache.org> on 2023/06/29 17:36:00 UTC

[jira] [Created] (HADOOP-18788) Support LZ4 compressionLevel

wenweijian created HADOOP-18788:
-----------------------------------

             Summary: Support LZ4 compressionLevel
                 Key: HADOOP-18788
                 URL: https://issues.apache.org/jira/browse/HADOOP-18788
             Project: Hadoop Common
          Issue Type: New Feature
          Components: common
            Reporter: wenweijian


The Hadoop's LZ4 compression codec now depends on lz4-java. There two type of compressor in lz4, `fastCompressor` and `highCompressor`. The default lz4 compressor in hadoop is fastCompressor, we also can use highCompressor by using config `IO_COMPRESSION_CODEC_LZ4_USELZ4HC_DEFAULT`

 

When we want to use highCompressor in hadoop, we only can use the default compression level, which is level 9. while highCompressor in lz4-java supports compression level from 1 to 17.

 

I think we can set a configuration to let users to choose different compresssionLevel for lz4 highCompressor.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org