[GitHub] [pinot] richardstartin edited a comment on pull request #7797: use `LZ4` as default compression mode

[GitBox <gi...@apache.org>]

richardstartin edited a comment on pull request #7797:
URL: https://github.com/apache/pinot/pull/7797#issuecomment-974519284


   > LZ4 gives nearly same (and even better sometimes) the compression ratio compared to SNAPPY and the time to compress is much faster.
   
   > ZSTD on the other hand as highest compression ratio (from our tests) but compression/decompression time was higher as well
   
   This is consistent with my experience, but it’s worth noting LZ4 has 16 compression levels which trade speed for ratio, and we default to level 1, whereas beyond level 3 a high compression mode is used - if ratio is really important the level could be exposed as a config parameter. But right now the motivation is faster decompression by default, rather than smaller file sizes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org