You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@parquet.apache.org by GitBox <gi...@apache.org> on 2020/04/30 18:16:53 UTC

[GitHub] [parquet-mr] dbtsai commented on pull request #671: PARQUET-1643 Use airlift codecs for LZ4, LZ0, GZIP

dbtsai commented on pull request #671:
URL: https://github.com/apache/parquet-mr/pull/671#issuecomment-622018995


   @samarthjain thanks for the work. I am looking to deploy zstd parquet into prod, but that requires new hadoop with native library support which is not practical in many prod use-cases.
   
   Since airlift is pure Java implementation, what's the performance implications for zstd? I saw there is a benchmark for GZIP, but I don't see benchmark for other codecs.
   
   Also, do we consider to use zstd-jin which is a Java library that packages native implementation of zstd for different platforms in jar?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org