You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@parquet.apache.org by GitBox <gi...@apache.org> on 2021/02/19 09:51:30 UTC

[GitHub] [parquet-mr] gszadovszky commented on pull request #872: PARQUET-1975 Disable BrotliCodec for non-x86_64

gszadovszky commented on pull request #872:
URL: https://github.com/apache/parquet-mr/pull/872#issuecomment-781965022


   > The problem is org.apache.hadoop.io.compress.CompressionCodecFactory#CODEC_PROVIDERS (hadoop-common:2.10.1) - https://github.com/apache/hadoop/blob/branch-2.10.1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/CompressionCodecFactory.java#L42.
   > It uses ServiceLoader API to load all implementations of org.apache.hadoop.io.compress.CompressionCodec and brotli-codec fails due to the missing binary.
   > So, having the dependency on the classpath breaks the **loading** of CompressionCodecFactory class and even code that does not use Brotli but any of the other compressions fails immediately.
   
   I see, thanks for the explanation. Then, you may try something like `<arch>!arm64</arch>` instead of using `amd64`. I think `amd64` is too strict and we may end up not testing something without noticing.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org