You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@orc.apache.org by GitBox <gi...@apache.org> on 2021/08/02 01:53:30 UTC

[GitHub] [orc] dongjoon-hyun opened a new pull request #802: ORC-895: Use snappy-java 1.1.8.4 in bench/core to support Apple Silicon

dongjoon-hyun opened a new pull request #802:
URL: https://github.com/apache/orc/pull/802


   ### What changes were proposed in this pull request?
   
   This PR aims to use `snappy-java` 1.1.8.4 instead of the one (1.1.8) of `parquet-hadoop` dependency.
   
   ### Why are the changes needed?
   
   To support benchmark in Apple Silicon, we need to use 1.1.8.2+.
   
   - https://github.com/xerial/snappy-java/releases/tag/1.1.8.2
      Support Apple Silicon (M1, Mac-aarch64)
   
   Otherwise, we hit the following error.
   ```
   $ java -jar core/target/orc-benchmarks-core-*-uber.jar generate data -d sales -f parquet
   Processing sales [parquet]
   [WARN ] Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   Exception in thread "main" org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] no native library is found for os.name=Mac and os.arch=aarch64
   ```
   
   ### How was this patch tested?
   
   Manually.
   ```
   $ cd java
   $ ./mvnw clean package -DskipTests -Pbenchmark
   $ cd bench
   $ java -jar core/target/orc-benchmarks-core-*-uber.jar generate data -d sales -f parquet
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@orc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [orc] dongjoon-hyun commented on pull request #802: ORC-895: Use snappy-java 1.1.8.4 in bench/core to support Apple Silicon

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #802:
URL: https://github.com/apache/orc/pull/802#issuecomment-890713196


   Could you review this please when you have some time, @williamhyun ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@orc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [orc] dongjoon-hyun commented on pull request #802: ORC-895: Use snappy-java 1.1.8.4 in bench/core to support Apple Silicon

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #802:
URL: https://github.com/apache/orc/pull/802#issuecomment-890722198


   Thank you so much!
   I backported this to branch-1.7/1.6.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@orc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [orc] williamhyun merged pull request #802: ORC-895: Use snappy-java 1.1.8.4 in bench/core to support Apple Silicon

Posted by GitBox <gi...@apache.org>.
williamhyun merged pull request #802:
URL: https://github.com/apache/orc/pull/802


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@orc.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org