You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@orc.apache.org by do...@apache.org on 2021/08/02 05:14:35 UTC
[orc] branch branch-1.6 updated: ORC-895: Use snappy-java 1.1.8.4
in bench/core to support Apple Silicon (#802)
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-1.6
in repository https://gitbox.apache.org/repos/asf/orc.git
The following commit(s) were added to refs/heads/branch-1.6 by this push:
new 2993ce1 ORC-895: Use snappy-java 1.1.8.4 in bench/core to support Apple Silicon (#802)
2993ce1 is described below
commit 2993ce1581d525d9d75d48bc54531941c210f32f
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Sun Aug 1 22:00:55 2021 -0700
ORC-895: Use snappy-java 1.1.8.4 in bench/core to support Apple Silicon (#802)
### What changes were proposed in this pull request?
This PR aims to use `snappy-java` 1.1.8.4 instead of the one (1.1.8) of `parquet-hadoop` dependency.
### Why are the changes needed?
To support benchmark in Apple Silicon, we need to use 1.1.8.2+.
- https://github.com/xerial/snappy-java/releases/tag/1.1.8.2
Support Apple Silicon (M1, Mac-aarch64)
Otherwise, we hit the following error.
```
$ java -jar core/target/orc-benchmarks-core-*-uber.jar generate data -d sales -f parquet
Processing sales [parquet]
[WARN ] Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] no native library is found for os.name=Mac and os.arch=aarch64
```
### How was this patch tested?
Manually.
```
$ cd java
$ ./mvnw clean package -DskipTests -Pbenchmark
$ cd bench
$ java -jar core/target/orc-benchmarks-core-*-uber.jar generate data -d sales -f parquet
```
(cherry picked from commit c1e83ca8b430b365724ac86816a4b90df6e8e432)
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
java/bench/core/pom.xml | 4 ++++
java/bench/pom.xml | 11 +++++++++++
2 files changed, 15 insertions(+)
diff --git a/java/bench/core/pom.xml b/java/bench/core/pom.xml
index b070ef5..8a7cdba 100644
--- a/java/bench/core/pom.xml
+++ b/java/bench/core/pom.xml
@@ -94,6 +94,10 @@
<artifactId>parquet-hadoop</artifactId>
</dependency>
<dependency>
+ <groupId>org.xerial.snappy</groupId>
+ <artifactId>snappy-java</artifactId>
+ </dependency>
+ <dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-core</artifactId>
</dependency>
diff --git a/java/bench/pom.xml b/java/bench/pom.xml
index f16565f..e5819d9 100644
--- a/java/bench/pom.xml
+++ b/java/bench/pom.xml
@@ -337,6 +337,17 @@
<groupId>org.apache.parquet</groupId>
<artifactId>parquet-hadoop</artifactId>
<version>${parquet.version}</version>
+ <exclusions>
+ <exclusion>
+ <groupId>org.xerial.snappy</groupId>
+ <artifactId>snappy-java</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>org.xerial.snappy</groupId>
+ <artifactId>snappy-java</artifactId>
+ <version>1.1.8.4</version>
</dependency>
<dependency>
<groupId>org.apache.parquet</groupId>