[jira] [Created] (ARROW-17267) Failure loading arrow_dataset_jni library on Mac M1

["Alexander Cai (Jira)" <ji...@apache.org>]

Alexander Cai created ARROW-17267:
-------------------------------------

             Summary: Failure loading arrow_dataset_jni library on Mac M1
                 Key: ARROW-17267
                 URL: https://issues.apache.org/jira/browse/ARROW-17267
             Project: Apache Arrow
          Issue Type: Bug
          Components: Java
    Affects Versions: 8.0.0
         Environment: Mac M1, 16 GB memory
            Reporter: Alexander Cai


On a Mac M1, using Java 11 and Maven, I was initially trying to load a Parquet file using FileSystemDatasetFactory. However, when calling NativeMemoryPool.getDefault(), it throws an UnsatisfiedLinkError:

{code:java}
java.lang.UnsatisfiedLinkError: Can't load library: /var/folders/c9/h03f8v4s037gvwlfgs98b_y00000gp/T/jnilib-18262759266549290103.tmp
    at java.lang.ClassLoader.loadLibrary (ClassLoader.java:2633)
    at java.lang.Runtime.load0 (Runtime.java:768)
    at java.lang.System.load (System.java:1837)
    at org.apache.arrow.dataset.jni.JniLoader.load (JniLoader.java:88)
    at org.apache.arrow.dataset.jni.JniLoader.loadRemaining (JniLoader.java:73)
    at org.apache.arrow.dataset.jni.JniLoader.ensureLoaded (JniLoader.java:60)
    at org.apache.arrow.dataset.jni.NativeMemoryPool.<clinit> (NativeMemoryPool.java:27)
    at com.example.App.main (App.java:8)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
    at java.lang.Thread.run (Thread.java:829){code}

I've created a minimum reproducible example [here|https://gist.github.com/adzcai/7cb1b9316d8529cf2c0ffe19f0901adc], where the only line of code is {noformat}NativeMemoryPool.getDefault(){noformat}. At the call to {noformat}org.apache.arrow.dataset.jni.JniLoader.load{noformat} in the stack trace above, "arrow_dataset_jni" is passed as the name, and "libarrow_dataset_jni.dylib" is the library name, which it successfully locates.

However, this works fine in Java 8. Running the same code in an EC2 container on Java 11 also works fine, so I'm fairly certain this is an M1 issue.


 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)