You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/02/09 12:47:51 UTC
[GitHub] [arrow] soma-cepel opened a new issue #12376: Error loading native libraries: java.io.FileNotFoundException: libarrow_dataset_jni.dylib
soma-cepel opened a new issue #12376:
URL: https://github.com/apache/arrow/issues/12376
I got this error when I tried to run a simple Java program that uses the **Dataset API** available in version 7.0.0.
The error occurred when trying to get a NativeMemoryPool using the getDefault() method.
```java
. . .
NativeMemoryPool nativeMemoryPool = NativeMemoryPool.getDefault();
FileSystemDatasetFactory factory = new FileSystemDatasetFactory(
new RootAllocator(Long.MAX_VALUE), nativeMemoryPool, FileFormat.PARQUET, uri);
. . .
```
By the way, I installed all the binary libraries for my platform using the instructions available at https://arrow.apache.org/install/ and correctly assigned the property `java.library.path` in my Java Program. The fact is that the library `libarrow_dataset_jni.dylib` was not actually installed even though `libarrow_dataset.700.dylib` and several others were installed with the respective symlinks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] soma-cepel commented on issue #12376: Error loading native libraries: java.io.FileNotFoundException: libarrow_dataset_jni.dylib
Posted by GitBox <gi...@apache.org>.
soma-cepel commented on issue #12376:
URL: https://github.com/apache/arrow/issues/12376#issuecomment-1036363893
hi @davisusanibar, I removed all files in Maven's local cache using `rm -rf ~/.m2/repository/org/apache/arrow` and then built again. Now the `~/.m2/repository/org/apache/arrow/arrow-dataset/7.0.0/arrow-dataset-7.0.0.jar` file contains the .so and .dylib LIBs
The previous incomplete `arrow-dataset-7.0.0.jar` file that was in my local Maven cache I had created in the Arrow build process using the documented procedures and therefore did not contain the LIBs.
Now the error no longer occurs. I'm closing the Issue. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] soma-cepel closed issue #12376: Error loading native libraries: java.io.FileNotFoundException: libarrow_dataset_jni.dylib
Posted by GitBox <gi...@apache.org>.
soma-cepel closed issue #12376:
URL: https://github.com/apache/arrow/issues/12376
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on issue #12376: Error loading native libraries: java.io.FileNotFoundException: libarrow_dataset_jni.dylib
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on issue #12376:
URL: https://github.com/apache/arrow/issues/12376#issuecomment-1035366519
Hi @soma-cepel could you help us to share the error log message to we could review that.
For our testing I see this error but when configure arrow-dataset:7.0.0-SNAPSHOT:
Caused by: java.lang.IllegalStateException: error loading native libraries: java.io.FileNotFoundException: libarrow_dataset_jni.dylib
For arrow-dataset:7.0.0 I did not see errors. The jar contains *.dylib and *.so
```
(base) ➜ otool otool -L libarrow_dataset_jni.dylib
libarrow_dataset_jni.dylib:
@rpath/libarrow_dataset_jni.700.dylib (compatibility version 700.0.0, current version 700.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1200.3.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.0.0)
```
Testing:
```
.. testcode::
import org.apache.arrow.dataset.file.FileFormat;
import org.apache.arrow.dataset.file.FileSystemDatasetFactory;
import org.apache.arrow.dataset.jni.NativeMemoryPool;
import org.apache.arrow.dataset.source.DatasetFactory;
import org.apache.arrow.memory.RootAllocator;
import org.apache.arrow.vector.types.pojo.Schema;
import org.apache.arrow.util.AutoCloseables;
String uri = "file:" + System.getProperty("user.dir") + "/thirdpartydeps/parquetfiles/data1.parquet";
RootAllocator rootAllocator = new RootAllocator(Long.MAX_VALUE);
DatasetFactory datasetFactory = new FileSystemDatasetFactory(rootAllocator, NativeMemoryPool.getDefault(), FileFormat.PARQUET, uri);
Schema schema = datasetFactory.inspect();
AutoCloseables.close(datasetFactory);
System.out.println(schema);
.. testoutput::
Schema<id: Int(32, true), name: Utf8>(metadata: {parquet.avro.schema={"type":"record","name":"User","namespace":"org.apache.arrow.dataset","fields":[{"name":"id","type":["int","null"]},{"name":"name","type":["string","null"]}]}, writer.model.name=avro})
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org