You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/09/13 14:36:44 UTC
[GitHub] [arrow] davisusanibar commented on a diff in pull request #13881: ARROW-17405: [Doc][Java] C Data Interface library able to compile with mvn command
davisusanibar commented on code in PR #13881:
URL: https://github.com/apache/arrow/pull/13881#discussion_r969714079
##########
docs/source/developers/java/building.rst:
##########
@@ -47,25 +47,156 @@ repository:
$ cd arrow
$ git submodule update --init --recursive
-Basic Installation
-------------------
+These are the options available to compile Arrow Java modules with:
+- Maven build tool
+- Docker compose
+- Archery
+
+Building Java Modules
+---------------------
To build the default modules, go to the project root and execute:
+Maven
+~~~~~
+
.. code-block::
$ cd arrow/java
$ export JAVA_HOME=<absolute path to your java home>
$ java --version
$ mvn clean install
-Building JNI Libraries on Linux
--------------------------------
+Docker compose
+~~~~~~~~~~~~~~
+
+.. code-block::
+
+ $ cd arrow/java
+ $ export JAVA_HOME=<absolute path to your java home>
+ $ java --version
+ $ docker-compose run debian-java
+
+Archery
+~~~~~~~
+
+.. code-block::
+
+ $ cd arrow/java
+ $ export JAVA_HOME=<absolute path to your java home>
+ $ java --version
+ $ archery docker run debian-java
+
+Building JNI Libraries (.dylib / .so)
+-------------------------------------
First, we need to build the `C++ shared libraries`_ that the JNI bindings will use.
We can build these manually or we can use `Archery`_ to build them using a Docker container
(This will require installing Docker, Docker Compose, and Archery).
+Note: If you are building on Apple Silicon, be sure to use a JDK version that was compiled
+for that architecture. See, for example, the `Azul JDK <https://www.azul.com/downloads/?os=macos&architecture=arm-64-bit&package=jdk>`_.
+
+Maven
+~~~~~
+
+- To build only the C Data Interface library:
+
+ .. code-block::
+
+ $ cd arrow/java
+ $ export JAVA_HOME=<absolute path to your java home>
+ $ java --version
+ $ mvn clean generate-resources -Pgenerate-cdata-dylib_so -N
+ $ ls -latr ../java-dist/lib
+ |__ libarrow_cdata_jni.dylib
+
+- To build other JNI libraries:
+
+ Not options available on Maven.
+
+CMake
+~~~~~
+
+- To build only the JNI C Data Interface library:
+
+ .. code-block::
+
+ $ cd arrow
+ $ mkdir -p java-dist java-jni
+ $ cmake \
+ -S java \
+ -B java-jni \
+ -DCMAKE_BUILD_TYPE=Release \
+ -DCMAKE_INSTALL_PREFIX=java-dist/lib \
+ -DARROW_JAVA_JNI_ENABLE_DEFAULT=OFF \
+ -DARROW_JAVA_JNI_ENABLE_C=ON \
+ -DBUILD_TESTING=OFF
Review Comment:
Sorted
##########
docs/source/developers/java/building.rst:
##########
@@ -47,25 +47,156 @@ repository:
$ cd arrow
$ git submodule update --init --recursive
-Basic Installation
-------------------
+These are the options available to compile Arrow Java modules with:
+- Maven build tool
+- Docker compose
+- Archery
+
+Building Java Modules
+---------------------
To build the default modules, go to the project root and execute:
+Maven
+~~~~~
+
.. code-block::
$ cd arrow/java
$ export JAVA_HOME=<absolute path to your java home>
$ java --version
$ mvn clean install
-Building JNI Libraries on Linux
--------------------------------
+Docker compose
+~~~~~~~~~~~~~~
+
+.. code-block::
+
+ $ cd arrow/java
+ $ export JAVA_HOME=<absolute path to your java home>
+ $ java --version
+ $ docker-compose run debian-java
+
+Archery
+~~~~~~~
+
+.. code-block::
+
+ $ cd arrow/java
+ $ export JAVA_HOME=<absolute path to your java home>
+ $ java --version
+ $ archery docker run debian-java
+
+Building JNI Libraries (.dylib / .so)
+-------------------------------------
First, we need to build the `C++ shared libraries`_ that the JNI bindings will use.
We can build these manually or we can use `Archery`_ to build them using a Docker container
(This will require installing Docker, Docker Compose, and Archery).
+Note: If you are building on Apple Silicon, be sure to use a JDK version that was compiled
+for that architecture. See, for example, the `Azul JDK <https://www.azul.com/downloads/?os=macos&architecture=arm-64-bit&package=jdk>`_.
+
+Maven
+~~~~~
+
+- To build only the C Data Interface library:
+
+ .. code-block::
+
+ $ cd arrow/java
+ $ export JAVA_HOME=<absolute path to your java home>
+ $ java --version
+ $ mvn clean generate-resources -Pgenerate-cdata-dylib_so -N
+ $ ls -latr ../java-dist/lib
+ |__ libarrow_cdata_jni.dylib
+
+- To build other JNI libraries:
+
+ Not options available on Maven.
+
+CMake
+~~~~~
+
+- To build only the JNI C Data Interface library:
+
+ .. code-block::
+
+ $ cd arrow
+ $ mkdir -p java-dist java-jni
+ $ cmake \
+ -S java \
+ -B java-jni \
+ -DCMAKE_BUILD_TYPE=Release \
+ -DCMAKE_INSTALL_PREFIX=java-dist/lib \
+ -DARROW_JAVA_JNI_ENABLE_DEFAULT=OFF \
+ -DARROW_JAVA_JNI_ENABLE_C=ON \
+ -DBUILD_TESTING=OFF
+ $ cmake --build java-jni --target install --config Release
+ $ ls -latr java-dist/lib
+ |__ libarrow_cdata_jni.dylib
+
+- To build JNI ORC & Gandiva libraries:
+
+ .. code-block::
+
+ $ cd arrow
+ $ brew bundle --file=cpp/Brewfile
+ Homebrew Bundle complete! 25 Brewfile dependencies now installed.
+ $ brew uninstall aws-sdk-cpp
+ (We can't use aws-sdk-cpp installed by Homebrew because it has
+ an issue: https://github.com/aws/aws-sdk-cpp/issues/1809 )
+ $ export JAVA_HOME=<absolute path to your java home>
+ $ mkdir -p java-dist cpp-jni
+ $ cmake \
+ -S cpp \
+ -B cpp-jni \
+ -DARROW_DEPENDENCY_USE_SHARED=OFF\
+ -DARROW_CSV=ON \
+ -DARROW_DATASET=ON \
+ -DARROW_FILESYSTEM=ON \
+ -DARROW_GANDIVA=ON \
+ -DARROW_GANDIVA_JAVA=ON \
+ -DARROW_GANDIVA_STATIC_LIBSTDCPP=ON \
+ -DARROW_JNI=ON \
+ -DARROW_ORC=ON \
+ -DARROW_PARQUET=ON \
+ -DARROW_PLASMA=ON \
+ -DARROW_PLASMA_JAVA_CLIENT=ON \
+ -DARROW_S3=ON \
+ -DARROW_USE_CCACHE=ON \
+ -DCMAKE_BUILD_TYPE=Release \
+ -DCMAKE_INSTALL_LIBDIR=lib \
+ -DCMAKE_INSTALL_PREFIX=java-dist \
+ -DCMAKE_UNITY_BUILD=ON \
+ -DARROW_DEPENDENCY_SOURCE=BUNDLED
Review Comment:
Sorted
##########
docs/source/developers/java/building.rst:
##########
@@ -47,25 +47,156 @@ repository:
$ cd arrow
$ git submodule update --init --recursive
-Basic Installation
-------------------
+These are the options available to compile Arrow Java modules with:
+- Maven build tool
+- Docker compose
+- Archery
+
+Building Java Modules
+---------------------
To build the default modules, go to the project root and execute:
+Maven
+~~~~~
+
.. code-block::
$ cd arrow/java
$ export JAVA_HOME=<absolute path to your java home>
$ java --version
$ mvn clean install
-Building JNI Libraries on Linux
--------------------------------
+Docker compose
+~~~~~~~~~~~~~~
+
+.. code-block::
+
+ $ cd arrow/java
+ $ export JAVA_HOME=<absolute path to your java home>
+ $ java --version
+ $ docker-compose run debian-java
+
+Archery
+~~~~~~~
+
+.. code-block::
+
+ $ cd arrow/java
+ $ export JAVA_HOME=<absolute path to your java home>
+ $ java --version
+ $ archery docker run debian-java
+
+Building JNI Libraries (.dylib / .so)
+-------------------------------------
First, we need to build the `C++ shared libraries`_ that the JNI bindings will use.
We can build these manually or we can use `Archery`_ to build them using a Docker container
(This will require installing Docker, Docker Compose, and Archery).
+Note: If you are building on Apple Silicon, be sure to use a JDK version that was compiled
+for that architecture. See, for example, the `Azul JDK <https://www.azul.com/downloads/?os=macos&architecture=arm-64-bit&package=jdk>`_.
+
+Maven
+~~~~~
+
+- To build only the C Data Interface library:
+
+ .. code-block::
+
+ $ cd arrow/java
+ $ export JAVA_HOME=<absolute path to your java home>
+ $ java --version
+ $ mvn clean generate-resources -Pgenerate-cdata-dylib_so -N
+ $ ls -latr ../java-dist/lib
+ |__ libarrow_cdata_jni.dylib
+
+- To build other JNI libraries:
+
+ Not options available on Maven.
+
+CMake
+~~~~~
+
+- To build only the JNI C Data Interface library:
+
+ .. code-block::
+
+ $ cd arrow
+ $ mkdir -p java-dist java-jni
+ $ cmake \
+ -S java \
+ -B java-jni \
+ -DCMAKE_BUILD_TYPE=Release \
+ -DCMAKE_INSTALL_PREFIX=java-dist/lib \
+ -DARROW_JAVA_JNI_ENABLE_DEFAULT=OFF \
+ -DARROW_JAVA_JNI_ENABLE_C=ON \
+ -DBUILD_TESTING=OFF
+ $ cmake --build java-jni --target install --config Release
+ $ ls -latr java-dist/lib
+ |__ libarrow_cdata_jni.dylib
+
+- To build JNI ORC & Gandiva libraries:
+
+ .. code-block::
+
+ $ cd arrow
+ $ brew bundle --file=cpp/Brewfile
+ Homebrew Bundle complete! 25 Brewfile dependencies now installed.
+ $ brew uninstall aws-sdk-cpp
+ (We can't use aws-sdk-cpp installed by Homebrew because it has
+ an issue: https://github.com/aws/aws-sdk-cpp/issues/1809 )
+ $ export JAVA_HOME=<absolute path to your java home>
+ $ mkdir -p java-dist cpp-jni
+ $ cmake \
+ -S cpp \
+ -B cpp-jni \
+ -DARROW_DEPENDENCY_USE_SHARED=OFF\
+ -DARROW_CSV=ON \
+ -DARROW_DATASET=ON \
+ -DARROW_FILESYSTEM=ON \
+ -DARROW_GANDIVA=ON \
+ -DARROW_GANDIVA_JAVA=ON \
+ -DARROW_GANDIVA_STATIC_LIBSTDCPP=ON \
+ -DARROW_JNI=ON \
+ -DARROW_ORC=ON \
+ -DARROW_PARQUET=ON \
+ -DARROW_PLASMA=ON \
+ -DARROW_PLASMA_JAVA_CLIENT=ON \
+ -DARROW_S3=ON \
+ -DARROW_USE_CCACHE=ON \
+ -DCMAKE_BUILD_TYPE=Release \
+ -DCMAKE_INSTALL_LIBDIR=lib \
+ -DCMAKE_INSTALL_PREFIX=java-dist \
+ -DCMAKE_UNITY_BUILD=ON \
+ -DARROW_DEPENDENCY_SOURCE=BUNDLED
+ $ cmake --build cpp-jni --target install --config Release
+ $ ls -latr java-dist/lib
+ |__ libarrow_orc_jni.dylib
+ |__ libgandiva_jni.dylib
+
+- To build only the Dataset library:
+
+ .. code-block::
+
+ $ cd arrow
+ $ mkdir -p java-dist java-jni
+ $ cmake \
+ -S java \
+ -B java-jni \
+ -DCMAKE_BUILD_TYPE=Release \
+ -DCMAKE_INSTALL_PREFIX=java-dist/lib \
+ -DCMAKE_PREFIX_PATH=$PWD/java-dist \
+ -DARROW_JAVA_JNI_ENABLE_DEFAULT=OFF \
+ -DARROW_JAVA_JNI_ENABLE_DATASET=ON \
+ -DBUILD_TESTING=OFF
Review Comment:
Sorted
##########
java/pom.xml:
##########
@@ -905,6 +905,70 @@
</build>
</profile>
+ <profile>
+ <id>generate-cdata-dylib_so</id>
+ <properties>
+ <arrow.c.jni.dist.dir>java-dist/lib</arrow.c.jni.dist.dir>
+ </properties>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>exec-maven-plugin</artifactId>
+ <version>3.1.0</version>
+ <executions>
+ <execution>
+ <id>cdatadir</id>
+ <phase>generate-resources</phase>
+ <goals>
+ <goal>exec</goal>
+ </goals>
+ <configuration>
+ <executable>mkdir</executable>
+ <commandlineArgs>-p java-dist java-jni</commandlineArgs>
+ <workingDirectory>../</workingDirectory>
+ </configuration>
+ </execution>
+ <execution>
+ <id>cdatadefine</id>
+ <phase>generate-resources</phase>
+ <goals>
+ <goal>exec</goal>
+ </goals>
+ <configuration>
+ <executable>cmake</executable>
+ <commandlineArgs>
+ -S java
+ -B java-jni
+ -DCMAKE_BUILD_TYPE=Release
+ -DCMAKE_INSTALL_PREFIX=${arrow.c.jni.dist.dir}
+ -DARROW_JAVA_JNI_ENABLE_DEFAULT=OFF
+ -DARROW_JAVA_JNI_ENABLE_C=ON
+ -DBUILD_TESTING=OFF
Review Comment:
Sorted
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org