You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "jaynewchen (via GitHub)" <gi...@apache.org> on 2024/04/14 07:18:23 UTC
[I] Failed when building JNI libraries (*.so on Linux) for arrow-java-dataset, has anyone ever see the same question? [arrow]
jaynewchen opened a new issue, #41191:
URL: https://github.com/apache/arrow/issues/41191
### Describe the usage question you have. Please include as many useful details as possible.
My purpose：To build arrow java dataset module (including the JNI part)
I hava built the JNI C Data Interface library successful in a x86 redhat Linux machine，followed the doc：https://arrow.apache.org/docs/developers/java/building.html#building-java-jni-modules
with these commands:
```shell
cd arrow/java
mvn generate-resources -Pgenerate-libs-cdata-all-os -N
```
But when I run these commands to build all JNI libraries except the JNI C Data Interface library (including dataset module)，I get this output：
```shell
mvn generate-resources -Pgenerate-libs-jni-macos-linux -N
```
```shell
INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: linux
[INFO] os.detected.arch: x86_64
[INFO] os.detected.bitness: 64
[INFO] os.detected.version: 3.10
[INFO] os.detected.version.major: 3
[INFO] os.detected.version.minor: 10
[INFO] os.detected.release: euleros
[INFO] os.detected.release.version: 2.0
[INFO] os.detected.release.like.euleros: true
[INFO] os.detected.release.like.rhel: true
[INFO] os.detected.release.like.fedora: true
[INFO] os.detected.release.like.centos: true
[INFO] os.detected.classifier: linux-x86_64
[INFO]
[INFO] ------------------< org.apache.arrow:arrow-java-root >------------------
[INFO] Building Apache Arrow Java Root POM 14.0.1
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- apache-rat-plugin:0.13:check (rat-checks) @ arrow-java-root ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 62 implicit excludes (use -debug for more details).
[INFO] 39 explicit excludes (use -debug for more details).
[INFO] 1196 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 0, approved: 1193 licenses.
[INFO]
[INFO] --- maven-checkstyle-plugin:3.1.0:check (validate) @ arrow-java-root ---
[INFO] Starting audit...
Audit done.
[INFO]
[INFO] --- git-commit-id-plugin:4.0.5:revision (for-jars) @ arrow-java-root ---
[INFO]
[INFO] --- git-commit-id-plugin:4.0.5:revision (for-source-tarball) @ arrow-java-root ---
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ arrow-java-root ---
[INFO]
[INFO] --- exec-maven-plugin:3.1.0:exec (jni-cpp-cmake) @ arrow-java-root ---
-- Building using CMake version: 3.21.0
-- Arrow version: 14.0.1 (full: '14.0.1')
-- Arrow SO version: 1400 (full: 1400.1.0)
-- clang-tidy 14 not found
-- clang-format 14 not found
-- Could NOT find ClangTools (missing: CLANG_FORMAT_BIN CLANG_TIDY_BIN)
-- infer not found
-- Could NOT find Python3 (missing: Python3_EXECUTABLE Interpreter)
Reason given by package:
Interpreter: Cannot use the interpreter "/opt/shangh/ck_tools/python-3.9.11/bin/python3.9"
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
-- Found cpplint executable at /srv/cjx/project/arrow-release-14.0.1-rc0/cpp/build-support/cpplint.py
-- System processor: x86_64
-- Arrow build warning level: PRODUCTION
-- Using ld linker
-- Build Type: RELEASE
-- Using BUNDLED approach to find dependencies
-- ARROW_ABSL_BUILD_VERSION: 20211102.0
-- ARROW_ABSL_BUILD_SHA256_CHECKSUM: dcf71b9cba8dc0ca9940c4b316a0c796be8fab42b070bb6b7cab62b48f0e66c4
-- ARROW_AWS_C_AUTH_BUILD_VERSION: v0.6.22
-- ARROW_AWS_C_AUTH_BUILD_SHA256_CHECKSUM: 691a6b4418afcd3dc141351b6ad33fccd8e3ff84df0e9e045b42295d284ee14c
-- ARROW_AWS_C_CAL_BUILD_VERSION: v0.5.20
-- ARROW_AWS_C_CAL_BUILD_SHA256_CHECKSUM: acc352359bd06f8597415c366cf4ec4f00d0b0da92d637039a73323dd55b6cd0
-- ARROW_AWS_C_COMMON_BUILD_VERSION: v0.8.9
-- ARROW_AWS_C_COMMON_BUILD_SHA256_CHECKSUM: 2f3fbaf7c38eae5a00e2a816d09b81177f93529ae8ba1b82dc8f31407565327a
-- ARROW_AWS_C_COMPRESSION_BUILD_VERSION: v0.2.16
-- ARROW_AWS_C_COMPRESSION_BUILD_SHA256_CHECKSUM: 044b1dbbca431a07bde8255ef9ec443c300fc60d4c9408d4b862f65e496687f4
-- ARROW_AWS_C_EVENT_STREAM_BUILD_VERSION: v0.2.18
-- ARROW_AWS_C_EVENT_STREAM_BUILD_SHA256_CHECKSUM: 310ca617f713bf664e4c7485a3d42c1fb57813abd0107e49790d107def7cde4f
-- ARROW_AWS_C_HTTP_BUILD_VERSION: v0.7.3
-- ARROW_AWS_C_HTTP_BUILD_SHA256_CHECKSUM: 07e16c6bf5eba6f0dea96b6f55eae312a7c95b736f4d2e4a210000f45d8265ae
-- ARROW_AWS_C_IO_BUILD_VERSION: v0.13.14
-- ARROW_AWS_C_IO_BUILD_SHA256_CHECKSUM: 12b66510c3d9a4f7e9b714e9cfab2a5bf835f8b9ce2f909d20ae2a2128608c71
-- ARROW_AWS_C_MQTT_BUILD_VERSION: v0.8.4
-- ARROW_AWS_C_MQTT_BUILD_SHA256_CHECKSUM: 232eeac63e72883d460c686a09b98cdd811d24579affac47c5c3f696f956773f
-- ARROW_AWS_C_S3_BUILD_VERSION: v0.2.3
-- ARROW_AWS_C_S3_BUILD_SHA256_CHECKSUM: a00b3c9f319cd1c9aa2c3fa15098864df94b066dcba0deaccbb3caa952d902fe
-- ARROW_AWS_C_SDKUTILS_BUILD_VERSION: v0.1.6
-- ARROW_AWS_C_SDKUTILS_BUILD_SHA256_CHECKSUM: 8a2951344b2fb541eab1e9ca17c18a7fcbfd2aaff4cdd31d362d1fad96111b91
-- ARROW_AWS_CHECKSUMS_BUILD_VERSION: v0.1.13
-- ARROW_AWS_CHECKSUMS_BUILD_SHA256_CHECKSUM: 0f897686f1963253c5069a0e495b85c31635ba146cd3ac38cc2ea31eaf54694d
-- ARROW_AWS_CRT_CPP_BUILD_VERSION: v0.18.16
-- ARROW_AWS_CRT_CPP_BUILD_SHA256_CHECKSUM: 9e69bc1dc4b50871d1038aa9ff6ddeb4c9b28f7d6b5e5b1b69041ccf50a13483
-- ARROW_AWS_LC_BUILD_VERSION: v1.3.0
-- ARROW_AWS_LC_BUILD_SHA256_CHECKSUM: ae96a3567161552744fc0cae8b4d68ed88b1ec0f3d3c98700070115356da5a37
-- ARROW_AWSSDK_BUILD_VERSION: 1.10.55
-- ARROW_AWSSDK_BUILD_SHA256_CHECKSUM: 2d552fb1a84bef4a9b65e34aa7031851ed2aef5319e02cc6e4cb735c48aa30de
-- ARROW_AZURE_SDK_BUILD_VERSION: azure-core_1.10.3
-- ARROW_AZURE_SDK_BUILD_SHA256_CHECKSUM: dd624c2f86adf474d2d0a23066be6e27af9cbd7e3f8d9d8fd7bf981e884b7b48
-- ARROW_BOOST_BUILD_VERSION: 1.81.0
-- ARROW_BOOST_BUILD_SHA256_CHECKSUM: 9e0ffae35528c35f90468997bc8d99500bf179cbae355415a89a600c38e13574
-- ARROW_BROTLI_BUILD_VERSION: v1.0.9
-- ARROW_BROTLI_BUILD_SHA256_CHECKSUM: f9e8d81d0405ba66d181529af42a3354f838c939095ff99930da6aa9cdf6fe46
-- ARROW_BZIP2_BUILD_VERSION: 1.0.8
-- ARROW_BZIP2_BUILD_SHA256_CHECKSUM: ab5a03176ee106d3f0fa90e381da478ddae405918153cca248e682cd0c4a2269
-- ARROW_CARES_BUILD_VERSION: 1.17.2
-- ARROW_CARES_BUILD_SHA256_CHECKSUM: 4803c844ce20ce510ef0eb83f8ea41fa24ecaae9d280c468c582d2bb25b3913d
-- ARROW_CRC32C_BUILD_VERSION: 1.1.2
-- ARROW_CRC32C_BUILD_SHA256_CHECKSUM: ac07840513072b7fcebda6e821068aa04889018f24e10e46181068fb214d7e56
-- ARROW_GBENCHMARK_BUILD_VERSION: v1.7.1
-- ARROW_GBENCHMARK_BUILD_SHA256_CHECKSUM: 6430e4092653380d9dc4ccb45a1e2dc9259d581f4866dc0759713126056bc1d7
-- ARROW_GFLAGS_BUILD_VERSION: v2.2.2
-- ARROW_GFLAGS_BUILD_SHA256_CHECKSUM: 34af2f15cf7367513b352bdcd2493ab14ce43692d2dcd9dfc499492966c64dcf
-- ARROW_GLOG_BUILD_VERSION: v0.5.0
-- ARROW_GLOG_BUILD_SHA256_CHECKSUM: eede71f28371bf39aa69b45de23b329d37214016e2055269b3b5e7cfd40b59f5
-- ARROW_GOOGLE_CLOUD_CPP_BUILD_VERSION: v2.12.0
-- ARROW_GOOGLE_CLOUD_CPP_BUILD_SHA256_CHECKSUM: 8cda870803925c62de8716a765e03eb9d34249977e5cdb7d0d20367e997a55e2
-- ARROW_GRPC_BUILD_VERSION: v1.46.3
-- ARROW_GRPC_BUILD_SHA256_CHECKSUM: d6cbf22cb5007af71b61c6be316a79397469c58c82a942552a62e708bce60964
-- ARROW_GTEST_BUILD_VERSION: 1.11.0
-- ARROW_GTEST_BUILD_SHA256_CHECKSUM: b4870bf121ff7795ba20d20bcdd8627b8e088f2d1dab299a031c1034eddc93d5
-- ARROW_JEMALLOC_BUILD_VERSION: 5.3.0
-- ARROW_JEMALLOC_BUILD_SHA256_CHECKSUM: 2db82d1e7119df3e71b7640219b6dfe84789bc0537983c3b7ac4f7189aecfeaa
-- ARROW_LZ4_BUILD_VERSION: v1.9.4
-- ARROW_LZ4_BUILD_SHA256_CHECKSUM: 0b0e3aa07c8c063ddf40b082bdf7e37a1562bda40a0ff5272957f3e987e0e54b
-- ARROW_MIMALLOC_BUILD_VERSION: v2.0.6
-- ARROW_MIMALLOC_BUILD_SHA256_CHECKSUM: 9f05c94cc2b017ed13698834ac2a3567b6339a8bde27640df5a1581d49d05ce5
-- ARROW_NLOHMANN_JSON_BUILD_VERSION: v3.10.5
-- ARROW_NLOHMANN_JSON_BUILD_SHA256_CHECKSUM: 5daca6ca216495edf89d167f808d1d03c4a4d929cef7da5e10f135ae1540c7e4
-- ARROW_OPENTELEMETRY_BUILD_VERSION: v1.8.1
-- ARROW_OPENTELEMETRY_BUILD_SHA256_CHECKSUM: 3d640201594b07f08dade9cd1017bd0b59674daca26223b560b9bb6bf56264c2
-- ARROW_OPENTELEMETRY_PROTO_BUILD_VERSION: v0.17.0
-- ARROW_OPENTELEMETRY_PROTO_BUILD_SHA256_CHECKSUM: f269fbcb30e17b03caa1decd231ce826e59d7651c0f71c3b28eb5140b4bb5412
-- ARROW_ORC_BUILD_VERSION: 1.9.0
-- ARROW_ORC_BUILD_SHA256_CHECKSUM: 0dca8bbccdb2ee87e59ba964933436beebd02ea78c4134424828a8127fbc4faa
-- ARROW_PROTOBUF_BUILD_VERSION: v21.3
-- ARROW_PROTOBUF_BUILD_SHA256_CHECKSUM: 2f723218f6cb709ae4cdc4fb5ed56a5951fc5d466f0128ce4c946b8c78c8c49f
-- ARROW_RAPIDJSON_BUILD_VERSION: 232389d4f1012dddec4ef84861face2d2ba85709
-- ARROW_RAPIDJSON_BUILD_SHA256_CHECKSUM: b9290a9a6d444c8e049bd589ab804e0ccf2b05dc5984a19ed5ae75d090064806
-- ARROW_RE2_BUILD_VERSION: 2022-06-01
-- ARROW_RE2_BUILD_SHA256_CHECKSUM: f89c61410a072e5cbcf8c27e3a778da7d6fd2f2b5b1445cd4f4508bee946ab0f
-- ARROW_SNAPPY_BUILD_VERSION: 1.1.10
-- ARROW_SNAPPY_BUILD_SHA256_CHECKSUM: 49d831bffcc5f3d01482340fe5af59852ca2fe76c3e05df0e67203ebbe0f1d90
-- ARROW_SUBSTRAIT_BUILD_VERSION: v0.27.0
-- ARROW_SUBSTRAIT_BUILD_SHA256_CHECKSUM: 4ed375f69d972a57fdc5ec406c17003a111831d8640d3f1733eccd4b3ff45628
-- ARROW_S2N_TLS_BUILD_VERSION: v1.3.35
-- ARROW_S2N_TLS_BUILD_SHA256_CHECKSUM: 9d32b26e6bfcc058d98248bf8fc231537e347395dd89cf62bb432b55c5da990d
-- ARROW_THRIFT_BUILD_VERSION: 0.16.0
-- ARROW_THRIFT_BUILD_SHA256_CHECKSUM: f460b5c1ca30d8918ff95ea3eb6291b3951cf518553566088f3f2be8981f6209
-- ARROW_UCX_BUILD_VERSION: 1.12.1
-- ARROW_UCX_BUILD_SHA256_CHECKSUM: 9bef31aed0e28bf1973d28d74d9ac4f8926c43ca3b7010bd22a084e164e31b71
-- ARROW_UTF8PROC_BUILD_VERSION: v2.7.0
-- ARROW_UTF8PROC_BUILD_SHA256_CHECKSUM: 4bb121e297293c0fd55f08f83afab6d35d48f0af4ecc07523ad8ec99aa2b12a1
-- ARROW_XSIMD_BUILD_VERSION: 9.0.1
-- ARROW_XSIMD_BUILD_SHA256_CHECKSUM: b1bb5f92167fd3a4f25749db0be7e61ed37e0a5d943490f3accdcd2cd2918cc0
-- ARROW_ZLIB_BUILD_VERSION: 1.2.13
-- ARROW_ZLIB_BUILD_SHA256_CHECKSUM: b3a24de97a8fdbc835b9833169501030b8977031bcb54b3b3ac13740f846ab30
-- ARROW_ZSTD_BUILD_VERSION: 1.5.5
-- ARROW_ZSTD_BUILD_SHA256_CHECKSUM: 9c4396cc829cfae319a6e2615202e82aad41372073482fce286fac78646d3ee4
-- Boost include dir:
-- Providing CMake module for OpenSSLAlt as part of Arrow CMake package
-- Found OpenSSL Crypto Library: /usr/local/openssl/lib/libcrypto.a
-- Building with OpenSSL (Version: 1.1.1k) support
-- Building Apache Thrift from source
-- Building Protocol Buffers from source
-- Found protoc: /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/protobuf_ep-install/bin/protoc
-- Building Substrait from source
-- Building jemalloc from source
-- Building RapidJSON from source
-- Building xsimd from source
-- Building ZLIB from source
-- Building RE2 from source
-- Building utf8proc from source
-- Found hdfs.h at: /srv/cjx/project/arrow-release-14.0.1-rc0/cpp/thirdparty/hadoop/include/hdfs.h
-- Building AWS C++ SDK from source
-- Found AWS SDK headers: /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/awssdk_ep-install/include
-- Found AWS SDK libraries: aws-cpp-sdk-identity-management;aws-cpp-sdk-sts;aws-cpp-sdk-cognito-identity;aws-cpp-sdk-s3;aws-cpp-sdk-core;AWS::aws-crt-cpp;AWS::aws-c-s3;AWS::aws-c-auth;AWS:
:aws-c-mqtt;AWS::aws-c-http;AWS::aws-c-compression;AWS::aws-c-sdkutils;AWS::aws-c-event-stream;AWS::aws-c-io;AWS::aws-c-cal;AWS::aws-checksums;AWS::aws-c-common;AWS::s2n-tls
-- All bundled static libraries: thrift::thrift;arrow::protobuf::libprotobuf;substrait;jemalloc::jemalloc;ZLIB::ZLIB;re2::re2;utf8proc::utf8proc;aws-cpp-sdk-identity-management;aws-cpp-sdk
-sts;aws-cpp-sdk-cognito-identity;aws-cpp-sdk-s3;aws-cpp-sdk-core;AWS::aws-crt-cpp;AWS::aws-c-s3;AWS::aws-c-auth;AWS::aws-c-mqtt;AWS::aws-c-http;AWS::aws-c-compression;AWS::aws-c-sdkutils;
AWS::aws-c-event-stream;AWS::aws-c-io;AWS::aws-c-cal;AWS::aws-checksums;AWS::aws-c-common;AWS::s2n-tls
-- CMAKE_C_FLAGS: -Wall -fno-semantic-interposition -msse4.2
-- CMAKE_CXX_FLAGS: -Wno-noexcept-type -Wno-subobject-linkage -fdiagnostics-color=always -Wall -fno-semantic-interposition -msse4.2
-- CMAKE_C_FLAGS_RELEASE: -O3 -DNDEBUG -O2 -ftree-vectorize
-- CMAKE_CXX_FLAGS_RELEASE: -O3 -DNDEBUG -O2 -ftree-vectorize
-- Creating bundled static library target arrow_bundled_dependencies at /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/release/libarrow_bundled_dependencies.a
CMake Error at src/arrow/CMakeLists.txt:715 (target_compile_features):
target_compile_features no known features for CXX compiler
"GNU"
version 9.4.0.
CMake Error at src/arrow/CMakeLists.txt:715 (target_compile_features):
target_compile_features no known features for CXX compiler
"GNU"
version 9.4.0.
-- ---------------------------------------------------------------------
-- Arrow version: 14.0.1
--
-- Build configuration summary:
-- Generator: Unix Makefiles
-- Build type: RELEASE
-- Source directory: /srv/cjx/project/arrow-release-14.0.1-rc0/cpp
-- Install prefix: /srv/cjx/project/arrow-release-14.0.1-rc0/java-dist
--
-- Compile and link options:
--
-- ARROW_CXXFLAGS="" [default=""]
-- Compiler flags to append when compiling Arrow
-- ARROW_BUILD_STATIC=ON [default=ON]
-- Build static libraries
-- ARROW_BUILD_SHARED=OFF [default=ON]
-- Build shared libraries
-- ARROW_PACKAGE_KIND="" [default=""]
-- Arbitrary string that identifies the kind of package
-- (for informational purposes)
-- ARROW_GIT_ID="" [default=""]
-- The Arrow git commit id (if any)
-- ARROW_GIT_DESCRIPTION="" [default=""]
-- The Arrow git commit description (if any)
-- ARROW_NO_DEPRECATED_API=OFF [default=OFF]
-- Exclude deprecated APIs from build
-- ARROW_POSITION_INDEPENDENT_CODE=ON [default=ON]
-- Whether to create position-independent target
-- ARROW_USE_CCACHE=ON [default=ON]
-- Use ccache when compiling (if available)
-- ARROW_USE_SCCACHE=ON [default=ON]
-- Use sccache when compiling (if available),
-- takes precedence over ccache if a storage backend is configured
-- ARROW_USE_LD_GOLD=OFF [default=OFF]
-- Use ld.gold for linking on Linux (if available)
-- ARROW_USE_PRECOMPILED_HEADERS=OFF [default=OFF]
-- Use precompiled headers when compiling
-- ARROW_SIMD_LEVEL=SSE4_2 [default=DEFAULT|NONE|SSE4_2|AVX2|AVX512|NEON|SVE|SVE128|SVE256|SVE512]
-- Compile-time SIMD optimization level
-- ARROW_RUNTIME_SIMD_LEVEL=MAX [default=MAX|NONE|SSE4_2|AVX2|AVX512]
-- Max runtime SIMD optimization level
-- ARROW_ALTIVEC=ON [default=ON]
-- Build with Altivec if compiler has support
-- ARROW_RPATH_ORIGIN=OFF [default=OFF]
-- Build Arrow libraries with RATH set to $ORIGIN
-- ARROW_INSTALL_NAME_RPATH=ON [default=ON]
-- Build Arrow libraries with install_name set to @rpath
-- ARROW_GGDB_DEBUG=ON [default=ON]
-- Pass -ggdb flag to debug builds
-- ARROW_WITH_MUSL=OFF [default=OFF]
-- Whether the system libc is musl or not
-- ARROW_ENABLE_THREADING=ON [default=ON]
-- Enable threading in Arrow core
--
-- Test and benchmark options:
--
-- ARROW_BUILD_EXAMPLES=OFF [default=OFF]
-- Build the Arrow examples
-- ARROW_BUILD_TESTS=OFF [default=OFF]
-- Build the Arrow googletest unit tests
-- ARROW_ENABLE_TIMING_TESTS=ON [default=ON]
-- Enable timing-sensitive tests
-- ARROW_BUILD_INTEGRATION=OFF [default=OFF]
-- Build the Arrow integration test executables
-- ARROW_BUILD_BENCHMARKS=OFF [default=OFF]
-- Build the Arrow micro benchmarks
-- ARROW_BUILD_BENCHMARKS_REFERENCE=OFF [default=OFF]
-- Build the Arrow micro reference benchmarks
-- ARROW_BUILD_OPENMP_BENCHMARKS=OFF [default=OFF]
-- Build the Arrow benchmarks that rely on OpenMP
-- ARROW_BUILD_DETAILED_BENCHMARKS=OFF [default=OFF]
-- Build benchmarks that do a longer exploration of performance
-- ARROW_TEST_LINKAGE=static [default=static|shared]
-- Linkage of Arrow libraries with unit tests executables.
-- ARROW_FUZZING=OFF [default=OFF]
-- Build Arrow Fuzzing executables
-- ARROW_LARGE_MEMORY_TESTS=OFF [default=OFF]
-- Enable unit tests which use large memory
--
-- Lint options:
--
-- ARROW_ONLY_LINT=OFF [default=OFF]
-- Only define the lint and check-format targets
-- ARROW_VERBOSE_LINT=OFF [default=OFF]
-- If off, 'quiet' flags will be passed to linting tools
-- ARROW_GENERATE_COVERAGE=OFF [default=OFF]
-- Build with C++ code coverage enabled
--
-- Checks options:
--
-- ARROW_TEST_MEMCHECK=OFF [default=OFF]
-- Run the test suite using valgrind --tool=memcheck
-- ARROW_USE_ASAN=OFF [default=OFF]
-- Enable Address Sanitizer checks
-- ARROW_USE_TSAN=OFF [default=OFF]
-- Enable Thread Sanitizer checks
-- ARROW_USE_UBSAN=OFF [default=OFF]
-- Enable Undefined Behavior sanitizer checks
--
-- Project component options:
--
-- ARROW_ACERO=ON [default=OFF]
-- Build the Arrow Acero Engine Module
-- ARROW_AZURE=OFF [default=OFF]
-- Build Arrow with Azure support (requires the Azure SDK for C++)
-- ARROW_BUILD_UTILITIES=OFF [default=OFF]
-- Build Arrow commandline utilities
-- ARROW_COMPUTE=ON [default=OFF]
-- Build all Arrow Compute kernels
-- ARROW_CSV=ON [default=OFF]
-- Build the Arrow CSV Parser Module
-- ARROW_CUDA=OFF [default=OFF]
-- Build the Arrow CUDA extensions (requires CUDA toolkit)
-- ARROW_DATASET=ON [default=OFF]
-- Build the Arrow Dataset Modules
-- ARROW_FILESYSTEM=ON [default=OFF]
-- Build the Arrow Filesystem Layer
-- ARROW_FLIGHT=OFF [default=OFF]
-- Build the Arrow Flight RPC System (requires GRPC, Protocol Buffers)
-- ARROW_FLIGHT_SQL=OFF [default=OFF]
-- Build the Arrow Flight SQL extension
-- ARROW_GANDIVA=OFF [default=OFF]
-- Build the Gandiva libraries
-- ARROW_GCS=OFF [default=OFF]
-- Build Arrow with GCS support (requires the GCloud SDK for C++)
-- ARROW_HDFS=OFF [default=OFF]
-- Build the Arrow HDFS bridge
-- ARROW_IPC=ON [default=ON]
-- Build the Arrow IPC extensions
-- ARROW_JEMALLOC=ON [default=ON]
-- Build the Arrow jemalloc-based allocator
-- ARROW_JSON=ON [default=OFF]
-- Build Arrow with JSON support (requires RapidJSON)
-- ARROW_MIMALLOC=OFF [default=OFF]
-- Build the Arrow mimalloc-based allocator
-- ARROW_PARQUET=ON [default=OFF]
-- Build the Parquet libraries
-- ARROW_ORC=OFF [default=OFF]
-- Build the Arrow ORC adapter
-- ARROW_PYTHON=OFF [default=OFF]
-- Build some components needed by PyArrow.
-- (This is a deprecated option. Use CMake presets instead.)
-- ARROW_S3=ON [default=OFF]
-- Build Arrow with S3 support (requires the AWS SDK for C++)
-- ARROW_SKYHOOK=OFF [default=OFF]
-- Build the Skyhook libraries
-- ARROW_SUBSTRAIT=ON [default=OFF]
-- Build the Arrow Substrait Consumer Module
-- ARROW_TENSORFLOW=OFF [default=OFF]
-- Build Arrow with TensorFlow support enabled
-- ARROW_TESTING=OFF [default=OFF]
-- Build the Arrow testing libraries
--
-- Thirdparty toolchain options:
--
-- ARROW_DEPENDENCY_SOURCE=BUNDLED [default=AUTO|BUNDLED|SYSTEM|CONDA|VCPKG|BREW]
-- Method to use for acquiring arrow's build dependencies
-- ARROW_VERBOSE_THIRDPARTY_BUILD=OFF [default=OFF]
-- Show output from ExternalProjects rather than just logging to files
-- ARROW_DEPENDENCY_USE_SHARED=OFF [default=ON]
-- Link to shared libraries
-- ARROW_BOOST_USE_SHARED=OFF [default=OFF]
-- Rely on Boost shared libraries where relevant
-- ARROW_BROTLI_USE_SHARED=OFF [default=OFF]
-- Rely on Brotli shared libraries where relevant
-- ARROW_BZ2_USE_SHARED=OFF [default=OFF]
-- Rely on Bz2 shared libraries where relevant
-- ARROW_GFLAGS_USE_SHARED=OFF [default=OFF]
-- Rely on GFlags shared libraries where relevant
-- ARROW_GRPC_USE_SHARED=OFF [default=OFF]
-- Rely on gRPC shared libraries where relevant
-- ARROW_JEMALLOC_USE_SHARED=OFF [default=OFF]
-- Rely on jemalloc shared libraries where relevant
-- ARROW_LLVM_USE_SHARED=OFF [default=OFF]
-- Rely on LLVM shared libraries where relevant
-- ARROW_LZ4_USE_SHARED=OFF [default=OFF]
-- Rely on lz4 shared libraries where relevant
-- ARROW_OPENSSL_USE_SHARED=OFF [default=OFF]
-- Rely on OpenSSL shared libraries where relevant
-- ARROW_PROTOBUF_USE_SHARED=OFF [default=OFF]
-- Rely on Protocol Buffers shared libraries where relevant
-- ARROW_SNAPPY_USE_SHARED=OFF [default=OFF]
-- Rely on snappy shared libraries where relevant
-- ARROW_THRIFT_USE_SHARED=OFF [default=OFF]
-- Rely on thrift shared libraries where relevant
-- ARROW_UTF8PROC_USE_SHARED=OFF [default=OFF]
-- Rely on utf8proc shared libraries where relevant
-- ARROW_ZSTD_USE_SHARED=OFF [default=OFF]
-- Rely on zstd shared libraries where relevant
-- ARROW_USE_GLOG=OFF [default=OFF]
-- Build libraries with glog support for pluggable logging
-- ARROW_WITH_BACKTRACE=ON [default=ON]
-- Build with backtrace support
-- ARROW_WITH_OPENTELEMETRY=OFF [default=OFF]
-- Build libraries with OpenTelemetry support for distributed tracing
-- ARROW_WITH_BROTLI=OFF [default=OFF]
-- Build with Brotli compression
-- ARROW_WITH_BZ2=OFF [default=OFF]
-- Build with BZ2 compression
-- ARROW_WITH_LZ4=OFF [default=OFF]
-- Build with lz4 compression
-- ARROW_WITH_SNAPPY=OFF [default=OFF]
-- Build with Snappy compression
-- ARROW_WITH_ZLIB=ON [default=OFF]
-- Build with zlib compression
-- ARROW_WITH_ZSTD=OFF [default=OFF]
-- Build with zstd compression
-- ARROW_WITH_UCX=OFF [default=OFF]
-- Build with UCX transport for Arrow Flight
-- (only used if ARROW_FLIGHT is ON)
-- ARROW_WITH_UTF8PROC=ON [default=ON]
-- Build with support for Unicode properties using the utf8proc library
-- (only used if ARROW_COMPUTE is ON or ARROW_GANDIVA is ON)
-- ARROW_WITH_RE2=ON [default=ON]
-- Build with support for regular expressions using the re2 library
-- (only used if ARROW_COMPUTE or ARROW_GANDIVA is ON)
--
-- Parquet options:
--
-- PARQUET_MINIMAL_DEPENDENCY=OFF [default=OFF]
-- Depend only on Thirdparty headers to build libparquet.
-- Always OFF if building binaries
-- PARQUET_BUILD_EXECUTABLES=OFF [default=OFF]
-- Build the Parquet executable CLI tools. Requires static libraries to be built.
-- PARQUET_BUILD_EXAMPLES=OFF [default=OFF]
-- Build the Parquet examples. Requires static libraries to be built.
-- PARQUET_REQUIRE_ENCRYPTION=OFF [default=OFF]
-- Build support for encryption. Fail if OpenSSL is not found
--
-- Gandiva options:
--
-- ARROW_GANDIVA_STATIC_LIBSTDCPP=ON [default=OFF]
-- Include -static-libstdc++ -static-libgcc when linking with
-- Gandiva static libraries
-- ARROW_GANDIVA_PC_CXX_FLAGS="" [default=""]
-- Compiler flags to append when pre-compiling Gandiva operations
--
-- Advanced developer options:
--
-- ARROW_EXTRA_ERROR_CONTEXT=OFF [default=OFF]
-- Compile with extra error context (line numbers, code)
-- ARROW_OPTIONAL_INSTALL=OFF [default=OFF]
-- If enabled install ONLY targets that have already been built. Please be
-- advised that if this is enabled 'install' will fail silently on components
-- that have not been built
-- ARROW_GDB_INSTALL_DIR="" [default=""]
-- Use a custom install directory for GDB plugin.
-- In general, you don't need to specify this because the default
-- (CMAKE_INSTALL_FULL_BINDIR on Windows, CMAKE_INSTALL_FULL_LIBDIR otherwise)
-- is reasonable.
-- Outputting build configuration summary to /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/cmake_summary.json
-- Configuring incomplete, errors occurred!
See also "/srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeOutput.log".
See also "/srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeError.log".
[ERROR] Command execution failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
at org.apache.commons.exec.DefaultExecutor.executeInternal (DefaultExecutor.java:404)
at org.apache.commons.exec.DefaultExecutor.execute (DefaultExecutor.java:166)
at org.codehaus.mojo.exec.ExecMojo.executeCommandLine (ExecMojo.java:1000)
at org.codehaus.mojo.exec.ExecMojo.executeCommandLine (ExecMojo.java:947)
at org.codehaus.mojo.exec.ExecMojo.execute (ExecMojo.java:471)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:954)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.045 s
[INFO] Finished at: 2024-04-14T15:09:14+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.1.0:exec (jni-cpp-cmake) on project arrow-java-root: Command execution failed.: Process exited with an error: 1 (Exit v
alue: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
```
Firstly, I suspect that it's a pthread problem or openssl problem or missing LLVM dependency，but after trying a while, it is found that the failure is not caused by them.
Here's the output of CMakeError.log：
```shell
Performing C SOURCE FILE Test CMAKE_HAVE_LIBC_PTHREAD failed with the following output:
Change Dir: /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp
Run Build Command(s):/usr/local/bin/make -f Makefile cmTC_daf7e/fast && /usr/local/bin/make -f CMakeFiles/cmTC_daf7e.dir/build.make CMakeFiles/cmTC_daf7e.dir/build
make[1]: Entering directory `/srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_daf7e.dir/src.c.o
/opt/gcc-9.4.0/bin/gcc-9 -DCMAKE_HAVE_LIBC_PTHREAD -fPIE -std=gnu11 -o CMakeFiles/cmTC_daf7e.dir/src.c.o -c /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp/src.c
Linking C executable cmTC_daf7e
/opt/soft/cmake-3.21.0-linux-x86_64/bin/cmake -E cmake_link_script CMakeFiles/cmTC_daf7e.dir/link.txt --verbose=1
/opt/gcc-9.4.0/bin/gcc-9 CMakeFiles/cmTC_daf7e.dir/src.c.o -o cmTC_daf7e
CMakeFiles/cmTC_daf7e.dir/src.c.o: In function `main':
src.c:(.text+0x2f): undefined reference to `pthread_create'
src.c:(.text+0x3b): undefined reference to `pthread_detach'
src.c:(.text+0x47): undefined reference to `pthread_cancel'
src.c:(.text+0x58): undefined reference to `pthread_join'
src.c:(.text+0x6c): undefined reference to `pthread_atfork'
collect2: error: ld returned 1 exit status
make[1]: *** [cmTC_daf7e] Error 1
make[1]: Leaving directory `/srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp'
make: *** [cmTC_daf7e/fast] Error 2
Source file was:
#include <pthread.h>
static void* test_func(void* data)
{
return data;
}
int main(void)
{
pthread_t thread;
pthread_create(&thread, NULL, test_func, NULL);
pthread_detach(thread);
pthread_cancel(thread);
pthread_join(thread, NULL);
pthread_atfork(NULL, NULL, NULL);
pthread_exit(NULL);
return 0;
}
Determining if the _M_ARM64 exist failed with the following output:
Change Dir: /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp
Run Build Command(s):/usr/local/bin/make -f Makefile cmTC_faa90/fast && /usr/local/bin/make -f CMakeFiles/cmTC_faa90.dir/build.make CMakeFiles/cmTC_faa90.dir/build
make[1]: Entering directory `/srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_faa90.dir/CheckSymbolExists.cxx.o
/opt/gcc-9.4.0/bin/g++-9 -fdiagnostics-color=always -fPIE -std=c++17 -o CMakeFiles/cmTC_faa90.dir/CheckSymbolExists.cxx.o -c /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx
^[[01m^[[K/srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx:^[[m^[[K In function ‘^[[01m^[[Kint main(int, char**)^[[m^[[K’:
^[[01m^[[K/srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx:7:19:^[[m^[[K ^[[01;31m^[[Kerror: ^[[m^[[K‘^[[01m^[[K_M_ARM64^[[m^[[K’ was not declared in this scope
7 | return ((int*)(&^[[01;31m^[[K_M_ARM64^[[m^[[K))[argc];
| ^[[01;31m^[[K^~~~~~~~^[[m^[[K
make[1]: *** [CMakeFiles/cmTC_faa90.dir/CheckSymbolExists.cxx.o] Error 1
make[1]: Leaving directory `/srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp'
make: *** [cmTC_faa90/fast] Error 2
File /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx:
/* */
int main(int argc, char** argv)
{
(void)argv;
#ifndef _M_ARM64
return ((int*)(&_M_ARM64))[argc];
#else
(void)argc;
return 0;
#endif
}
```
The arrow version I'm using is 14.0.1.rc0, JDK version is 1.8, and compiler is gcc-9.4.0 and g++-9
Can someone help me? thank you very much
### Component(s)
C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Java] Failed when building JNI libraries (*.so on Linux) for arrow-java-dataset, has anyone ever see the same question? [arrow]
Posted by "jaynewchen (via GitHub)" <gi...@apache.org>.
jaynewchen commented on issue #41191:
URL: https://github.com/apache/arrow/issues/41191#issuecomment-2058065049
> Could you try 15.0.2 instead of 14.0.0.1-rc0?
>
> What is your RHEL version?
>
> FYI: Here are Dockerfiles to build our Apache Arrow C++ packages for RHEL8/9:
>
> * For RHEL8: https://github.com/apache/arrow/blob/main/dev/tasks/linux-packages/apache-arrow/yum/almalinux-8/Dockerfile
> * For RHEL9: https://github.com/apache/arrow/blob/main/dev/tasks/linux-packages/apache-arrow/yum/almalinux-9/Dockerfile
My RHEL version is：
```shell
# cat /etc/redhat-release
EulerOS release 2.0 (SP5)
# cat /etc/os-release
NAME="EulerOS"
VERSION="2.0 (SP5)"
ID="euleros"
ID_LIKE="rhel fedora centos"
VERSION_ID="2.0"
PRETTY_NAME="EulerOS 2.0 (SP5)"
ANSI_COLOR="0;31"
```
EulerOS is a self-developed operating system.
Do you mean that I have to build the Arrow-java-dataset JNI code based on the OS image you provide?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Java] Failed when building JNI libraries (*.so on Linux) for arrow-java-dataset, has anyone ever see the same question? [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #41191:
URL: https://github.com/apache/arrow/issues/41191#issuecomment-2054245240
Could you try 15.0.2 instead of 14.0.0.1-rc0?
What is your RHEL version?
FYI: Here are Dockerfiles to build our Apache Arrow C++ packages for RHEL8/9:
* For RHEL8: https://github.com/apache/arrow/blob/main/dev/tasks/linux-packages/apache-arrow/yum/almalinux-8/Dockerfile
* For RHEL9: https://github.com/apache/arrow/blob/main/dev/tasks/linux-packages/apache-arrow/yum/almalinux-9/Dockerfile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Java] Failed when building JNI libraries (*.so on Linux) for arrow-java-dataset, has anyone ever see the same question? [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou closed issue #41191: [Java] Failed when building JNI libraries (*.so on Linux) for arrow-java-dataset, has anyone ever see the same question?
URL: https://github.com/apache/arrow/issues/41191
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Java] Failed when building JNI libraries (*.so on Linux) for arrow-java-dataset, has anyone ever see the same question? [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #41191:
URL: https://github.com/apache/arrow/issues/41191#issuecomment-2058078176
> EulerOS is a self-developed operating system.
OK. I can't help you more because I can't use it.
> Do you mean that I have to build the Arrow-java-dataset JNI code based on the OS image you provide?
No. I mean that you can refer them how to prepare your build environment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Java] Failed when building JNI libraries (*.so on Linux) for arrow-java-dataset, has anyone ever see the same question? [arrow]
Posted by "jaynewchen (via GitHub)" <gi...@apache.org>.
jaynewchen commented on issue #41191:
URL: https://github.com/apache/arrow/issues/41191#issuecomment-2058315097
> > EulerOS is a self-developed operating system.
>
> OK. I can't help you more because I can't use it.
>
> > Do you mean that I have to build the Arrow-java-dataset JNI code based on the OS image you provide?
>
> No. I mean that you can refer them how to prepare your build environment.
thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org