You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by GitBox <gi...@apache.org> on 2023/01/19 14:03:58 UTC
[GitHub] [arrow] raulcd opened a new issue, #33786: [Release][C++] Release verification tasks fail with libxsimd-dev installed on ubuntu 22.04
raulcd opened a new issue, #33786:
URL: https://github.com/apache/arrow/issues/33786
### Describe the bug, including details regarding any error messages, version, and platform.
As pointed during the Release verification for 11.0.0 RC 0 the build failed on Ubuntu 22.04 with:
```
-- Building xsimd from source
CMake Error at cmake_modules/ThirdpartyToolchain.cmake:2295 (add_library):
add_library cannot create imported target "xsimd" because another target
with the same name already exists.
Call Stack (most recent call first):
CMakeLists.txt:498 (include)
```
Full log shared by @pitrou here: https://gist.github.com/pitrou/3fdca2460fa71bba731b0706703b70b2
I have been able to reproduce when installing: `$ sudo apt install libxsimd-dev` on my Ubuntu 22.04.
Mail thread where the issue was raised: https://lists.apache.org/thread/bxkd8xb90pf83mp17xjv3gms46yzyz2q
### Component(s)
C++, Release
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on issue #33786: [Release][C++] Release verification tasks fail with libxsimd-dev installed on ubuntu 22.04
Posted by GitBox <gi...@apache.org>.
assignUser commented on issue #33786:
URL: https://github.com/apache/arrow/issues/33786#issuecomment-1397747484
It was introduced in #13958 to handle the SameMajor version compatibility required by xsimd but the `xsimdConfigVersion.xmake` provided even by xsimd checks for this and will correctly abort `find_package` if the installed version is 9.0.0 and required is 8.1.0.
I have tested this by rewriting `xsimdConfigVersion.cmake` to 9.0.0:
```
CMake Warning at cmake_modules/ThirdpartyToolchain.cmake:261 (find_package):
Could not find a configuration file for package "xsimd" that is compatible
with requested version "8.1.0".
The following configuration files were considered but not accepted:
/usr/lib/x86_64-linux-gnu/cmake/xsimd/xsimdConfig.cmake, version: 9.0.0
/lib/x86_64-linux-gnu/cmake/xsimd/xsimdConfig.cmake, version: 9.0.0
Call Stack (most recent call first):
cmake_modules/ThirdpartyToolchain.cmake:2288 (resolve_dependency)
CMakeLists.txt:498 (include)
```
@kou I will open a PR soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on issue #33786: [Release][C++] Release verification tasks fail with libxsimd-dev installed on ubuntu 22.04
Posted by GitBox <gi...@apache.org>.
assignUser commented on issue #33786:
URL: https://github.com/apache/arrow/issues/33786#issuecomment-1397421912
Ah yes this seems to be the case as 22.04 has xsimd 7.6.0 and we require 8.1.0:
`Unpacking libxsimd-dev:amd64 (7.6.0-2) `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on issue #33786: [Release][C++] Release verification tasks fail with libxsimd-dev installed on ubuntu 22.04
Posted by GitBox <gi...@apache.org>.
assignUser commented on issue #33786:
URL: https://github.com/apache/arrow/issues/33786#issuecomment-1397736892
The problem is caused here:
https://github.com/apache/arrow/blob/f10f5cfd1376fb0e602334588b3f3624d41dee7d/cpp/cmake_modules/ThirdpartyToolchain.cmake#L251-L252
Removing `AND NOT ARG_FORCE_ANY_NEWER_VERSION` fixes this case. Though I don't quite understand the purpose of the arg `FORCE_ANY_NEWER_VERSION` as in the lines above it prevents any version check from happening and a bit further down it is used to double check cmakes default behaviour for version checks (the given version is treated as a minimum version unless `EXACT` is added):
https://github.com/apache/arrow/blob/f10f5cfd1376fb0e602334588b3f3624d41dee7d/cpp/cmake_modules/ThirdpartyToolchain.cmake#L263-L267
I think `FORCE_ANY_NEWER_VERSION` can just be removed completely? I will check when it was added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on issue #33786: [Release][C++] Release verification tasks fail with libxsimd-dev installed on ubuntu 22.04
Posted by GitBox <gi...@apache.org>.
kou commented on issue #33786:
URL: https://github.com/apache/arrow/issues/33786#issuecomment-1397818270
If we need 11.0.0 RC1, we can include a fix of this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on issue #33786: [Release][C++] Release verification tasks fail with libxsimd-dev installed on ubuntu 22.04
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #33786:
URL: https://github.com/apache/arrow/issues/33786#issuecomment-1399160270
We can use `... ARROW_CMAKE_OPTIONS="-Dxsimd_SOURCE=BUNDLED" dev/release/verify-release-candidate.sh ...` instead of changing `verify-release-candidate.sh`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on issue #33786: [Release][C++] Release verification tasks fail with libxsimd-dev installed on ubuntu 22.04
Posted by GitBox <gi...@apache.org>.
assignUser commented on issue #33786:
URL: https://github.com/apache/arrow/issues/33786#issuecomment-1397824572
> BTW, I think that this is not a blocker because we have a -Dxsimd_SOURCE=BUNDLED workaround.
I agree
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on issue #33786: [Release][C++] Release verification tasks fail with libxsimd-dev installed on ubuntu 22.04
Posted by GitBox <gi...@apache.org>.
kou commented on issue #33786:
URL: https://github.com/apache/arrow/issues/33786#issuecomment-1397817897
BTW, I think that this is not a blocker because we have a `-Dxsimd_SOURCE=BUNDLED` workaround.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou closed issue #33786: [Release][C++] Release verification tasks fail with libxsimd-dev installed on ubuntu 22.04
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou closed issue #33786: [Release][C++] Release verification tasks fail with libxsimd-dev installed on ubuntu 22.04
URL: https://github.com/apache/arrow/issues/33786
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on issue #33786: [Release][C++] Release verification tasks fail with libxsimd-dev installed on ubuntu 22.04
Posted by GitBox <gi...@apache.org>.
assignUser commented on issue #33786:
URL: https://github.com/apache/arrow/issues/33786#issuecomment-1397705655
Yes this is exactly what happens:
```
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(2288): resolve_dependency(xsimd REQUIRED_VERSION 8.1.0 FORCE_ANY_NEWER_VERSION TRUE )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(225): set(options )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(226): set(one_value_args FORCE_ANY_NEWER_VERSION HAVE_ALT IS_RUNTIME_DEPENDENCY REQUIRED_VERSION USE_CONFIG )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(232): set(multi_value_args COMPONENTS PC_PACKAGE_NAMES )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(233): cmake_parse_arguments(ARG ${options} ${one_value_args} ${multi_value_args} REQUIRED_VERSION;8.1.0;FORCE_ANY_NEWER_VERSION;TRUE )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(238): if(ARG_UNPARSED_ARGUMENTS )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(241): if(${ARG_IS_RUNTIME_DEPENDENCY} STREQUAL )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(242): set(ARG_IS_RUNTIME_DEPENDENCY TRUE )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(245): if(ARG_HAVE_ALT )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(247): else()
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(248): set(PACKAGE_NAME xsimd )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(250): set(FIND_PACKAGE_ARGUMENTS ${PACKAGE_NAME} )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(251): if(ARG_REQUIRED_VERSION AND NOT ARG_FORCE_ANY_NEWER_VERSION )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(254): if(ARG_USE_CONFIG )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(257): if(ARG_COMPONENTS )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(260): if(xsimd_SOURCE STREQUAL AUTO )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(261): find_package(${FIND_PACKAGE_ARGUMENTS} )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(262): set(COMPATIBLE ${${PACKAGE_NAME}_FOUND} )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(263): if(COMPATIBLE AND ARG_FORCE_ANY_NEWER_VERSION AND ARG_REQUIRED_VERSION )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(266): if(${${PACKAGE_NAME}_VERSION} VERSION_LESS ${ARG_REQUIRED_VERSION} )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(267): message(DEBUG Couldn't find xsimd >= ${ARG_REQUIRED_VERSION} )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(268): set(COMPATIBLE FALSE )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(271): if(COMPATIBLE )
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(273): else()
/arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake(274): build_dependency(xsimd )
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on issue #33786: [Release][C++] Release verification tasks fail with libxsimd-dev installed on ubuntu 22.04
Posted by GitBox <gi...@apache.org>.
kou commented on issue #33786:
URL: https://github.com/apache/arrow/issues/33786#issuecomment-1397817009
We can just reuse the existing `xsimd` target:
```diff
diff --git a/cpp/cmake_modules/ThirdpartyToolchain.cmake b/cpp/cmake_modules/ThirdpartyToolchain.cmake
index 3eda538fb2..b03ebbbf11 100644
--- a/cpp/cmake_modules/ThirdpartyToolchain.cmake
+++ b/cpp/cmake_modules/ThirdpartyToolchain.cmake
@@ -2292,7 +2292,10 @@ if(ARROW_USE_XSIMD)
TRUE)
if(xsimd_SOURCE STREQUAL "BUNDLED")
- add_library(xsimd INTERFACE IMPORTED)
+ # Reuse xsimd target defined by find_package(xsimd)
+ if(NOT TARGET xsimd)
+ add_library(xsimd INTERFACE IMPORTED)
+ endif()
if(CMAKE_VERSION VERSION_LESS 3.11)
set_target_properties(xsimd PROPERTIES INTERFACE_INCLUDE_DIRECTORIES
"${XSIMD_INCLUDE_DIR}")
```
Or we can use other target name like we did for ProtoBuf:
```diff
diff --git a/cpp/CMakeLists.txt b/cpp/CMakeLists.txt
index 8398a394e7..b193ea80d4 100644
--- a/cpp/CMakeLists.txt
+++ b/cpp/CMakeLists.txt
@@ -798,8 +798,8 @@ if(ARROW_WITH_RAPIDJSON)
endif()
if(ARROW_USE_XSIMD)
- list(APPEND ARROW_SHARED_LINK_LIBS xsimd)
- list(APPEND ARROW_STATIC_LINK_LIBS xsimd)
+ list(APPEND ARROW_SHARED_LINK_LIBS ${ARROW_XSIMD})
+ list(APPEND ARROW_STATIC_LINK_LIBS ${ARROW_XSIMD})
endif()
add_custom_target(arrow_dependencies)
diff --git a/cpp/cmake_modules/ThirdpartyToolchain.cmake b/cpp/cmake_modules/ThirdpartyToolchain.cmake
index 3eda538fb2..3480baae2d 100644
--- a/cpp/cmake_modules/ThirdpartyToolchain.cmake
+++ b/cpp/cmake_modules/ThirdpartyToolchain.cmake
@@ -2292,15 +2292,17 @@ if(ARROW_USE_XSIMD)
TRUE)
if(xsimd_SOURCE STREQUAL "BUNDLED")
- add_library(xsimd INTERFACE IMPORTED)
+ add_library(arrow::xsimd INTERFACE IMPORTED)
if(CMAKE_VERSION VERSION_LESS 3.11)
- set_target_properties(xsimd PROPERTIES INTERFACE_INCLUDE_DIRECTORIES
- "${XSIMD_INCLUDE_DIR}")
+ set_target_properties(arrow::xsimd PROPERTIES INTERFACE_INCLUDE_DIRECTORIES
+ "${XSIMD_INCLUDE_DIR}")
else()
- target_include_directories(xsimd INTERFACE "${XSIMD_INCLUDE_DIR}")
+ target_include_directories(arrow::xsimd INTERFACE "${XSIMD_INCLUDE_DIR}")
endif()
+ set(ARROW_XSIMD arrow::xsimd)
else()
message(STATUS "xsimd found. Headers: ${xsimd_INCLUDE_DIRS}")
+ set(ARROW_XSIMD xsimd)
endif()
endif()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on issue #33786: [Release][C++] Release verification tasks fail with libxsimd-dev installed on ubuntu 22.04
Posted by GitBox <gi...@apache.org>.
assignUser commented on issue #33786:
URL: https://github.com/apache/arrow/issues/33786#issuecomment-1397755258
> to handle the SameMajor version compatibility required by xsimd
Oh I misunderstood the intention, `FORCE_ANY_NEWER_VERSION` is meant to circumvent the SameMajor checks that xsimd does to also make 9.0.0 work for us even if (as seen above) the default check makes it fail.
It might be possible to override the system provided `xsimdConfigVersion.cmake` by adding a custom one earlier on the search path. I will test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on issue #33786: [Release][C++] Release verification tasks fail with libxsimd-dev installed on ubuntu 22.04
Posted by GitBox <gi...@apache.org>.
raulcd commented on issue #33786:
URL: https://github.com/apache/arrow/issues/33786#issuecomment-1398403397
For completeness I've run the verification scripts locally on Ubuntu 22.04 with `libxsimd-dev` installed and was able to build using the proposed workaround:
```
$ git diff
diff --git a/dev/release/verify-release-candidate.sh b/dev/release/verify-release-candidate.sh
index 9e044d2..a659d00 100755
--- a/dev/release/verify-release-candidate.sh
+++ b/dev/release/verify-release-candidate.sh
@@ -643,6 +643,7 @@ test_and_install_cpp() {
-DPARQUET_BUILD_EXAMPLES=ON \
-DPARQUET_BUILD_EXECUTABLES=ON \
-DPARQUET_REQUIRE_ENCRYPTION=ON \
+ -Dxsimd_SOURCE=BUNDLED \
${ARROW_CMAKE_OPTIONS:-} \
${ARROW_SOURCE_DIR}/cpp
export CMAKE_BUILD_PARALLEL_LEVEL=${CMAKE_BUILD_PARALLEL_LEVEL:-${NPROC}}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on issue #33786: [Release][C++] Release verification tasks fail with libxsimd-dev installed on ubuntu 22.04
Posted by GitBox <gi...@apache.org>.
assignUser commented on issue #33786:
URL: https://github.com/apache/arrow/issues/33786#issuecomment-1397414258
resolve_dependency clearly does not correctly detect the system install (maybe discards it due to version?) and falls back on source build. So it is ossible that `Findxsimd.cmkae` was run, targets were created but `resolve_dependecy` discarded it causing the conflict?
I will investigate
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org