You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/06/03 13:37:52 UTC
[GitHub] [arrow] AlenkaF opened a new pull request, #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
AlenkaF opened a new pull request, #13311:
URL: https://github.com/apache/arrow/pull/13311
This PR moves `src/arrow/python` directory into `pyarrow` and arranges PyArrow to build it.
**TODO**
- [ ] Move the files and change cmake for C PyArrow
- [ ] Integrate C PyArrow build into setup.py
- [ ] How to run the tests for C PyArrow
- [ ] Make changes to the C++ build (what is not needed anymore)
- [ ] Review proposed changes for CMake options for ARROW_PYTHON
- [ ] Add a migration path for external packages
**Testing**
Until https://github.com/apache/arrow/pull/13282 is merged one additional step is needed to successfully build pyarrow. Needed header files from Arrow must be copied into `dist` folder:
```
cp -r cpp/src/arrow/vendored/xxhash/ dist/include/arrow/vendored/xxhash/
cp cpp/src/arrow/util/int_util_internal.h dist/include/arrow/util
cp -r cpp/src/arrow/vendored/portable-snippets/ dist/include/arrow/vendored/portable-snippets/
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1184044278
@github-actions crossbow submit *python*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1189986162
Revision: cc7e8c850b6aa600c80e0e3eebffab9f39655452
Submitted crossbow builds: [ursacomputing/crossbow @ actions-578f9b2474](https://github.com/ursacomputing/crossbow/branches/all?query=actions-578f9b2474)
|Task|Status|
|----|------|
|verify-rc-binaries-wheels-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-verify-rc-binaries-wheels-linux-almalinux-8-amd64)|
|verify-rc-binaries-wheels-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-verify-rc-binaries-wheels-linux-conda-latest-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64)|
|verify-rc-binaries-wheels-macos-10.15-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-verify-rc-binaries-wheels-macos-10.15-amd64)|
|verify-rc-binaries-wheels-macos-10.15-amd64-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-verify-rc-binaries-wheels-macos-10.15-amd64-conda)|
|verify-rc-binaries-wheels-macos-11-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-verify-rc-binaries-wheels-macos-11-amd64)|
|verify-rc-binaries-wheels-macos-11-amd64-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-verify-rc-binaries-wheels-macos-11-amd64-conda)|
|verify-rc-binaries-wheels-macos-11-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-verify-rc-binaries-wheels-macos-11-arm64)|
|verify-rc-binaries-wheels-windows|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-verify-rc-binaries-wheels-windows)|
|wheel-macos-big-sur-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-macos-big-sur-cp310-arm64)|
|wheel-macos-big-sur-cp310-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-macos-big-sur-cp310-universal2)|
|wheel-macos-big-sur-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-macos-big-sur-cp38-arm64)|
|wheel-macos-big-sur-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-macos-big-sur-cp39-arm64)|
|wheel-macos-big-sur-cp39-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-macos-big-sur-cp39-universal2)|
|wheel-macos-high-sierra-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-macos-high-sierra-cp310-amd64)|
|wheel-macos-high-sierra-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-macos-high-sierra-cp37-amd64)|
|wheel-macos-high-sierra-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-macos-high-sierra-cp38-amd64)|
|wheel-macos-high-sierra-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-macos-high-sierra-cp39-amd64)|
|wheel-macos-mavericks-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-macos-mavericks-cp310-amd64)|
|wheel-macos-mavericks-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-macos-mavericks-cp37-amd64)|
|wheel-macos-mavericks-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-macos-mavericks-cp38-amd64)|
|wheel-macos-mavericks-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-macos-mavericks-cp39-amd64)|
|wheel-manylinux2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-manylinux2014-cp310-amd64)|
|wheel-manylinux2014-cp310-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-manylinux2014-cp37-amd64)|
|wheel-manylinux2014-cp37-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-manylinux2014-cp38-amd64)|
|wheel-manylinux2014-cp38-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-manylinux2014-cp39-amd64)|
|wheel-manylinux2014-cp39-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-windows-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-windows-cp310-amd64)|
|wheel-windows-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-windows-cp37-amd64)|
|wheel-windows-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-windows-cp38-amd64)|
|wheel-windows-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-578f9b2474-github-wheel-windows-cp39-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r922958194
##########
cpp/cmake_modules/FindArrowPython.cmake:
##########
@@ -46,8 +46,9 @@ endif()
find_package(Arrow ${find_package_arguments})
if(ARROW_FOUND)
+ message(STATUS "CPYARROW_HOME: ${CPYARROW_HOME}")
arrow_find_package(ARROW_PYTHON
- "${ARROW_HOME}"
+ "${CPYARROW_HOME}"
Review Comment:
OK, will rename to `PYARROW_CPP_HOME` and will do the same other instances of arrow python/ C pyarrow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r922972234
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+include(SetupCxxFlags)
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
Review Comment:
I added `PYARROW_WITH_PARQUET_ENCRYPTION` in this commit: https://github.com/apache/arrow/pull/13311/commits/c56fbe222b1f5fd2d40410c4e522a4fb114efbcd because there is a case where we build with `PARQUET_REQUIRE_ENCRYPTION=on` but `PYARROW_WITH_PARQUET=off` and so `PYARROW_WITH_PARQUET_ENCRYPTION` is set to off due to Parquet not being built and not loaded:
https://github.com/apache/arrow/blob/385fe4315b2ccefc90860cee18a3661f67b57c3e/python/pyarrow/src_arrow/CMakeLists.txt#L117-L124
See failing build:
https://github.com/apache/arrow/runs/7229978593?check_suite_focus=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1191393326
@github-actions crossbow submit *wheel-manylinux*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1191834000
@github-actions crossbow submit *python*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [C++][Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1228113676
Will also make a JIRA for centos-9-stream-amd64 failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #13311: ARROW-16340: [C++][Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1228304381
Benchmark runs are scheduled for baseline = 7e7b8e1f0a2f02046e0d0d6a0b9ada6d64648b9c and contender = b832853ba62171d5fe5077681083fc6ea49bfd44. b832853ba62171d5fe5077681083fc6ea49bfd44 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/d784bfd21b5041e985391400daf42a01...ff8f29c842654cf3a7eea19d1203d6f8/)
[Failed] [test-mac-arm](https://conbench.ursa.dev/compare/runs/423a737ed5584499979999025a605642...e5b27e48858e43efbec2b413025f68c6/)
[Failed :arrow_down:6.03% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/b51831c1b58a4de7a872dc36d59d7a91...65dba3eeb9494382b7d4e0ef9ece4d7d/)
[Finished :arrow_down:0.25% :arrow_up:0.39%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/9e4392db1bf349a38c78436b86cef3b0...057f954b936a4b1cbc3ab818d05052f4/)
Buildkite builds:
[Finished] [`b832853b` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/1373)
[Failed] [`b832853b` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/1389)
[Failed] [`b832853b` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/1371)
[Finished] [`b832853b` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/1388)
[Finished] [`7e7b8e1f` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/1372)
[Finished] [`7e7b8e1f` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/1388)
[Failed] [`7e7b8e1f` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/1370)
[Finished] [`7e7b8e1f` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/1387)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1225206974
@github-actions crossbow submit -g almalinux -g centos
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1225551843
Revision: e58c7fe68a9a72a6c647d11021b07be6696e9bbc
Submitted crossbow builds: [ursacomputing/crossbow @ actions-011b454722](https://github.com/ursacomputing/crossbow/branches/all?query=actions-011b454722)
|Task|Status|
|----|------|
|example-python-minimal-build-ubuntu-venv|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-example-python-minimal-build-ubuntu-venv)|
|test-r-rhub-ubuntu-gcc-release-latest|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-011b454722-azure-test-r-rhub-ubuntu-gcc-release-latest)|
|test-r-ubuntu-22.04|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-test-r-ubuntu-22.04)|
|test-ubuntu-18.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-test-ubuntu-18.04-cpp)|
|test-ubuntu-18.04-cpp-release|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-test-ubuntu-18.04-cpp-release)|
|test-ubuntu-18.04-cpp-static|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-test-ubuntu-18.04-cpp-static)|
|test-ubuntu-18.04-r-sanitizer|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-011b454722-azure-test-ubuntu-18.04-r-sanitizer)|
|test-ubuntu-20.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-test-ubuntu-20.04-cpp)|
|test-ubuntu-20.04-cpp-14|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-test-ubuntu-20.04-cpp-14)|
|test-ubuntu-20.04-cpp-17|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-test-ubuntu-20.04-cpp-17)|
|test-ubuntu-20.04-cpp-bundled|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-test-ubuntu-20.04-cpp-bundled)|
|test-ubuntu-20.04-cpp-thread-sanitizer|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-test-ubuntu-20.04-cpp-thread-sanitizer)|
|test-ubuntu-20.04-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-011b454722-azure-test-ubuntu-20.04-python-3)|
|test-ubuntu-22.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-test-ubuntu-22.04-cpp)|
|test-ubuntu-c-glib|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-test-ubuntu-c-glib)|
|test-ubuntu-default-docs|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-011b454722-azure-test-ubuntu-default-docs)|
|test-ubuntu-ruby|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-test-ubuntu-ruby)|
|ubuntu-bionic-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-ubuntu-bionic-amd64)|
|ubuntu-bionic-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|ubuntu-focal-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-ubuntu-focal-amd64)|
|ubuntu-focal-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|ubuntu-jammy-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-ubuntu-jammy-amd64)|
|ubuntu-jammy-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|verify-rc-binaries-jars-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-binaries-jars-linux-ubuntu-18.04-amd64)|
|verify-rc-binaries-jars-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-binaries-jars-linux-ubuntu-20.04-amd64)|
|verify-rc-binaries-jars-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-binaries-jars-linux-ubuntu-22.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64)|
|verify-rc-source-cpp-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-cpp-linux-ubuntu-18.04-amd64)|
|verify-rc-source-cpp-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-cpp-linux-ubuntu-20.04-amd64)|
|verify-rc-source-cpp-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-cpp-linux-ubuntu-22.04-amd64)|
|verify-rc-source-csharp-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-csharp-linux-ubuntu-18.04-amd64)|
|verify-rc-source-csharp-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-csharp-linux-ubuntu-20.04-amd64)|
|verify-rc-source-csharp-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-csharp-linux-ubuntu-22.04-amd64)|
|verify-rc-source-go-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-go-linux-ubuntu-18.04-amd64)|
|verify-rc-source-go-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-go-linux-ubuntu-20.04-amd64)|
|verify-rc-source-go-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-go-linux-ubuntu-22.04-amd64)|
|verify-rc-source-integration-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-integration-linux-ubuntu-18.04-amd64)|
|verify-rc-source-integration-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-integration-linux-ubuntu-20.04-amd64)|
|verify-rc-source-integration-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-integration-linux-ubuntu-22.04-amd64)|
|verify-rc-source-java-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-java-linux-ubuntu-18.04-amd64)|
|verify-rc-source-java-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-java-linux-ubuntu-20.04-amd64)|
|verify-rc-source-java-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-java-linux-ubuntu-22.04-amd64)|
|verify-rc-source-js-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-js-linux-ubuntu-18.04-amd64)|
|verify-rc-source-js-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-js-linux-ubuntu-20.04-amd64)|
|verify-rc-source-js-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-js-linux-ubuntu-22.04-amd64)|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
|verify-rc-source-python-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-python-linux-ubuntu-20.04-amd64)|
|verify-rc-source-python-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-python-linux-ubuntu-22.04-amd64)|
|verify-rc-source-ruby-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-ruby-linux-ubuntu-18.04-amd64)|
|verify-rc-source-ruby-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-ruby-linux-ubuntu-20.04-amd64)|
|verify-rc-source-ruby-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-011b454722-github-verify-rc-source-ruby-linux-ubuntu-22.04-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1225672001
Revision: 153a1cb3c30d8774dde4f0da9d805e9ac0724854
Submitted crossbow builds: [ursacomputing/crossbow @ actions-ad6ef3d939](https://github.com/ursacomputing/crossbow/branches/all?query=actions-ad6ef3d939)
|Task|Status|
|----|------|
|ubuntu-bionic-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-ad6ef3d939-github-ubuntu-bionic-amd64)|
|ubuntu-bionic-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|ubuntu-focal-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-ad6ef3d939-github-ubuntu-focal-amd64)|
|ubuntu-focal-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|ubuntu-jammy-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-ad6ef3d939-github-ubuntu-jammy-amd64)|
|ubuntu-jammy-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1227079015
Revision: 0860306fb2d205cc9d36b83f71e09f5b882741cc
Submitted crossbow builds: [ursacomputing/crossbow @ actions-58857300ac](https://github.com/ursacomputing/crossbow/branches/all?query=actions-58857300ac)
|Task|Status|
|----|------|
|almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-almalinux-8-amd64)|
|almalinux-8-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|almalinux-9-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-almalinux-9-amd64)|
|almalinux-9-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|amazon-linux-2-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-amazon-linux-2-amd64)|
|amazon-linux-2-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|centos-7-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-centos-7-amd64)|
|centos-8-stream-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-centos-8-stream-amd64)|
|centos-8-stream-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|centos-9-stream-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-centos-9-stream-amd64)|
|centos-9-stream-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|conan-maximum|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-conan-maximum)|
|conan-minimum|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-conan-minimum)|
|conda-clean|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-clean)|
|conda-linux-gcc-py310-arm64|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py310-arm64)|
|conda-linux-gcc-py310-cpu|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py310-cpu)|
|conda-linux-gcc-py310-cuda|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py310-cuda)|
|conda-linux-gcc-py310-ppc64le|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py310-ppc64le)|
|conda-linux-gcc-py37-arm64|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py37-arm64)|
|conda-linux-gcc-py37-cpu-r40|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py37-cpu-r40)|
|conda-linux-gcc-py37-cpu-r41|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py37-cpu-r41)|
|conda-linux-gcc-py37-cuda|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py37-cuda)|
|conda-linux-gcc-py37-ppc64le|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py37-ppc64le)|
|conda-linux-gcc-py38-arm64|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py38-arm64)|
|conda-linux-gcc-py38-cpu|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py38-cpu)|
|conda-linux-gcc-py38-cuda|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py38-cuda)|
|conda-linux-gcc-py38-ppc64le|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py38-ppc64le)|
|conda-linux-gcc-py39-arm64|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py39-arm64)|
|conda-linux-gcc-py39-cpu|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py39-cpu)|
|conda-linux-gcc-py39-cuda|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py39-cuda)|
|conda-linux-gcc-py39-ppc64le|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-linux-gcc-py39-ppc64le)|
|conda-osx-arm64-clang-py310|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-osx-arm64-clang-py310)|
|conda-osx-arm64-clang-py38|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-osx-arm64-clang-py38)|
|conda-osx-arm64-clang-py39|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-osx-arm64-clang-py39)|
|conda-osx-clang-py310|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-osx-clang-py310)|
|conda-osx-clang-py37-r40|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-osx-clang-py37-r40)|
|conda-osx-clang-py37-r41|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-osx-clang-py37-r41)|
|conda-osx-clang-py38|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-osx-clang-py38)|
|conda-osx-clang-py39|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-osx-clang-py39)|
|conda-win-vs2017-py310|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-win-vs2017-py310)|
|conda-win-vs2017-py37-r40|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-win-vs2017-py37-r40)|
|conda-win-vs2017-py37-r41|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-win-vs2017-py37-r41)|
|conda-win-vs2017-py38|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-win-vs2017-py38)|
|conda-win-vs2017-py39|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-conda-win-vs2017-py39)|
|debian-bookworm-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-debian-bookworm-amd64)|
|debian-bookworm-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|debian-bullseye-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-debian-bullseye-amd64)|
|debian-bullseye-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|example-cpp-minimal-build-static|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-example-cpp-minimal-build-static)|
|example-cpp-minimal-build-static-system-dependency|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-example-cpp-minimal-build-static-system-dependency)|
|example-python-minimal-build-fedora-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-example-python-minimal-build-fedora-conda)|
|example-python-minimal-build-ubuntu-venv|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-example-python-minimal-build-ubuntu-venv)|
|homebrew-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-homebrew-cpp)|
|homebrew-r-autobrew|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-homebrew-r-autobrew)|
|homebrew-r-brew|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-homebrew-r-brew)|
|java-jars|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-java-jars)|
|nuget|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-nuget)|
|python-sdist|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-python-sdist)|
|r-binary-packages|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-r-binary-packages)|
|test-alpine-linux-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-alpine-linux-cpp)|
|test-build-cpp-fuzz|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-build-cpp-fuzz)|
|test-build-vcpkg-win|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-build-vcpkg-win)|
|test-conda-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-cpp)|
|test-conda-cpp-valgrind|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-conda-cpp-valgrind)|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.10)|
|test-conda-python-3.7|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.7)|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.7-hdfs-3.2.1)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-0.24|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.7-pandas-0.24)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-spark-v3.1.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.7-spark-v3.1.2)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.8)|
|test-conda-python-3.8-hypothesis|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.8-hypothesis)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.8-pandas-nightly)|
|test-conda-python-3.8-spark-v3.2.0|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.8-spark-v3.2.0)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.9)|
|test-conda-python-3.9-dask-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.9-dask-latest)|
|test-conda-python-3.9-dask-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.9-dask-master)|
|test-conda-python-3.9-pandas-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.9-pandas-master)|
|test-conda-python-3.9-spark-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-conda-python-3.9-spark-master)|
|test-debian-10-cpp-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-debian-10-cpp-amd64)|
|test-debian-10-cpp-i386|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-debian-10-cpp-i386)|
|test-debian-11-cpp-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-debian-11-cpp-amd64)|
|test-debian-11-cpp-i386|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-debian-11-cpp-i386)|
|test-debian-11-go-1.16|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-debian-11-go-1.16)|
|test-debian-11-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-debian-11-python-3)|
|test-debian-c-glib|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-debian-c-glib)|
|test-debian-ruby|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-debian-ruby)|
|test-fedora-35-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-fedora-35-cpp)|
|test-fedora-35-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-fedora-35-python-3)|
|test-fedora-r-clang-sanitizer|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-fedora-r-clang-sanitizer)|
|test-r-arrow-backwards-compatibility|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-r-arrow-backwards-compatibility)|
|test-r-depsource-bundled|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-r-depsource-bundled)|
|test-r-depsource-system|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-r-depsource-system)|
|test-r-dev-duckdb|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-r-dev-duckdb)|
|test-r-devdocs|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-r-devdocs)|
|test-r-gcc-11|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-r-gcc-11)|
|test-r-gcc-12|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-r-gcc-12)|
|test-r-install-local|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-r-install-local)|
|test-r-linux-as-cran|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-r-linux-as-cran)|
|test-r-linux-rchk|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-r-linux-rchk)|
|test-r-linux-valgrind|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-r-linux-valgrind)|
|test-r-minimal-build|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-r-minimal-build)|
|test-r-offline-maximal|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-r-offline-maximal)|
|test-r-offline-minimal|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-r-offline-minimal)|
|test-r-rhub-debian-gcc-devel-lto-latest|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-r-rhub-debian-gcc-devel-lto-latest)|
|test-r-rhub-debian-gcc-release-custom-ccache|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-r-rhub-debian-gcc-release-custom-ccache)|
|test-r-rhub-ubuntu-gcc-release-latest|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-r-rhub-ubuntu-gcc-release-latest)|
|test-r-rocker-r-base-latest|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-r-rocker-r-base-latest)|
|test-r-rstudio-r-base-4.1-opensuse153|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-r-rstudio-r-base-4.1-opensuse153)|
|test-r-rstudio-r-base-4.2-centos7-devtoolset-8|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-r-rstudio-r-base-4.2-centos7-devtoolset-8)|
|test-r-rstudio-r-base-4.2-focal|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-r-rstudio-r-base-4.2-focal)|
|test-r-ubuntu-22.04|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-r-ubuntu-22.04)|
|test-r-versions|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-r-versions)|
|test-skyhook-integration|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-skyhook-integration)|
|test-ubuntu-18.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-ubuntu-18.04-cpp)|
|test-ubuntu-18.04-cpp-release|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-ubuntu-18.04-cpp-release)|
|test-ubuntu-18.04-cpp-static|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-ubuntu-18.04-cpp-static)|
|test-ubuntu-18.04-r-sanitizer|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-ubuntu-18.04-r-sanitizer)|
|test-ubuntu-20.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-ubuntu-20.04-cpp)|
|test-ubuntu-20.04-cpp-14|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-ubuntu-20.04-cpp-14)|
|test-ubuntu-20.04-cpp-17|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-ubuntu-20.04-cpp-17)|
|test-ubuntu-20.04-cpp-bundled|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-ubuntu-20.04-cpp-bundled)|
|test-ubuntu-20.04-cpp-thread-sanitizer|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-ubuntu-20.04-cpp-thread-sanitizer)|
|test-ubuntu-20.04-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-ubuntu-20.04-python-3)|
|test-ubuntu-22.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-ubuntu-22.04-cpp)|
|test-ubuntu-c-glib|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-ubuntu-c-glib)|
|test-ubuntu-default-docs|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-58857300ac-azure-test-ubuntu-default-docs)|
|test-ubuntu-ruby|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-test-ubuntu-ruby)|
|ubuntu-bionic-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-ubuntu-bionic-amd64)|
|ubuntu-bionic-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|ubuntu-focal-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-ubuntu-focal-amd64)|
|ubuntu-focal-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|ubuntu-jammy-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-ubuntu-jammy-amd64)|
|ubuntu-jammy-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|verify-rc-source-cpp-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-cpp-linux-almalinux-8-amd64)|
|verify-rc-source-cpp-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-cpp-linux-conda-latest-amd64)|
|verify-rc-source-cpp-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-cpp-linux-ubuntu-18.04-amd64)|
|verify-rc-source-cpp-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-cpp-linux-ubuntu-20.04-amd64)|
|verify-rc-source-cpp-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-cpp-linux-ubuntu-22.04-amd64)|
|verify-rc-source-cpp-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-cpp-macos-amd64)|
|verify-rc-source-cpp-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-cpp-macos-arm64)|
|verify-rc-source-cpp-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-cpp-macos-conda-amd64)|
|verify-rc-source-csharp-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-csharp-linux-almalinux-8-amd64)|
|verify-rc-source-csharp-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-csharp-linux-conda-latest-amd64)|
|verify-rc-source-csharp-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-csharp-linux-ubuntu-18.04-amd64)|
|verify-rc-source-csharp-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-csharp-linux-ubuntu-20.04-amd64)|
|verify-rc-source-csharp-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-csharp-linux-ubuntu-22.04-amd64)|
|verify-rc-source-csharp-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-csharp-macos-amd64)|
|verify-rc-source-csharp-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-csharp-macos-arm64)|
|verify-rc-source-go-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-go-linux-almalinux-8-amd64)|
|verify-rc-source-go-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-go-linux-conda-latest-amd64)|
|verify-rc-source-go-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-go-linux-ubuntu-18.04-amd64)|
|verify-rc-source-go-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-go-linux-ubuntu-20.04-amd64)|
|verify-rc-source-go-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-go-linux-ubuntu-22.04-amd64)|
|verify-rc-source-go-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-go-macos-amd64)|
|verify-rc-source-go-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-go-macos-arm64)|
|verify-rc-source-integration-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-integration-linux-almalinux-8-amd64)|
|verify-rc-source-integration-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-integration-linux-conda-latest-amd64)|
|verify-rc-source-integration-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-integration-linux-ubuntu-18.04-amd64)|
|verify-rc-source-integration-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-integration-linux-ubuntu-20.04-amd64)|
|verify-rc-source-integration-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-integration-linux-ubuntu-22.04-amd64)|
|verify-rc-source-integration-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-integration-macos-amd64)|
|verify-rc-source-integration-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-integration-macos-arm64)|
|verify-rc-source-integration-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-integration-macos-conda-amd64)|
|verify-rc-source-java-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-java-linux-almalinux-8-amd64)|
|verify-rc-source-java-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-java-linux-conda-latest-amd64)|
|verify-rc-source-java-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-java-linux-ubuntu-18.04-amd64)|
|verify-rc-source-java-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-java-linux-ubuntu-20.04-amd64)|
|verify-rc-source-java-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-java-linux-ubuntu-22.04-amd64)|
|verify-rc-source-java-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-java-macos-amd64)|
|verify-rc-source-js-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-js-linux-almalinux-8-amd64)|
|verify-rc-source-js-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-js-linux-conda-latest-amd64)|
|verify-rc-source-js-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-js-linux-ubuntu-18.04-amd64)|
|verify-rc-source-js-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-js-linux-ubuntu-20.04-amd64)|
|verify-rc-source-js-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-js-linux-ubuntu-22.04-amd64)|
|verify-rc-source-js-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-js-macos-amd64)|
|verify-rc-source-js-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-js-macos-arm64)|
|verify-rc-source-python-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-python-linux-almalinux-8-amd64)|
|verify-rc-source-python-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-python-linux-conda-latest-amd64)|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
|verify-rc-source-python-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-python-linux-ubuntu-20.04-amd64)|
|verify-rc-source-python-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-python-linux-ubuntu-22.04-amd64)|
|verify-rc-source-python-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-python-macos-amd64)|
|verify-rc-source-python-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-python-macos-arm64)|
|verify-rc-source-python-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-python-macos-conda-amd64)|
|verify-rc-source-ruby-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-ruby-linux-almalinux-8-amd64)|
|verify-rc-source-ruby-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-ruby-linux-conda-latest-amd64)|
|verify-rc-source-ruby-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-ruby-linux-ubuntu-18.04-amd64)|
|verify-rc-source-ruby-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-ruby-linux-ubuntu-20.04-amd64)|
|verify-rc-source-ruby-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-ruby-linux-ubuntu-22.04-amd64)|
|verify-rc-source-ruby-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-ruby-macos-amd64)|
|verify-rc-source-ruby-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-ruby-macos-arm64)|
|verify-rc-source-windows|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-verify-rc-source-windows)|
|wheel-macos-big-sur-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-macos-big-sur-cp310-arm64)|
|wheel-macos-big-sur-cp310-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-macos-big-sur-cp310-universal2)|
|wheel-macos-big-sur-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-macos-big-sur-cp38-arm64)|
|wheel-macos-big-sur-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-macos-big-sur-cp39-arm64)|
|wheel-macos-big-sur-cp39-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-macos-big-sur-cp39-universal2)|
|wheel-macos-high-sierra-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-macos-high-sierra-cp310-amd64)|
|wheel-macos-high-sierra-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-macos-high-sierra-cp37-amd64)|
|wheel-macos-high-sierra-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-macos-high-sierra-cp38-amd64)|
|wheel-macos-high-sierra-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-macos-high-sierra-cp39-amd64)|
|wheel-macos-mavericks-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-macos-mavericks-cp310-amd64)|
|wheel-macos-mavericks-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-macos-mavericks-cp37-amd64)|
|wheel-macos-mavericks-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-macos-mavericks-cp38-amd64)|
|wheel-macos-mavericks-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-macos-mavericks-cp39-amd64)|
|wheel-manylinux2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-manylinux2014-cp310-amd64)|
|wheel-manylinux2014-cp310-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-manylinux2014-cp37-amd64)|
|wheel-manylinux2014-cp37-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-manylinux2014-cp38-amd64)|
|wheel-manylinux2014-cp38-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-manylinux2014-cp39-amd64)|
|wheel-manylinux2014-cp39-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-windows-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-windows-cp310-amd64)|
|wheel-windows-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-windows-cp37-amd64)|
|wheel-windows-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-windows-cp38-amd64)|
|wheel-windows-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-58857300ac-github-wheel-windows-cp39-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1169577278
We need `find_package(Parquet REQUIRED)` for `parquet_shared`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
lidavidm commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1173758670
Hmm, looks like we need to add a factory method for `SchemaResult`. There isn't anything available already
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1191170099
@github-actions crossbow submit *wheel*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926294396
##########
python/setup.py:
##########
@@ -401,15 +528,10 @@ def append_cmake_bool(value, varname):
def _bundle_arrow_cpp(self, build_prefix, build_lib):
print(pjoin(build_lib, 'pyarrow'))
move_shared_libs(build_prefix, build_lib, "arrow")
- move_shared_libs(build_prefix, build_lib, "arrow_python")
if self.with_cuda:
move_shared_libs(build_prefix, build_lib, "arrow_cuda")
if self.with_substrait:
move_shared_libs(build_prefix, build_lib, "arrow_substrait")
- if self.with_flight:
- move_shared_libs(build_prefix, build_lib, "arrow_flight")
Review Comment:
Oh, that's a mistake (don't remember correctly but I think I have mistaken it with Python Flight module that wouldn't need to be moved in this part)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r911759603
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,467 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+
+# normalize ARROW_HOME path
+cmake_path(CONVERT "$ENV{ARROW_HOME}" TO_CMAKE_PATH_LIST ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+find_package(Python3Alt 3.7 REQUIRED)
+
+# Needed due to
+# CMake Error: INSTALL(EXPORT) given unknown export "arrow_python_targets"
+option(ARROW_BUILD_SHARED "Link to the Arrow shared library" ON)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+if(ARROW_USE_XSIMD)
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_INSTALL_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS} # Defined in line 95
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
+ include_directories("${ARROW_SOURCE_DIR}/src" "${ARROW_SOURCE_DIR}/build/src")
+
+ # Using arrow_flight_shared, see example
+ # https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt
+ set(ARROW_GRPC_USE_SHARED ON)
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,--no-as-needed gRPC::grpc++_reflection
+ -Wl,--as-needed)
+
+ set(FLIGHT_PROTO_PATH "${ARROW_SOURCE}/format")
+ set(FLIGHT_PROTO "${ARROW_SOURCE}/format/Flight.proto")
+
+ set(FLIGHT_GENERATED_PROTO_FILES
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.h"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.h")
+
+ set(PROTO_DEPENDS ${FLIGHT_PROTO} gRPC::grpc_cpp_plugin)
+
+ add_custom_command(OUTPUT ${FLIGHT_GENERATED_PROTO_FILES}
+ COMMAND ${ARROW_PROTOBUF_PROTOC} "-I${FLIGHT_PROTO_PATH}"
+ "--cpp_out=${${ARROW_SOURCE_DIR}/build/src/arrow/flight/}" "${FLIGHT_PROTO}"
+ DEPENDS ${PROTO_DEPENDS} ARGS
+ COMMAND ${ARROW_PROTOBUF_PROTOC} "-I${FLIGHT_PROTO_PATH}"
+ "--grpc_out=${${ARROW_SOURCE_DIR}/build/src/arrow/flight/}"
+ "--plugin=protoc-gen-grpc=$<TARGET_FILE:gRPC::grpc_cpp_plugin>"
+ "${FLIGHT_PROTO}")
+
+ set_source_files_properties(${FLIGHT_GENERATED_PROTO_FILES} PROPERTIES GENERATED TRUE)
+ add_custom_target(flight_grpc_gen ALL DEPENDS ${FLIGHT_GENERATED_PROTO_FILES})
+
+ add_arrow_lib(arrow_python_flight
+ CMAKE_PACKAGE_NAME
+ ArrowPythonFlight
+ PKG_CONFIG_NAME
+ arrow-python-flight
+ SOURCES
+ flight.cc
+ OUTPUTS
+ ARROW_PYFLIGHT_LIBRARIES
+ DEPENDENCIES
+ flight_grpc_gen
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS} # Defined in line 95
+ SHARED_LINK_LIBS
+ arrow_python_shared
+ arrow_flight_shared
+ STATIC_LINK_LIBS
+ ${PYTHON_OTHER_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}"
+ PRIVATE_INCLUDES
+ "${Protobuf_INCLUDE_DIRS}")
+
+ add_dependencies(arrow_python ${ARROW_PYFLIGHT_LIBRARIES})
+
+ foreach(LIB_TARGET ${ARROW_PYFLIGHT_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYFLIGHT_EXPORTING)
+ endforeach()
+
+ if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_flight_static PUBLIC ARROW_STATIC)
+ endif()
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ # Clang, be quiet. Python C API has lots of macros
+ set_property(SOURCE ${ARROW_PYTHON_SRCS}
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS -Wno-parentheses-equality)
+endif()
+
+arrow_install_all_headers("arrow/python")
+
+# ----------------------------------------------------------------------
+
+if(ARROW_BUILD_TESTS)
+ enable_testing()
+
+ # Set necessary paths for cmake to find GTest
+ set(GTEST_ROOT ${ARROW_SOURCE_DIR}/build/googletest_ep-prefix)
Review Comment:
Oh, haven't thought about that.
Sure, that would be easy to implement and I think it is a good idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
raulcd commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1222962214
> @raulcd if u have spare time, could you pls confirm that the CI fails are not connected?
I don't think any of the failures are related to your changes.
verify-rc-source-python-macos-conda-amd64 --> The timeout on `arrow-flight-sql-test` is happening on other PRs. I.e. [this job](https://github.com/ursacomputing/crossbow/runs/7956736605?check_suite_focus=true) on [this PR](https://github.com/apache/arrow/pull/13943#pullrequestreview-1081095116)
The verify-rc-binaries-wheels are expected to fail as they only run on release candidates at the moment.
And I have seen the same R failure and the same Appveyor failure on [a different PR](https://github.com/apache/arrow/pull/13939).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1224152396
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1221865295
None of the failures seem related.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou merged pull request #13311: ARROW-16340: [C++][Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou merged PR #13311:
URL: https://github.com/apache/arrow/pull/13311
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1226936500
Yes. Thanks.
Sorry, I miss one more thing. Could you also remove `python*-devel` and `python*-numpy` from `dev/tasks/linux-packages/apache-arrow/yum/*/Dockerfile`s?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1174988730
Thanks for the comments!
I will try to research the segfault and will let u know if I get nowhere ...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1193324349
@github-actions crossbow submit *wheel*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1193910670
Thank you @raulcd for confirming the CI checks!
I will revert the pytest change and have another look at the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
lidavidm commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1178992752
> Comment and a question about Arrow Flight build in this PR.
>
> I bumped into a situation where the build has `ARROW_GRPC_USE_SHARED=off` and `ARROW_BUILD_STATIC=off`:
I took a quick look at what I think is going on but did not build the wheel myself to confirm. But to summarize, the Flight linking example is doing extra work because it is trying to link Flight and gRPC directly, and so it has to be consistent about how everything is linked. The library here only needs to link Flight so there should be no issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r899110501
##########
cpp/cmake_modules/FindArrowPython.cmake:
##########
@@ -46,8 +46,9 @@ endif()
find_package(Arrow ${find_package_arguments})
if(ARROW_FOUND)
+ message(STATUS "CPYARROW_HOME: ${CPYARROW_HOME}")
arrow_find_package(ARROW_PYTHON
- "${ARROW_HOME}"
+ "${CPYARROW_HOME}"
Review Comment:
> On long term we should probably have libarrow_python.so just be one of the shared objects constituting pyarrow.
That's actually what is already happening, I think. The libarrow_python.so gets copied into the pyarrow directory (either the repo itself for an inplace build, or the build directory, before that one gets copied to site-packages for a normal install).
But so for building the cython extensions, we need to point to where libarrow_python.so can be found (which is now a different location as where libarrow.so can be found). And doing that with a argument to cmake seems a good way to do that? (alternatively we might need to edit the `arrow_find_package` to work with this new situation?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
pitrou commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1219440115
It looks this CI failure is related:
https://github.com/apache/arrow/runs/7896212198?check_suite_focus=true#step:7:10333
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r907102453
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -19,8 +19,45 @@
# arrow_python
#
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+# Add cmake_modules and lib/cmake/arrow folder to CMAKE_MODULE_PATH
+# TODO: remove lib/cmake/arrow folder as it should not be necessary as find_package(Arrow)
+# should load all the cmake variables (ex ARROW_DATASET)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_SOURCE}/dist/lib/cmake/arrow")
+message(STATUS "CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH}")
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+find_package(Arrow REQUIRED)
+# Loading ArrowOptions, they should be loaded by default with find_package(Arrow REQUIRED)
+include(ArrowOptions)
Review Comment:
I had to include `ArrowOptions` separately but I think they should be found with `find_package(Arrow REQUIRED)`. The CI now fails as the search for `ArrowOptions` only works for me locally.
@kou any tips on how to help cmake find ArrowOptions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926418706
##########
python/pyarrow/src/api.h:
##########
@@ -17,14 +17,14 @@
#pragma once
-#include "arrow/python/arrow_to_pandas.h"
-#include "arrow/python/common.h"
-#include "arrow/python/datetime.h"
-#include "arrow/python/deserialize.h"
-#include "arrow/python/helpers.h"
-#include "arrow/python/inference.h"
-#include "arrow/python/io.h"
-#include "arrow/python/numpy_convert.h"
-#include "arrow/python/numpy_to_arrow.h"
-#include "arrow/python/python_to_arrow.h"
-#include "arrow/python/serialize.h"
+#include "arrow_to_pandas.h"
Review Comment:
Done: https://issues.apache.org/jira/browse/ARROW-17160
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1191184183
Revision: b0ea5ddfaceb9a56f60b1ed85fbf720c6baedc47
Submitted crossbow builds: [ursacomputing/crossbow @ actions-8117240504](https://github.com/ursacomputing/crossbow/branches/all?query=actions-8117240504)
|Task|Status|
|----|------|
|verify-rc-binaries-wheels-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-verify-rc-binaries-wheels-linux-almalinux-8-amd64)|
|verify-rc-binaries-wheels-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-verify-rc-binaries-wheels-linux-conda-latest-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64)|
|verify-rc-binaries-wheels-macos-10.15-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-verify-rc-binaries-wheels-macos-10.15-amd64)|
|verify-rc-binaries-wheels-macos-10.15-amd64-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-verify-rc-binaries-wheels-macos-10.15-amd64-conda)|
|verify-rc-binaries-wheels-macos-11-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-verify-rc-binaries-wheels-macos-11-amd64)|
|verify-rc-binaries-wheels-macos-11-amd64-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-verify-rc-binaries-wheels-macos-11-amd64-conda)|
|verify-rc-binaries-wheels-macos-11-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-verify-rc-binaries-wheels-macos-11-arm64)|
|verify-rc-binaries-wheels-windows|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-verify-rc-binaries-wheels-windows)|
|wheel-macos-big-sur-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-macos-big-sur-cp310-arm64)|
|wheel-macos-big-sur-cp310-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-macos-big-sur-cp310-universal2)|
|wheel-macos-big-sur-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-macos-big-sur-cp38-arm64)|
|wheel-macos-big-sur-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-macos-big-sur-cp39-arm64)|
|wheel-macos-big-sur-cp39-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-macos-big-sur-cp39-universal2)|
|wheel-macos-high-sierra-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-macos-high-sierra-cp310-amd64)|
|wheel-macos-high-sierra-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-macos-high-sierra-cp37-amd64)|
|wheel-macos-high-sierra-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-macos-high-sierra-cp38-amd64)|
|wheel-macos-high-sierra-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-macos-high-sierra-cp39-amd64)|
|wheel-macos-mavericks-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-macos-mavericks-cp310-amd64)|
|wheel-macos-mavericks-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-macos-mavericks-cp37-amd64)|
|wheel-macos-mavericks-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-macos-mavericks-cp38-amd64)|
|wheel-macos-mavericks-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-macos-mavericks-cp39-amd64)|
|wheel-manylinux2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-manylinux2014-cp310-amd64)|
|wheel-manylinux2014-cp310-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-manylinux2014-cp37-amd64)|
|wheel-manylinux2014-cp37-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-manylinux2014-cp38-amd64)|
|wheel-manylinux2014-cp38-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-manylinux2014-cp39-amd64)|
|wheel-manylinux2014-cp39-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-windows-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-windows-cp310-amd64)|
|wheel-windows-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-windows-cp37-amd64)|
|wheel-windows-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-windows-cp38-amd64)|
|wheel-windows-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8117240504-github-wheel-windows-cp39-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
raulcd commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1196522125
The MacOS 10.15 failures are expected failures as per the Github actions deprecation schedule. These failures should be temporary but are part of bringing awareness of the deprecation. We have a PR to move to MacOS 11: https://github.com/apache/arrow/pull/13684
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1169190828
> On the CI I get linker errors for python flight module (not locally on M1):
> https://github.com/apache/arrow/runs/7087113174?check_suite_focus=true
It seems that this link is wrong. (This doesn't have the error message.)
> What could be the setup work I am missing so that the libraries would be linked correctly?
We need `find_package(ArrowTesting REQUIRED)` to use `arrow_testing_shared` target.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1169583786
Thank you @kou!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1188671274
@github-actions crossbow submit *python*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1188787219
@github-actions crossbow submit verify-rc-source-python-linux-ubuntu-18.04-amd64
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1185304484
Small nitpick on the naming of the src directory: it is now `/pyarrow/src_arrow/`, but maybe we can make this just `pyarrow/src/` ? (the `_arrow` doesn't add much, and using `src` as the directory for C/C++ code in a python package is a pattern I have seen in other packages as well)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1193332836
@github-actions crossbow submit *python*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r911707915
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,467 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+
+# normalize ARROW_HOME path
+cmake_path(CONVERT "$ENV{ARROW_HOME}" TO_CMAKE_PATH_LIST ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+find_package(Python3Alt 3.7 REQUIRED)
+
+# Needed due to
+# CMake Error: INSTALL(EXPORT) given unknown export "arrow_python_targets"
+option(ARROW_BUILD_SHARED "Link to the Arrow shared library" ON)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+if(ARROW_USE_XSIMD)
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_INSTALL_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS} # Defined in line 95
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
+ include_directories("${ARROW_SOURCE_DIR}/src" "${ARROW_SOURCE_DIR}/build/src")
+
+ # Using arrow_flight_shared, see example
+ # https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt
+ set(ARROW_GRPC_USE_SHARED ON)
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,--no-as-needed gRPC::grpc++_reflection
+ -Wl,--as-needed)
+
+ set(FLIGHT_PROTO_PATH "${ARROW_SOURCE}/format")
+ set(FLIGHT_PROTO "${ARROW_SOURCE}/format/Flight.proto")
+
+ set(FLIGHT_GENERATED_PROTO_FILES
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.h"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.h")
+
+ set(PROTO_DEPENDS ${FLIGHT_PROTO} gRPC::grpc_cpp_plugin)
+
+ add_custom_command(OUTPUT ${FLIGHT_GENERATED_PROTO_FILES}
+ COMMAND ${ARROW_PROTOBUF_PROTOC} "-I${FLIGHT_PROTO_PATH}"
+ "--cpp_out=${${ARROW_SOURCE_DIR}/build/src/arrow/flight/}" "${FLIGHT_PROTO}"
Review Comment:
No, my mistake.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1172148758
> Hmm, we should try to make sure the Python Flight bindings can compile without including some internal headers.
Are you thinking in terms of this PR or in general?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r925230559
##########
docs/source/developers/python.rst:
##########
@@ -131,6 +131,29 @@ for ``.py`` files or
for ``.pyx`` and ``.pxi`` files. In this case you will also need to
install the `pytest-cython <https://github.com/lgpage/pytest-cython>`_ plugin.
+Testing Arrow Python (C++ code)
+-------------------------------
+If you want to run ctest for the tests that are included in the Arrow Python
Review Comment:
PyArrow C++?
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
Review Comment:
PyArrow version?
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
Review Comment:
pyarrow_cpp?
##########
ci/scripts/python_test.sh:
##########
@@ -54,4 +55,14 @@ export PYARROW_TEST_ORC
export PYARROW_TEST_PARQUET
export PYARROW_TEST_S3
+# Testing Arrow Python
Review Comment:
PyArrow C++?
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
Review Comment:
PyArrow C++?
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
Review Comment:
`PYARROW_CPP_VERSION`?
##########
docs/source/developers/python.rst:
##########
@@ -131,6 +131,29 @@ for ``.py`` files or
for ``.pyx`` and ``.pxi`` files. In this case you will also need to
install the `pytest-cython <https://github.com/lgpage/pytest-cython>`_ plugin.
+Testing Arrow Python (C++ code)
Review Comment:
PyArrow C++?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926311166
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
Review Comment:
> I think that all `python/pyarrow/src/` are rewritten in Cython eventually
I am not aware of any current plans to rewrite the C++ code in Cython (rather the opposite?)
(there is a ticket to rewrite the C++ _tests_ in cython, to avoid having both pytest and ctest tests for testing the python package)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1225542288
@github-actions crossbow submit *ubuntu*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1224329317
Revision: 007345d09b137344639ae708649c05f41a9daac4
Submitted crossbow builds: [ursacomputing/crossbow @ actions-da5580469c](https://github.com/ursacomputing/crossbow/branches/all?query=actions-da5580469c)
|Task|Status|
|----|------|
|almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-almalinux-8-amd64)|
|almalinux-8-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|almalinux-9-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-almalinux-9-amd64)|
|almalinux-9-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|amazon-linux-2-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-amazon-linux-2-amd64)|
|amazon-linux-2-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|centos-7-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-centos-7-amd64)|
|centos-8-stream-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-centos-8-stream-amd64)|
|centos-8-stream-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|centos-9-stream-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-centos-9-stream-amd64)|
|centos-9-stream-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|conan-maximum|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-conan-maximum)|
|conan-minimum|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-conan-minimum)|
|conda-clean|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-clean)|
|conda-linux-gcc-py310-arm64|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py310-arm64)|
|conda-linux-gcc-py310-cpu|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py310-cpu)|
|conda-linux-gcc-py310-cuda|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py310-cuda)|
|conda-linux-gcc-py310-ppc64le|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py310-ppc64le)|
|conda-linux-gcc-py37-arm64|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py37-arm64)|
|conda-linux-gcc-py37-cpu-r40|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py37-cpu-r40)|
|conda-linux-gcc-py37-cpu-r41|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py37-cpu-r41)|
|conda-linux-gcc-py37-cuda|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py37-cuda)|
|conda-linux-gcc-py37-ppc64le|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py37-ppc64le)|
|conda-linux-gcc-py38-arm64|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py38-arm64)|
|conda-linux-gcc-py38-cpu|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py38-cpu)|
|conda-linux-gcc-py38-cuda|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py38-cuda)|
|conda-linux-gcc-py38-ppc64le|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py38-ppc64le)|
|conda-linux-gcc-py39-arm64|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py39-arm64)|
|conda-linux-gcc-py39-cpu|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py39-cpu)|
|conda-linux-gcc-py39-cuda|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py39-cuda)|
|conda-linux-gcc-py39-ppc64le|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-linux-gcc-py39-ppc64le)|
|conda-osx-arm64-clang-py310|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-osx-arm64-clang-py310)|
|conda-osx-arm64-clang-py38|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-osx-arm64-clang-py38)|
|conda-osx-arm64-clang-py39|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-osx-arm64-clang-py39)|
|conda-osx-clang-py310|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-osx-clang-py310)|
|conda-osx-clang-py37-r40|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-osx-clang-py37-r40)|
|conda-osx-clang-py37-r41|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-osx-clang-py37-r41)|
|conda-osx-clang-py38|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-osx-clang-py38)|
|conda-osx-clang-py39|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-osx-clang-py39)|
|conda-win-vs2017-py310|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-win-vs2017-py310)|
|conda-win-vs2017-py37-r40|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-win-vs2017-py37-r40)|
|conda-win-vs2017-py37-r41|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-win-vs2017-py37-r41)|
|conda-win-vs2017-py38|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-win-vs2017-py38)|
|conda-win-vs2017-py39|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-conda-win-vs2017-py39)|
|debian-bookworm-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-debian-bookworm-amd64)|
|debian-bookworm-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|debian-bullseye-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-debian-bullseye-amd64)|
|debian-bullseye-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|example-cpp-minimal-build-static|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-example-cpp-minimal-build-static)|
|example-cpp-minimal-build-static-system-dependency|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-example-cpp-minimal-build-static-system-dependency)|
|example-python-minimal-build-fedora-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-example-python-minimal-build-fedora-conda)|
|example-python-minimal-build-ubuntu-venv|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-example-python-minimal-build-ubuntu-venv)|
|homebrew-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-homebrew-cpp)|
|homebrew-r-autobrew|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-homebrew-r-autobrew)|
|homebrew-r-brew|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-homebrew-r-brew)|
|java-jars|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-java-jars)|
|nuget|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-nuget)|
|python-sdist|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-python-sdist)|
|r-binary-packages|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-r-binary-packages)|
|test-alpine-linux-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-alpine-linux-cpp)|
|test-build-cpp-fuzz|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-build-cpp-fuzz)|
|test-build-vcpkg-win|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-build-vcpkg-win)|
|test-conda-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-cpp)|
|test-conda-cpp-valgrind|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-conda-cpp-valgrind)|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.10)|
|test-conda-python-3.7|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.7)|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.7-hdfs-3.2.1)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-0.24|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.7-pandas-0.24)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-spark-v3.1.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.7-spark-v3.1.2)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.8)|
|test-conda-python-3.8-hypothesis|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.8-hypothesis)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.8-pandas-nightly)|
|test-conda-python-3.8-spark-v3.2.0|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.8-spark-v3.2.0)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.9)|
|test-conda-python-3.9-dask-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.9-dask-latest)|
|test-conda-python-3.9-dask-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.9-dask-master)|
|test-conda-python-3.9-pandas-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.9-pandas-master)|
|test-conda-python-3.9-spark-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-conda-python-3.9-spark-master)|
|test-debian-10-cpp-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-debian-10-cpp-amd64)|
|test-debian-10-cpp-i386|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-debian-10-cpp-i386)|
|test-debian-11-cpp-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-debian-11-cpp-amd64)|
|test-debian-11-cpp-i386|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-debian-11-cpp-i386)|
|test-debian-11-go-1.16|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-debian-11-go-1.16)|
|test-debian-11-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-debian-11-python-3)|
|test-debian-c-glib|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-debian-c-glib)|
|test-debian-ruby|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-debian-ruby)|
|test-fedora-35-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-fedora-35-cpp)|
|test-fedora-35-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-fedora-35-python-3)|
|test-fedora-r-clang-sanitizer|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-fedora-r-clang-sanitizer)|
|test-r-arrow-backwards-compatibility|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-r-arrow-backwards-compatibility)|
|test-r-depsource-bundled|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-r-depsource-bundled)|
|test-r-depsource-system|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-r-depsource-system)|
|test-r-dev-duckdb|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-r-dev-duckdb)|
|test-r-devdocs|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-r-devdocs)|
|test-r-gcc-11|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-r-gcc-11)|
|test-r-gcc-12|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-r-gcc-12)|
|test-r-install-local|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-r-install-local)|
|test-r-linux-as-cran|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-r-linux-as-cran)|
|test-r-linux-rchk|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-r-linux-rchk)|
|test-r-linux-valgrind|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-r-linux-valgrind)|
|test-r-minimal-build|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-r-minimal-build)|
|test-r-offline-maximal|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-r-offline-maximal)|
|test-r-offline-minimal|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-r-offline-minimal)|
|test-r-rhub-debian-gcc-devel-lto-latest|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-r-rhub-debian-gcc-devel-lto-latest)|
|test-r-rhub-debian-gcc-release-custom-ccache|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-r-rhub-debian-gcc-release-custom-ccache)|
|test-r-rhub-ubuntu-gcc-release-latest|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-r-rhub-ubuntu-gcc-release-latest)|
|test-r-rocker-r-base-latest|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-r-rocker-r-base-latest)|
|test-r-rstudio-r-base-4.1-opensuse153|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-r-rstudio-r-base-4.1-opensuse153)|
|test-r-rstudio-r-base-4.2-centos7-devtoolset-8|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-r-rstudio-r-base-4.2-centos7-devtoolset-8)|
|test-r-rstudio-r-base-4.2-focal|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-r-rstudio-r-base-4.2-focal)|
|test-r-ubuntu-22.04|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-r-ubuntu-22.04)|
|test-r-versions|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-r-versions)|
|test-skyhook-integration|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-skyhook-integration)|
|test-ubuntu-18.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-ubuntu-18.04-cpp)|
|test-ubuntu-18.04-cpp-release|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-ubuntu-18.04-cpp-release)|
|test-ubuntu-18.04-cpp-static|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-ubuntu-18.04-cpp-static)|
|test-ubuntu-18.04-r-sanitizer|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-ubuntu-18.04-r-sanitizer)|
|test-ubuntu-20.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-ubuntu-20.04-cpp)|
|test-ubuntu-20.04-cpp-14|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-ubuntu-20.04-cpp-14)|
|test-ubuntu-20.04-cpp-17|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-ubuntu-20.04-cpp-17)|
|test-ubuntu-20.04-cpp-bundled|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-ubuntu-20.04-cpp-bundled)|
|test-ubuntu-20.04-cpp-thread-sanitizer|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-ubuntu-20.04-cpp-thread-sanitizer)|
|test-ubuntu-20.04-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-ubuntu-20.04-python-3)|
|test-ubuntu-22.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-ubuntu-22.04-cpp)|
|test-ubuntu-c-glib|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-ubuntu-c-glib)|
|test-ubuntu-default-docs|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-da5580469c-azure-test-ubuntu-default-docs)|
|test-ubuntu-ruby|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-test-ubuntu-ruby)|
|ubuntu-bionic-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-ubuntu-bionic-amd64)|
|ubuntu-bionic-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|ubuntu-focal-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-ubuntu-focal-amd64)|
|ubuntu-focal-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|ubuntu-jammy-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-ubuntu-jammy-amd64)|
|ubuntu-jammy-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|verify-rc-source-cpp-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-cpp-linux-almalinux-8-amd64)|
|verify-rc-source-cpp-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-cpp-linux-conda-latest-amd64)|
|verify-rc-source-cpp-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-cpp-linux-ubuntu-18.04-amd64)|
|verify-rc-source-cpp-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-cpp-linux-ubuntu-20.04-amd64)|
|verify-rc-source-cpp-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-cpp-linux-ubuntu-22.04-amd64)|
|verify-rc-source-cpp-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-cpp-macos-amd64)|
|verify-rc-source-cpp-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-cpp-macos-arm64)|
|verify-rc-source-cpp-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-cpp-macos-conda-amd64)|
|verify-rc-source-csharp-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-csharp-linux-almalinux-8-amd64)|
|verify-rc-source-csharp-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-csharp-linux-conda-latest-amd64)|
|verify-rc-source-csharp-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-csharp-linux-ubuntu-18.04-amd64)|
|verify-rc-source-csharp-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-csharp-linux-ubuntu-20.04-amd64)|
|verify-rc-source-csharp-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-csharp-linux-ubuntu-22.04-amd64)|
|verify-rc-source-csharp-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-csharp-macos-amd64)|
|verify-rc-source-csharp-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-csharp-macos-arm64)|
|verify-rc-source-go-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-go-linux-almalinux-8-amd64)|
|verify-rc-source-go-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-go-linux-conda-latest-amd64)|
|verify-rc-source-go-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-go-linux-ubuntu-18.04-amd64)|
|verify-rc-source-go-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-go-linux-ubuntu-20.04-amd64)|
|verify-rc-source-go-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-go-linux-ubuntu-22.04-amd64)|
|verify-rc-source-go-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-go-macos-amd64)|
|verify-rc-source-go-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-go-macos-arm64)|
|verify-rc-source-integration-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-integration-linux-almalinux-8-amd64)|
|verify-rc-source-integration-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-integration-linux-conda-latest-amd64)|
|verify-rc-source-integration-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-integration-linux-ubuntu-18.04-amd64)|
|verify-rc-source-integration-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-integration-linux-ubuntu-20.04-amd64)|
|verify-rc-source-integration-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-integration-linux-ubuntu-22.04-amd64)|
|verify-rc-source-integration-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-integration-macos-amd64)|
|verify-rc-source-integration-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-integration-macos-arm64)|
|verify-rc-source-integration-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-integration-macos-conda-amd64)|
|verify-rc-source-java-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-java-linux-almalinux-8-amd64)|
|verify-rc-source-java-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-java-linux-conda-latest-amd64)|
|verify-rc-source-java-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-java-linux-ubuntu-18.04-amd64)|
|verify-rc-source-java-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-java-linux-ubuntu-20.04-amd64)|
|verify-rc-source-java-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-java-linux-ubuntu-22.04-amd64)|
|verify-rc-source-java-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-java-macos-amd64)|
|verify-rc-source-js-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-js-linux-almalinux-8-amd64)|
|verify-rc-source-js-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-js-linux-conda-latest-amd64)|
|verify-rc-source-js-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-js-linux-ubuntu-18.04-amd64)|
|verify-rc-source-js-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-js-linux-ubuntu-20.04-amd64)|
|verify-rc-source-js-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-js-linux-ubuntu-22.04-amd64)|
|verify-rc-source-js-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-js-macos-amd64)|
|verify-rc-source-js-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-js-macos-arm64)|
|verify-rc-source-python-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-python-linux-almalinux-8-amd64)|
|verify-rc-source-python-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-python-linux-conda-latest-amd64)|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
|verify-rc-source-python-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-python-linux-ubuntu-20.04-amd64)|
|verify-rc-source-python-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-python-linux-ubuntu-22.04-amd64)|
|verify-rc-source-python-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-python-macos-amd64)|
|verify-rc-source-python-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-python-macos-arm64)|
|verify-rc-source-python-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-python-macos-conda-amd64)|
|verify-rc-source-ruby-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-ruby-linux-almalinux-8-amd64)|
|verify-rc-source-ruby-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-ruby-linux-conda-latest-amd64)|
|verify-rc-source-ruby-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-ruby-linux-ubuntu-18.04-amd64)|
|verify-rc-source-ruby-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-ruby-linux-ubuntu-20.04-amd64)|
|verify-rc-source-ruby-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-ruby-linux-ubuntu-22.04-amd64)|
|verify-rc-source-ruby-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-ruby-macos-amd64)|
|verify-rc-source-ruby-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-ruby-macos-arm64)|
|verify-rc-source-windows|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-verify-rc-source-windows)|
|wheel-macos-big-sur-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-macos-big-sur-cp310-arm64)|
|wheel-macos-big-sur-cp310-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-macos-big-sur-cp310-universal2)|
|wheel-macos-big-sur-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-macos-big-sur-cp38-arm64)|
|wheel-macos-big-sur-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-macos-big-sur-cp39-arm64)|
|wheel-macos-big-sur-cp39-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-macos-big-sur-cp39-universal2)|
|wheel-macos-high-sierra-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-macos-high-sierra-cp310-amd64)|
|wheel-macos-high-sierra-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-macos-high-sierra-cp37-amd64)|
|wheel-macos-high-sierra-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-macos-high-sierra-cp38-amd64)|
|wheel-macos-high-sierra-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-macos-high-sierra-cp39-amd64)|
|wheel-macos-mavericks-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-macos-mavericks-cp310-amd64)|
|wheel-macos-mavericks-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-macos-mavericks-cp37-amd64)|
|wheel-macos-mavericks-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-macos-mavericks-cp38-amd64)|
|wheel-macos-mavericks-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-macos-mavericks-cp39-amd64)|
|wheel-manylinux2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-manylinux2014-cp310-amd64)|
|wheel-manylinux2014-cp310-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-manylinux2014-cp37-amd64)|
|wheel-manylinux2014-cp37-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-manylinux2014-cp38-amd64)|
|wheel-manylinux2014-cp38-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-manylinux2014-cp39-amd64)|
|wheel-manylinux2014-cp39-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-windows-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-windows-cp310-amd64)|
|wheel-windows-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-windows-cp37-amd64)|
|wheel-windows-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-windows-cp38-amd64)|
|wheel-windows-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-da5580469c-github-wheel-windows-cp39-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1223825265
I have gone through the nightly failures and there are some that are connected to this PR.
@raulcd @kou could I get your feedback on the first three?
**almalinkux and centos failures**
---
Some of the Arrow modules (`gandiva-devel`, `arrow-dataset-devel`) seem to think they need python libs and they can not find them as they are not in the Arrow C++ part anymore.
<details>
<summary>almalinkux example log</summary>
<br>
```
...
Provides: cmake(Gandiva) = 10.0.0 cmake(gandiva) = 10.0.0 gandiva-devel = 9.0.0.dev738-1.el8 gandiva-devel(x86-64) = 9.0.0.dev738-1.el8 pkgconfig(gandiva) = 10.0.0-SNAPSHOT
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Requires: /usr/bin/pkg-config cmake-filesystem(x86-64) libgandiva.so.1000()(64bit) pkgconfig(arrow)
Processing files: arrow9-python-libs-9.0.0.dev738-1.el8.x86_64
error: File not found: /root/rpmbuild/BUILDROOT/arrow-9.0.0.dev738-1.el8.x86_64/usr/lib64/libarrow_python.so.*
Executing(%doc): /bin/sh -e /var/tmp/rpm-tmp.FezCIu
+ umask 022
+ cd /root/rpmbuild/BUILD
+ cd apache-arrow-9.0.0.dev738
+ DOCDIR=/root/rpmbuild/BUILDROOT/arrow-9.0.0.dev738-1.el8.x86_64/usr/share/doc/arrow9-python-libs
+ export LC_ALL=C
+ LC_ALL=C
+ export DOCDIR
+ /usr/bin/mkdir -p /root/rpmbuild/BUILDROOT/arrow-9.0.0.dev738-1.el8.x86_64/usr/share/doc/arrow9-python-libs
+ cp -pr README.md /root/rpmbuild/BUILDROOT/arrow-9.0.0.dev738-1.el8.x86_64/usr/share/doc/arrow9-python-libs
+ exit 0
Executing(%license): /bin/sh -e /var/tmp/rpm-tmp.4doqup
+ umask 022
+ cd /root/rpmbuild/BUILD
+ cd apache-arrow-9.0.0.dev738
+ LICENSEDIR=/root/rpmbuild/BUILDROOT/arrow-9.0.0.dev738-1.el8.x86_64/usr/share/licenses/arrow9-python-libs
+ export LC_ALL=C
+ LC_ALL=C
+ export LICENSEDIR
+ /usr/bin/mkdir -p /root/rpmbuild/BUILDROOT/arrow-9.0.0.dev738-1.el8.x86_64/usr/share/licenses/arrow9-python-libs
+ cp -pr LICENSE.txt /root/rpmbuild/BUILDROOT/arrow-9.0.0.dev738-1.el8.x86_64/usr/share/licenses/arrow9-python-libs
+ cp -pr NOTICE.txt /root/rpmbuild/BUILDROOT/arrow-9.0.0.dev738-1.el8.x86_64/usr/share/licenses/arrow9-python-libs
+ exit 0
Directory not found: /root/rpmbuild/BUILDROOT/arrow-9.0.0.dev738-1.el8.x86_64/usr/include/arrow/python
RPM build errors:
File not found: /root/rpmbuild/BUILDROOT/arrow-9.0.0.dev738-1.el8.x86_64/usr/lib64/libarrow_python.so.*
Failed ./build.sh
rake aborted!
```
</details>
[Link to the log](https://github.com/ursacomputing/crossbow/runs/7963525652?check_suite_focus=true#step:6:4710)
**conda-linux, conda-osx and conda-win**
---
Not really sure what is going on here (passes for conda-osx-arm64).
After pyarrow is built, on the step of [compiling .pyc files](https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=31969&view=logs&j=0da5d1d9-276d-5173-c4c4-9d4d4ed14fdb&t=d9b15392-e4ce-5e4c-0c8c-b69645229181&l=4631), there is a warning that `libarrow_python` and `libarrow_python_flight` are not found - looks like an issue with `RPATH:`
```
WARNING (pyarrow,lib/python3.10/site-packages/pyarrow/libarrow_python_flight.so.1000): $RPATH/libarrow_python.so.1000 not found in packages, sysroot(s) nor the missing_dso_whitelist.
.. is this binary repackaging?
```
Then `pyarrow-tests` are being built successfully but [fail after that](https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=31953&view=logs&j=0da5d1d9-276d-5173-c4c4-9d4d4ed14fdb&t=d9b15392-e4ce-5e4c-0c8c-b69645229181&l=5479) (due to `arrow_python` lib not being found).
<details>
<summary>conda-linux example log</summary>
<br>
```
...
export PREFIX=/build/arrow-cpp-ext_1661216150497/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol
export SRC_DIR=/build/arrow-cpp-ext_1661216150497/test_tmp
+ test -f /build/arrow-cpp-ext_1661216150497/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/include/arrow/api.h
+ test -f /build/arrow-cpp-ext_1661216150497/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/include/arrow/flight/types.h
+ test -f /build/arrow-cpp-ext_1661216150497/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/include/plasma/client.h
+ test -f /build/arrow-cpp-ext_1661216150497/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/include/gandiva/engine.h
+ test -f /build/arrow-cpp-ext_1661216150497/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/include/parquet/api/reader.h
+ test -f /build/arrow-cpp-ext_1661216150497/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/libarrow.so
+ test -f /build/arrow-cpp-ext_1661216150497/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/libarrow_dataset.so
+ test -f /build/arrow-cpp-ext_1661216150497/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/libarrow_flight.so
+ test -f /build/arrow-cpp-ext_1661216150497/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/libarrow_python.so
Tests failed for arrow-cpp-9.0.0.dev738-py310hf337226_0_cpu.tar.bz2 - moving package to /build/broken
WARNING:conda_build.build:Tests failed for arrow-cpp-9.0.0.dev738-py310hf337226_0_cpu.tar.bz2 - moving package to /build/broken
TESTS FAILED: arrow-cpp-9.0.0.dev738-py310hf337226_0_cpu.tar.bz2
```
</details>
[Link to the log](https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=31969&view=logs&j=0da5d1d9-276d-5173-c4c4-9d4d4ed14fdb&t=d9b15392-e4ce-5e4c-0c8c-b69645229181&l=5498)
**debian, ubuntu-focal and ubuntu-jammy (Crossbow) failures**
---
I think `dh_auto_clean` wants to remove `arrow_python` libs but can not find them. Not sure why is it trying to remove them from Arrow cpp if they are not built there anymore.
<details>
<summary>almalinkux example log</summary>
<br>
```
...
# Remove built files to reduce disk usage
dh_auto_clean \
--sourcedirectory=cpp \
--builddirectory=cpp_build
make[1]: Leaving directory '/build/apache-arrow-9.0.0.dev738'
dh_install
dh_install: warning: Cannot find (any matches for) "usr/lib/*/libarrow_python.so.*" (tried in ., debian/tmp)
dh_install: warning: libarrow-python1000 missing files: usr/lib/*/libarrow_python.so.*
dh_install: warning: Cannot find (any matches for) "usr/lib/*/libarrow_python_flight.so.*" (tried in ., debian/tmp)
dh_install: warning: libarrow-python-flight1000 missing files: usr/lib/*/libarrow_python_flight.so.*
dh_install: warning: Cannot find (any matches for) "usr/lib/*/cmake/arrow/ArrowPythonConfig*.cmake" (tried in ., debian/tmp)
dh_install: warning: libarrow-python-dev missing files: usr/lib/*/cmake/arrow/ArrowPythonConfig*.cmake
dh_install: warning: Cannot find (any matches for) "usr/lib/*/cmake/arrow/ArrowPythonTargets*.cmake" (tried in ., debian/tmp)
dh_install: warning: libarrow-python-dev missing files: usr/lib/*/cmake/arrow/ArrowPythonTargets*.cmake
dh_install: warning: Cannot find (any matches for) "usr/lib/*/cmake/arrow/FindArrowPython.cmake" (tried in ., debian/tmp)
dh_install: warning: libarrow-python-dev missing files: usr/lib/*/cmake/arrow/FindArrowPython.cmake
dh_install: warning: Cannot find (any matches for) "usr/lib/*/libarrow_python.a" (tried in ., debian/tmp)
dh_install: warning: libarrow-python-dev missing files: usr/lib/*/libarrow_python.a
dh_install: warning: Cannot find (any matches for) "usr/lib/*/libarrow_python.so" (tried in ., debian/tmp)
dh_install: warning: libarrow-python-dev missing files: usr/lib/*/libarrow_python.so
dh_install: warning: Cannot find (any matches for) "usr/lib/*/pkgconfig/arrow-python.pc" (tried in ., debian/tmp)
dh_install: warning: libarrow-python-dev missing files: usr/lib/*/pkgconfig/arrow-python.pc
dh_install: warning: Cannot find (any matches for) "usr/lib/*/cmake/arrow/ArrowPythonFlightConfig*.cmake" (tried in ., debian/tmp)
dh_install: warning: libarrow-python-flight-dev missing files: usr/lib/*/cmake/arrow/ArrowPythonFlightConfig*.cmake
dh_install: warning: Cannot find (any matches for) "usr/lib/*/cmake/arrow/ArrowPythonFlightTargets*.cmake" (tried in ., debian/tmp)
dh_install: warning: libarrow-python-flight-dev missing files: usr/lib/*/cmake/arrow/ArrowPythonFlightTargets*.cmake
dh_install: warning: Cannot find (any matches for) "usr/lib/*/cmake/arrow/FindArrowPythonFlight.cmake" (tried in ., debian/tmp)
dh_install: warning: libarrow-python-flight-dev missing files: usr/lib/*/cmake/arrow/FindArrowPythonFlight.cmake
dh_install: warning: Cannot find (any matches for) "usr/lib/*/libarrow_python_flight.a" (tried in ., debian/tmp)
dh_install: warning: libarrow-python-flight-dev missing files: usr/lib/*/libarrow_python_flight.a
dh_install: warning: Cannot find (any matches for) "usr/lib/*/libarrow_python_flight.so" (tried in ., debian/tmp)
dh_install: warning: libarrow-python-flight-dev missing files: usr/lib/*/libarrow_python_flight.so
dh_install: warning: Cannot find (any matches for) "usr/lib/*/pkgconfig/arrow-python-flight.pc" (tried in ., debian/tmp)
dh_install: warning: libarrow-python-flight-dev missing files: usr/lib/*/pkgconfig/arrow-python-flight.pc
dh_install: error: missing files, aborting
make: *** [debian/rules:14: binary] Error 25
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
debuild: fatal error at line 1182:
dpkg-buildpackage -us -uc -ui failed
Failed debuild -eCCACHE_COMPILERCHECK -eCCACHE_COMPRESS -eCCACHE_COMPRESSLEVEL -eCCACHE_DIR -eCCACHE_MAXSIZE --prepend-path=/usr/lib/ccache -us -uc
```
</details>
[Link to the log](https://github.com/ursacomputing/crossbow/runs/7963617736?check_suite_focus=true#step:6:2391)
**homebrew failures**
---
Do not seem related (R test faillure).
**test-r failures**
---
Do not seem related (R test faillure).
**test-ubuntu failures**
---
Do not seem related (`substrait/function_test`).
**ubuntu-jammy Travis failures**
---
Do not seem related (looks like acero error).
**verify-rc-source failures**
---
Do not seem related (Arrow C++ test failures).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1224014872
conda: It seems that we need to remove `*arrow_python*` in https://github.com/apache/arrow/blob/master/dev/tasks/conda-recipes/arrow-cpp/meta.yaml such as https://github.com/apache/arrow/blob/master/dev/tasks/conda-recipes/arrow-cpp/meta.yaml#L126
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1226859924
✅ **almalinux** and **centos** are passing
✅ **conda-linux, conda-osx** and **conda-win**
failing due to xsimd error but have passed after the corrections 2 days ago https://github.com/apache/arrow/pull/13311#issuecomment-1224329317
✅ **debian** and **ubuntu-focal** passing
❓ **ubuntu-jammy (Travis)** failing but doesn't look connected ([link to the build](https://app.travis-ci.com/github/ursacomputing/crossbow/builds/254788946))
Also the CI failures are not connected. @kou mind approving the changes for the nightlies?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
pitrou commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1226867672
@AlenkaF Perhaps you want to rebase/merge from master to get the latest CI fixes :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926196087
##########
cpp/cmake_modules/FindArrowPython.cmake:
##########
@@ -46,8 +46,9 @@ endif()
find_package(Arrow ${find_package_arguments})
if(ARROW_FOUND)
+ message(STATUS "PYARROW_CPP_HOME: ${PYARROW_CPP_HOME}")
Review Comment:
I don't think that it's customary. I'll rewrite or remove all our `Find*.cmake` in ARROW-12175. So we don't need to care much about `Find*.cmake` files here.
FYI, 33/160(20%) CMake's bundled `Find*.cmake` shows `STATUS` message:
```bash
n_statuses=0
n_finds=0
for find_cmake in /usr/share/cmake-3.23/Modules/Find*.cmake; do
n_finds=$(($n_finds + 1))
grep -q STATUS $find_cmake && n_statuses=$(($n_statuses + 1))
done
echo "$n_statuses/$n_finds ($(($n_statuses / ($n_finds + 0.0))))"
# => 33/160 (0.20624999999999999)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926425522
##########
python/setup.py:
##########
@@ -227,6 +228,118 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_pyarrow_cpp(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
+ raise ValueError("--build-type (or PYARROW_BUILD_TYPE) needs to "
+ "be 'release' or 'debug'")
+
+ # The directory containing this setup.py
+ source = os.path.dirname(os.path.abspath(__file__))
+ # The directory containing this PyArrow cpp CMakeLists.txt
+ source_pyarrow_cpp = pjoin(source, "pyarrow/src")
+
+ # The directory for the module being built
+ build_cmd = self.get_finalized_command('build')
+ saved_cwd = os.getcwd()
+ build_dir = pjoin(saved_cwd, 'build', 'dist')
+ build_include = pjoin(saved_cwd, 'build', 'dist', 'include')
+ build_lib = pjoin(os.getcwd(), build_cmd.build_lib)
+
+ # The directory containing Arrow C++ build
+ arrow_build_dir = os.environ.get('ARROW_BUILD_DIR', 'build')
+ if self.inplace:
+ # a bit hacky
+ build_lib = saved_cwd
+ if not os.path.isdir(build_dir):
+ self.mkpath(build_dir)
+ if not os.path.isdir(build_lib):
+ self.mkpath(build_lib)
+ if not os.path.isdir(build_include):
+ self.mkpath(build_include)
+
+ # Change to the build directory
+ with changed_dir(build_dir):
+ # cmake args
+ cmake_options = [
+ '-DCMAKE_INSTALL_PREFIX=' +
+ str(pjoin(saved_cwd, 'build/dist')),
+ '-DCMAKE_BUILD_TYPE={0}'.format(self.build_type.lower()),
+ '-DARROW_BUILD_DIR=' + str(arrow_build_dir),
+ '-DPYTHON_EXECUTABLE=%s' % sys.executable,
+ '-DPython3_EXECUTABLE=%s' % sys.executable,
+ ]
+
+ # Check for specific options
+ def append_cmake_bool(value, varname):
+ cmake_options.append('-D{0}={1}'.format(
+ varname, 'on' if value else 'off'))
+
+ append_cmake_bool(self.with_dataset, 'PYARROW_WITH_DATASET')
+ append_cmake_bool(self.with_parquet_encryption,
+ 'PYARROW_WITH_PARQUET_ENCRYPTION')
+ append_cmake_bool(self.with_hdfs,
+ 'PYARROW_WITH_HDFS')
+
+ # Windows
+ if self.cmake_generator:
+ cmake_options += ['-G', self.cmake_generator]
+
+ # build args
+ build_tool_args = []
+ if os.environ.get('PYARROW_PARALLEL'):
+ build_tool_args.append('--')
+ build_tool_args.append(
+ '-j{0}'.format(os.environ['PYARROW_PARALLEL']))
+
+ # run cmake
+ print("-- Running cmake for pyarrow cpp")
+ self.spawn(['cmake'] + cmake_options + [source_pyarrow_cpp])
+ print("-- Finished cmake for pyarrow cpp")
+ # run make & install
+ print("-- Running cmake build and install for pyarrow cpp")
+ self.spawn(['cmake', '--build', '.', '--target', 'install'] +
+ build_tool_args)
+ print("-- Finished cmake build and install for pyarrow cpp")
+
+ # Move the libraries to the place expected by the Python build
+ try:
+ os.makedirs(pjoin(build_lib, 'pyarrow'))
+ except OSError:
+ pass
+
+ # helper function
+ def copy_libs(folder_name):
+ for libname in os.listdir(pjoin(build_dir, folder_name)):
+ if "python" in libname:
+ libname_path = pjoin(build_lib, "pyarrow", libname)
+ if os.path.exists(libname_path):
+ os.remove(libname_path)
+ print(
+ f"Copying {pjoin(build_dir, folder_name, libname)}"
+ f" to {pjoin(build_lib, 'pyarrow', libname)}")
+ shutil.copy(pjoin(build_dir, folder_name, libname),
+ pjoin(build_lib, "pyarrow"))
+
+ # Move libraries to python/pyarrow
+ # For windows builds, move dll from bin
+ for folder in ['lib', 'lib64', 'bin']:
+ try:
+ copy_libs(folder)
+ except OSError:
Review Comment:
If the folder doesn't exist. Normally there is a `lib` folder but some builds have `bin` or `lib64`. But it can only be one of them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1180122806
@kou Unfortunately I have yet another issue, the XSIMD dependancy (quoting @jorisvandenbossche):
"If Arrow C++ is built with a certain xsimd source (eg bundled vs conda vs system), how can we get that information?
The goal is to use the same source for building PyArrow C++ code, but is this actually possible? (eg if xsimd is bundled into libarrow, does it expose it enough so we can use that in pyarrow? (to use #include <xsimd/xsimd.hpp> in pyarrow)) Or if the bundled approach was used, will pyarrow have to bundle it itself again?"
The current solution I am using is not only hackish but it also doesn't find the correct path if the source is not conda (though it works for me locally with bundled source):
https://github.com/apache/arrow/blob/9a0d9ec1d4882378b75deb23f77473349327c6b9/python/pyarrow/src_arrow/CMakeLists.txt#L263-L289
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
pitrou commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1180347276
https://github.com/apache/arrow/pull/13569 should remove the xsimd dependency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1191996457
Revision: 8947536858335b567d1c1790f1a82fe2152e13a9
Submitted crossbow builds: [ursacomputing/crossbow @ actions-8aef201439](https://github.com/ursacomputing/crossbow/branches/all?query=actions-8aef201439)
|Task|Status|
|----|------|
|verify-rc-binaries-wheels-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-verify-rc-binaries-wheels-linux-almalinux-8-amd64)|
|verify-rc-binaries-wheels-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-verify-rc-binaries-wheels-linux-conda-latest-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64)|
|verify-rc-binaries-wheels-macos-10.15-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-verify-rc-binaries-wheels-macos-10.15-amd64)|
|verify-rc-binaries-wheels-macos-10.15-amd64-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-verify-rc-binaries-wheels-macos-10.15-amd64-conda)|
|verify-rc-binaries-wheels-macos-11-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-verify-rc-binaries-wheels-macos-11-amd64)|
|verify-rc-binaries-wheels-macos-11-amd64-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-verify-rc-binaries-wheels-macos-11-amd64-conda)|
|verify-rc-binaries-wheels-macos-11-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-verify-rc-binaries-wheels-macos-11-arm64)|
|verify-rc-binaries-wheels-windows|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-verify-rc-binaries-wheels-windows)|
|wheel-macos-big-sur-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-macos-big-sur-cp310-arm64)|
|wheel-macos-big-sur-cp310-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-macos-big-sur-cp310-universal2)|
|wheel-macos-big-sur-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-macos-big-sur-cp38-arm64)|
|wheel-macos-big-sur-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-macos-big-sur-cp39-arm64)|
|wheel-macos-big-sur-cp39-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-macos-big-sur-cp39-universal2)|
|wheel-macos-high-sierra-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-macos-high-sierra-cp310-amd64)|
|wheel-macos-high-sierra-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-macos-high-sierra-cp37-amd64)|
|wheel-macos-high-sierra-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-macos-high-sierra-cp38-amd64)|
|wheel-macos-high-sierra-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-macos-high-sierra-cp39-amd64)|
|wheel-macos-mavericks-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-macos-mavericks-cp310-amd64)|
|wheel-macos-mavericks-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-macos-mavericks-cp37-amd64)|
|wheel-macos-mavericks-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-macos-mavericks-cp38-amd64)|
|wheel-macos-mavericks-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-macos-mavericks-cp39-amd64)|
|wheel-manylinux2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-manylinux2014-cp310-amd64)|
|wheel-manylinux2014-cp310-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-manylinux2014-cp37-amd64)|
|wheel-manylinux2014-cp37-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-manylinux2014-cp38-amd64)|
|wheel-manylinux2014-cp38-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-manylinux2014-cp39-amd64)|
|wheel-manylinux2014-cp39-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-windows-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-windows-cp310-amd64)|
|wheel-windows-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-windows-cp37-amd64)|
|wheel-windows-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-windows-cp38-amd64)|
|wheel-windows-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-8aef201439-github-wheel-windows-cp39-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r922965062
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
Review Comment:
I thought it was for `add_arrow_lib` but looking at it now it seems it is only used in case of a sdist build
https://github.com/apache/arrow/blob/385fe4315b2ccefc90860cee18a3661f67b57c3e/python/pyarrow/src_arrow/CMakeLists.txt#L199-L203
and so can be removed from here (line 40).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r922966341
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
Review Comment:
This path actually points to the `pyarrow` folder, meaning `arrow/python/pyarrow`. So I would leave it as is `(PYARROW_SOURCE_DIR)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1187345001
@jorisvandenbossche the name of the folder `src_arrow` is now changed to `src`.
@kou I have went through your suggestions. The PR is now ready for another round of review.
The name for C PyArrow is now PyArrow CPP =)
The CI fails are not related:
- [AMD64 MacOS 10.15 Python 3](https://github.com/apache/arrow/runs/7385730723?check_suite_focus=true#logs) is connected to [ARROW-17104](https://issues.apache.org/jira/browse/ARROW-17104)
- [C GLib failure](https://github.com/apache/arrow/runs/7388483254?check_suite_focus=true)
- [FAILED: debug/plasma-store-server](https://github.com/apache/arrow/runs/7388494669?check_suite_focus=true) - C++ protobuf failure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1184544262
@github-actions crossbow submit python-sdist
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r921915106
##########
cpp/cmake_modules/FindArrowPython.cmake:
##########
@@ -46,8 +46,9 @@ endif()
find_package(Arrow ${find_package_arguments})
if(ARROW_FOUND)
+ message(STATUS "CPYARROW_HOME: ${CPYARROW_HOME}")
arrow_find_package(ARROW_PYTHON
- "${ARROW_HOME}"
+ "${CPYARROW_HOME}"
Review Comment:
Is that clear enough that this is specifically for where the _C++_ arrow python libs are located, and not generically the _python_ arrow python module (pyarrow)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
raulcd commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r921954967
##########
python/pyarrow/tests/test_gdb.py:
##########
@@ -154,7 +154,7 @@ def select_frame(self, func_name):
# but it's not available on old GDB versions (such as 8.1.1),
# so instead parse the stack trace for a matching frame number.
out = self.run_command("info stack")
- pat = r"(?mi)^#(\d+)\s+.* in " + re.escape(func_name) + " "
+ pat = r"(?mi)^#(\d+)\s+.* in " + re.escape(func_name)
Review Comment:
tests were failing to select the correct frame because the output from gdb was slightly different:
```
-> out = self.run_command("info stack")
(Pdb) n
info stack
#0 psnip_trap () at /home/raulcd/code/arrow/cpp/src/arrow/vendored/portable-snippets/debug-trap.h:36
#1 0x00007ffff3f13096 in arrow::internal::DebugTrap () at /home/raulcd/code/arrow/cpp/src/arrow/util/debug.cc:28
#2 0x00007ffff24a8a42 in arrow::gdb::TestSession() () from /home/raulcd/code/arrow/python/pyarrow/libarrow_python.so.900
#3 0x00007ffff680fa1d in __pyx_pw_7pyarrow_3lib_7_gdb_test_session(_object*, _object*) () from /home/raulcd/code/arrow/python/pyarrow/lib.cpython-310-x86_64-linux-gnu.so
#4 0x00005555556aca1e in ?? ()
#5 0x0000555555698104 in _PyEval_EvalFrameDefault ()
#6 0x0000555555694956 in ?? ()
#7 0x0000555555788906 in PyEval_EvalCode ()
#8 0x00005555557b5b88 in ?? ()
#9 0x00005555557ae86b in ?? ()
#10 0x00005555557a6531 in PyRun_StringFlags ()
#11 0x00005555557a6471 in PyRun_SimpleStringFlags ()
#12 0x00005555557a56a5 in Py_RunMain ()
#13 0x000055555577b8dd in Py_BytesMain ()
#14 0x00007ffff7c7ad90 in __libc_start_call_main (main=main@entry=0x55555577b8a0, argc=argc@entry=3, argv=argv@entry=0x7fffffffec68) at ../sysdeps/nptl/libc_start_call_main.h:58
#15 0x00007ffff7c7ae40 in __libc_start_main_impl (main=0x55555577b8a0, argc=3, argv=0x7fffffffec68, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffec58) at ../csu/libc-start.c:392
#16 0x000055555577b7d5 in _start ()
(gdb) > /home/raulcd/code/arrow/python/pyarrow/tests/test_gdb.py(162)select_frame()
```
the regex to select the frame expects something like `#2 0x00007ffff24a8a42 in arrow::gdb::TestSession ()` instead of `#2 0x00007ffff24a8a42 in arrow::gdb::TestSession() ()` (see the space between the ())
Why the output from gdb was different, I am not entirely sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1169572419
> > On the CI I get linker errors for python flight module (not locally on M1):
> > https://github.com/apache/arrow/runs/7087113174?check_suite_focus=true
>
> It seems that this link is wrong. (This doesn't have the error message.)
Maybe this way would be easier:
<details>
<summary>Error with make build of C PyArrow</summary>
<br>
```
-- Running cmake for C pyarrow
cmake -DCMAKE_INSTALL_PREFIX=/arrow/python/build/dist -DCMAKE_BUILD_TYPE=debug -DPYARROW_WITH_DATASET=on -DPYARROW_WITH_PARQUET_ENCRYPTION=on -DPYARROW_WITH_HDFS=on /arrow/python/pyarrow/src_arrow
-- CMAKE_MODULE_PATH /arrow/python/cmake_modules;/opt/conda/envs/arrow/lib/cmake/arrow
-- The C compiler identification is GNU 10.3.0
-- The CXX compiler identification is GNU 10.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/conda/envs/arrow/bin/x86_64-conda-linux-gnu-cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/conda/envs/arrow/bin/x86_64-conda-linux-gnu-c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PkgConfig: /opt/conda/envs/arrow/bin/pkg-config (found version "0.29.2")
-- Found Arrow: /opt/conda/envs/arrow/include (found version "9.0.0")
-- Arrow version: 9.0.0 (HOME: /opt/conda/envs/arrow)
-- Arrow SO and ABI version: 900
-- Arrow full SO version: 900.0.0
-- Found the Arrow core shared library: /opt/conda/envs/arrow/lib/libarrow.so
-- Found the Arrow core import library: /opt/conda/envs/arrow/lib/libarrow.so
-- Found the Arrow core static library: ARROW_static_lib-NOTFOUND
-- Found Python3: /opt/conda/envs/arrow/bin/python3.9 (found suitable version "3.9.13", minimum required is "3.7") found components: Interpreter Development.Module NumPy
-- Found Python3Alt: /opt/conda/envs/arrow/bin/python3.9 (Required is at least version "3.7")
CMake Warning (dev) at CMakeLists.txt:55 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.
For compatibility with older versions of CMake, option is clearing the
normal variable 'ARROW_BUILD_SHARED'.
This warning is for project developers. Use -Wno-dev to suppress it.
-- Found ArrowDataset: /opt/conda/envs/arrow/include (found version "9.0.0")
-- Found the Arrow Dataset by HOME: /opt/conda/envs/arrow
-- Found the Arrow Dataset shared library: /opt/conda/envs/arrow/lib/libarrow_dataset.so
-- Found the Arrow Dataset import library: /opt/conda/envs/arrow/lib/libarrow_dataset.so
-- Found the Arrow Dataset static library: ARROW_DATASET_static_lib-NOTFOUND
-- Performing Test CXX_LINKER_SUPPORTS_VERSION_SCRIPT
-- Performing Test CXX_LINKER_SUPPORTS_VERSION_SCRIPT - Success
-- Found ArrowFlight: /opt/conda/envs/arrow/include (found version "9.0.0")
-- Found the Arrow Flight by HOME: /opt/conda/envs/arrow
-- Found the Arrow Flight shared library: /opt/conda/envs/arrow/lib/libarrow_flight.so
-- Found the Arrow Flight import library: /opt/conda/envs/arrow/lib/libarrow_flight.so
-- Found the Arrow Flight static library: ARROW_FLIGHT_static_lib-NOTFOUND
-- Found ZLIB: /opt/conda/envs/arrow/lib/libz.so (found version "1.2.12")
-- Found Protobuf: /opt/conda/envs/arrow/lib/libprotobuf.so;-lpthread (found version "3.19.4")
-- Found OpenSSL: /opt/conda/envs/arrow/lib/libcrypto.so (found version "1.1.1o")
-- Found c-ares: /opt/conda/envs/arrow/lib/cmake/c-ares/c-ares-config.cmake (found version "1.18.1")
-- Found Threads: TRUE
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found RE2 via CMake.
-- Configuring done
-- Generating done
-- Build files have been written to: /arrow/python/build/dist/temp
-- Finished cmake for C pyarrow
-- Running make build and install for C pyarrow
make -j4
[ 3%] Generating /arrow/cpp/build/src/arrow/flight/Flight.pb.cc, /arrow/cpp/build/src/arrow/flight/Flight.pb.h, /arrow/cpp/build/src/arrow/flight/Flight.grpc.pb.cc, /arrow/cpp/build/src/arrow/flight/Flight.grpc.pb.h
make[2]: I/arrow/format: No such file or directory
make[2]: I/arrow/format: No such file or directory
[ 7%] Building CXX object CMakeFiles/arrow_python_objlib.dir/arrow_to_pandas.cc.o
[ 11%] Building CXX object CMakeFiles/arrow_python_objlib.dir/benchmark.cc.o
[ 15%] Building CXX object CMakeFiles/arrow_python_objlib.dir/common.cc.o
[ 15%] Built target flight_grpc_gen
[ 19%] Building CXX object CMakeFiles/arrow_python_objlib.dir/datetime.cc.o
[ 23%] Building CXX object CMakeFiles/arrow_python_objlib.dir/decimal.cc.o
[ 26%] Building CXX object CMakeFiles/arrow_python_objlib.dir/deserialize.cc.o
[ 30%] Building CXX object CMakeFiles/arrow_python_objlib.dir/extension_type.cc.o
[ 34%] Building CXX object CMakeFiles/arrow_python_objlib.dir/gdb.cc.o
[ 38%] Building CXX object CMakeFiles/arrow_python_objlib.dir/helpers.cc.o
[ 42%] Building CXX object CMakeFiles/arrow_python_objlib.dir/inference.cc.o
[ 46%] Building CXX object CMakeFiles/arrow_python_objlib.dir/init.cc.o
[ 50%] Building CXX object CMakeFiles/arrow_python_objlib.dir/io.cc.o
[ 53%] Building CXX object CMakeFiles/arrow_python_objlib.dir/ipc.cc.o
[ 57%] Building CXX object CMakeFiles/arrow_python_objlib.dir/numpy_convert.cc.o
[ 61%] Building CXX object CMakeFiles/arrow_python_objlib.dir/numpy_to_arrow.cc.o
[ 65%] Building CXX object CMakeFiles/arrow_python_objlib.dir/python_to_arrow.cc.o
[ 69%] Building CXX object CMakeFiles/arrow_python_objlib.dir/pyarrow.cc.o
[ 73%] Building CXX object CMakeFiles/arrow_python_objlib.dir/serialize.cc.o
[ 76%] Building CXX object CMakeFiles/arrow_python_objlib.dir/udf.cc.o
[ 80%] Building CXX object CMakeFiles/arrow_python_objlib.dir/csv.cc.o
[ 84%] Building CXX object CMakeFiles/arrow_python_objlib.dir/filesystem.cc.o
[ 88%] Building CXX object CMakeFiles/arrow_python_objlib.dir/parquet_encryption.cc.o
[ 88%] Built target arrow_python_objlib
[ 92%] Linking CXX shared library libarrow_python.so
/opt/conda/envs/arrow/bin/../lib/gcc/x86_64-conda-linux-gnu/10.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: cannot find -lparquet_shared
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/arrow_python_shared.dir/build.make:126: libarrow_python.so.900.0.0] Error 1
make[1]: *** [CMakeFiles/Makefile2:202: CMakeFiles/arrow_python_shared.dir/all] Error 2
make: *** [Makefile:136: all] Error 2
error: command '/opt/conda/envs/arrow/bin/make' failed with exit code 2
error: subprocess-exited-with-error
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r911755346
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,467 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+
+# normalize ARROW_HOME path
+cmake_path(CONVERT "$ENV{ARROW_HOME}" TO_CMAKE_PATH_LIST ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+find_package(Python3Alt 3.7 REQUIRED)
+
+# Needed due to
+# CMake Error: INSTALL(EXPORT) given unknown export "arrow_python_targets"
+option(ARROW_BUILD_SHARED "Link to the Arrow shared library" ON)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+if(ARROW_USE_XSIMD)
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_INSTALL_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS} # Defined in line 95
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
+ include_directories("${ARROW_SOURCE_DIR}/src" "${ARROW_SOURCE_DIR}/build/src")
+
+ # Using arrow_flight_shared, see example
+ # https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt
+ set(ARROW_GRPC_USE_SHARED ON)
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,--no-as-needed gRPC::grpc++_reflection
+ -Wl,--as-needed)
+
+ set(FLIGHT_PROTO_PATH "${ARROW_SOURCE}/format")
+ set(FLIGHT_PROTO "${ARROW_SOURCE}/format/Flight.proto")
+
+ set(FLIGHT_GENERATED_PROTO_FILES
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.h"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.h")
Review Comment:
AFAIU when building C PyArrow `${CMAKE_CURRENT_BINARY_DIR}` points to the build dir of the C PyArrow build which is in arrow/python. But here I need files from Arrow C++ build. I had no other idea than to use relative path like so.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r908027466
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -19,8 +19,45 @@
# arrow_python
#
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+# Add cmake_modules and lib/cmake/arrow folder to CMAKE_MODULE_PATH
+# TODO: remove lib/cmake/arrow folder as it should not be necessary as find_package(Arrow)
+# should load all the cmake variables (ex ARROW_DATASET)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_SOURCE}/dist/lib/cmake/arrow")
+message(STATUS "CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH}")
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+find_package(Arrow REQUIRED)
+# Loading ArrowOptions, they should be loaded by default with find_package(Arrow REQUIRED)
+include(ArrowOptions)
Review Comment:
Could you set `CMAKE_PREFPX_PATH=/opt/conda/envs/arrow` environment variable (not CMake variable)? Then `find_package(Arrow REQUIRED)` loads `ArrowOptions` too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1220513214
@github-actions crossbow submit *python*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1223347064
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r922974267
##########
ci/scripts/python_test.sh:
##########
@@ -54,4 +55,11 @@ export PYARROW_TEST_ORC
export PYARROW_TEST_PARQUET
export PYARROW_TEST_S3
+# Testing Arrow Python
+if [ "${ARROW_BUILD_TESTS}" == "ON" ]; then
+ pushd ${test_dir}
+ ctest
Review Comment:
Sure, will add 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1196162051
@github-actions crossbow submit *wheel*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926292425
##########
python/setup.py:
##########
@@ -354,6 +468,19 @@ def append_cmake_bool(value, varname):
shutil.move(pjoin(build_prefix, 'include'),
pjoin(build_lib, 'pyarrow'))
+ # We need to, again, add the PyArrow cpp include folder
+ build_pyarrow_cpp_include = pjoin(
+ saved_cwd, 'build/dist/include')
+ if not os.path.isdir(
+ pjoin(build_pyarrow_cpp_include, "arrow", "python")):
+ self.mkpath(
Review Comment:
Hope I understood correctly, the changes I made (together with updated comment from https://github.com/apache/arrow/pull/13311#discussion_r926243308):
```python
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1225643354
@github-actions crossbow submit ubuntu*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1225259744
@github-actions crossbow submit *almalinux* *centos*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
lidavidm commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1172270020
I think it should be easy enough to remove the need for `serialization_internal.h` inside `python/flight.cc` (use [Make](https://github.com/apache/arrow/blob/38918ef0d59387895759223bee2d704c293bcfb9/cpp/src/arrow/flight/types.h#L433-L436) instead of directly constructing the structure)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
lidavidm commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r913768189
##########
python/pyarrow/src_arrow/flight.cc:
##########
@@ -371,24 +370,21 @@ Status CreateFlightInfo(const std::shared_ptr<arrow::Schema>& schema,
const std::vector<arrow::flight::FlightEndpoint>& endpoints,
int64_t total_records, int64_t total_bytes,
std::unique_ptr<arrow::flight::FlightInfo>* out) {
- arrow::flight::FlightInfo::Data flight_data;
- RETURN_NOT_OK(arrow::flight::internal::SchemaToString(*schema, &flight_data.schema));
- flight_data.descriptor = descriptor;
- flight_data.endpoints = endpoints;
- flight_data.total_records = total_records;
- flight_data.total_bytes = total_bytes;
- arrow::flight::FlightInfo value(flight_data);
- *out = std::unique_ptr<arrow::flight::FlightInfo>(new arrow::flight::FlightInfo(value));
+ auto result =
+ arrow::flight::FlightInfo::Make(*schema,
+ descriptor,
+ endpoints,
+ total_records,
+ total_bytes);
+ *out = std::unique_ptr<arrow::flight::FlightInfo>(&(*result));
Review Comment:
Oh sorry it should have been `std::unique_ptr<...>(new FlightInfo(std::move(result)))`
That also probably explains your crash! Since it currently creates a unique_ptr backed by a stack-allocated (temporary) FlightInfo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1180144637
We need xsimd because `arrow/util/utf8.h` uses xsimd and `numpy_to_arrow.cc` uses `arrow/util/utf8.h`, right?
@pitrou Can we move `ValidateAsciiSimd()` in https://github.com/apache/arrow/blob/master/cpp/src/arrow/util/utf8.h to `utf8.cc`? Or can we move `ValidateAsciiSimd()` to `utf8_internal.h`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1180163876
> We need xsimd because `arrow/util/utf8.h` uses xsimd and `numpy_to_arrow.cc` uses `arrow/util/utf8.h`, right?
Yes, this should be correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
pitrou commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1172087972
Hmm, we should try to make sure the Python Flight bindings can compile without including some internal headers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1178767296
We are building Python Flight, not the examples, but I am using the example that I pasted to link to gRPC in same way. Sorry for the confusion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r949084234
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,457 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
Review Comment:
Related to Antoine's comment about the failing build, I suppose it is the above line that needs to be updated (and which was probably updated on master in the other location, but updating this branch on top of master didn't bring along that change)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1219346725
Revision: e0cb6e1f22d7c5f3f7741ff079bb7d63ed44cb10
Submitted crossbow builds: [ursacomputing/crossbow @ actions-bc26c3cadd](https://github.com/ursacomputing/crossbow/branches/all?query=actions-bc26c3cadd)
|Task|Status|
|----|------|
|example-python-minimal-build-fedora-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-example-python-minimal-build-fedora-conda)|
|example-python-minimal-build-ubuntu-venv|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-example-python-minimal-build-ubuntu-venv)|
|python-sdist|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-python-sdist)|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.10)|
|test-conda-python-3.7|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.7)|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.7-hdfs-3.2.1)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-0.24|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.7-pandas-0.24)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-spark-v3.1.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.7-spark-v3.1.2)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.8)|
|test-conda-python-3.8-hypothesis|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.8-hypothesis)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.8-pandas-nightly)|
|test-conda-python-3.8-spark-v3.2.0|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.8-spark-v3.2.0)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.9)|
|test-conda-python-3.9-dask-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.9-dask-latest)|
|test-conda-python-3.9-dask-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.9-dask-master)|
|test-conda-python-3.9-pandas-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.9-pandas-master)|
|test-conda-python-3.9-spark-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-test-conda-python-3.9-spark-master)|
|test-debian-11-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-bc26c3cadd-azure-test-debian-11-python-3)|
|test-fedora-35-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-bc26c3cadd-azure-test-fedora-35-python-3)|
|test-ubuntu-20.04-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-bc26c3cadd-azure-test-ubuntu-20.04-python-3)|
|verify-rc-source-python-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-verify-rc-source-python-linux-almalinux-8-amd64)|
|verify-rc-source-python-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-verify-rc-source-python-linux-conda-latest-amd64)|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
|verify-rc-source-python-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-verify-rc-source-python-linux-ubuntu-20.04-amd64)|
|verify-rc-source-python-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-verify-rc-source-python-linux-ubuntu-22.04-amd64)|
|verify-rc-source-python-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-verify-rc-source-python-macos-amd64)|
|verify-rc-source-python-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-verify-rc-source-python-macos-arm64)|
|verify-rc-source-python-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-bc26c3cadd-github-verify-rc-source-python-macos-conda-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1226922077
I have rebased the PR and corrected the `arrow.soec.in` file, hope I got it right @kou? [The changes can be found here.](https://github.com/apache/arrow/pull/13311/commits/767dda9bd900199b68166cc7bca18ad90bb9316d)
If yes I can run the nightly checks again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1228043100
Lots of green 😍 😆
@kou @pitrou the failures do not look related, all issues from before in the nightlies should be fixed 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1196537673
The CI is green, the extended builds are also green (failures are expected, see https://github.com/apache/arrow/pull/13311#issuecomment-1196522125 and https://github.com/apache/arrow/pull/13311#pullrequestreview-1049009931) so this **PR is ready for the 10.0.0** version.
_I would like to have the approval of @jorisvandenbossche and @pitrou before that._
The suggestions were all looked at and commented or implemented. There are couple of follow-up issues that are planned for the next release - all saved under the umbrella issue ARROW-17122:
ARROW-17172: [C++][Python] test_cython_api fails on windows
ARROW-17016: [C++][Python] Move Arrow Python C++ tests into Cython
ARROW-17154: [C++] Change cmake project name from arrow_python to pyarrow_cpp
ARROW-17017: [C++][Python] Enable automate re-build of Arrow Python
ARROW-17160: [C++] Create a base directory for PyArrow CPP header files
**A note about the PyArrow cpp tests**
The tests are actually not run on any of the CI builds at the moment as none of the builds have GTest bundled.
We could change the GTest source for one of the conda builds temporary, if we feel it is necessary.
Otherwise the check can be done locally until the tests are moved to Cython with ARROW-17016 (in the release 10.0.0)
Thank you all for the help and guidance!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1184326563
Revision: b19a0b20fba6644ec41d5ea934fc8fbadf5509c7
Submitted crossbow builds: [ursacomputing/crossbow @ actions-551d5ac2f4](https://github.com/ursacomputing/crossbow/branches/all?query=actions-551d5ac2f4)
|Task|Status|
|----|------|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-551d5ac2f4-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1184564485
Revision: 3dee49782182c1835dd60ec7cc83ca7ba85279ec
Submitted crossbow builds: [ursacomputing/crossbow @ actions-61cbd11ff4](https://github.com/ursacomputing/crossbow/branches/all?query=actions-61cbd11ff4)
|Task|Status|
|----|------|
|python-sdist|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-61cbd11ff4-github-python-sdist)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] amol- commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
amol- commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r894525752
##########
cpp/cmake_modules/FindArrowPython.cmake:
##########
@@ -46,8 +46,9 @@ endif()
find_package(Arrow ${find_package_arguments})
if(ARROW_FOUND)
+ message(STATUS "CPYARROW_HOME: ${CPYARROW_HOME}")
arrow_find_package(ARROW_PYTHON
- "${ARROW_HOME}"
+ "${CPYARROW_HOME}"
Review Comment:
This smells a bit hackish. Feels a bit like we are building two totally different projects.
On long term we should probably have `libarrow_python.so` just be one of the shared objects constituting pyarrow. `libarrow_python.so` should probably be possible to integrate with the rest of the built files using `add_subdirectory` or something equivalent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926238665
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
Review Comment:
Created https://issues.apache.org/jira/browse/ARROW-17154 to change the cmake project name as a follow-up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1187305533
@github-actions crossbow submit *python*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r925437185
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
Review Comment:
Let's make it a separate JIRA where we can decide whether we want to do it or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1191608497
Revision: 57cd0d6b7a3a64a495ee2c03ad10aaece9ecc0a9
Submitted crossbow builds: [ursacomputing/crossbow @ actions-19a9a8240b](https://github.com/ursacomputing/crossbow/branches/all?query=actions-19a9a8240b)
|Task|Status|
|----|------|
|wheel-manylinux2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-19a9a8240b-github-wheel-manylinux2014-cp310-amd64)|
|wheel-manylinux2014-cp310-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-19a9a8240b-github-wheel-manylinux2014-cp37-amd64)|
|wheel-manylinux2014-cp37-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-19a9a8240b-github-wheel-manylinux2014-cp38-amd64)|
|wheel-manylinux2014-cp38-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-19a9a8240b-github-wheel-manylinux2014-cp39-amd64)|
|wheel-manylinux2014-cp39-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1188682075
Revision: 0c2a662c87a1f7a53ed33ed5763787cc403adce6
Submitted crossbow builds: [ursacomputing/crossbow @ actions-50d481d323](https://github.com/ursacomputing/crossbow/branches/all?query=actions-50d481d323)
|Task|Status|
|----|------|
|example-python-minimal-build-fedora-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-example-python-minimal-build-fedora-conda)|
|example-python-minimal-build-ubuntu-venv|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-example-python-minimal-build-ubuntu-venv)|
|python-sdist|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-python-sdist)|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.10)|
|test-conda-python-3.7|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.7)|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.7-hdfs-3.2.1)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-0.24|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.7-pandas-0.24)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-spark-v3.1.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.7-spark-v3.1.2)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.8)|
|test-conda-python-3.8-hypothesis|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.8-hypothesis)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.8-pandas-nightly)|
|test-conda-python-3.8-spark-v3.2.0|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.8-spark-v3.2.0)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.9)|
|test-conda-python-3.9-dask-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.9-dask-latest)|
|test-conda-python-3.9-dask-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.9-dask-master)|
|test-conda-python-3.9-pandas-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.9-pandas-master)|
|test-conda-python-3.9-spark-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-test-conda-python-3.9-spark-master)|
|test-debian-11-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-50d481d323-azure-test-debian-11-python-3)|
|test-fedora-35-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-50d481d323-azure-test-fedora-35-python-3)|
|test-ubuntu-20.04-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-50d481d323-azure-test-ubuntu-20.04-python-3)|
|verify-rc-source-python-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-verify-rc-source-python-linux-almalinux-8-amd64)|
|verify-rc-source-python-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-verify-rc-source-python-linux-conda-latest-amd64)|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
|verify-rc-source-python-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-verify-rc-source-python-linux-ubuntu-20.04-amd64)|
|verify-rc-source-python-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-verify-rc-source-python-linux-ubuntu-22.04-amd64)|
|verify-rc-source-python-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-verify-rc-source-python-macos-amd64)|
|verify-rc-source-python-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-verify-rc-source-python-macos-arm64)|
|verify-rc-source-python-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-50d481d323-github-verify-rc-source-python-macos-conda-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926243308
##########
python/setup.py:
##########
@@ -354,6 +468,19 @@ def append_cmake_bool(value, varname):
shutil.move(pjoin(build_prefix, 'include'),
pjoin(build_lib, 'pyarrow'))
+ # We need to, again, add the PyArrow cpp include folder
Review Comment:
It is because of these lines right before:
https://github.com/apache/arrow/blob/66cd4f9dffb067deed9af0bdb805e99d68acbcc0/python/setup.py#L464-L469
where the whole include folder is deleted in case of re-build. As the pyarrow cpp includes are added in the previous step (`_run_cmake_pyarrow_cpp`) they always get deleted. But are also needed when the `_run_cmake` starts.
Will add some more info to the comment +1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1222748835
@raulcd if u have spare time, could you pls confirm that the CI fails are not connected?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #13311: ARROW-16340: [C++][Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1228150462
Thanks!
I merge this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #13311: ARROW-16340: [C++][Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1228304521
['Python', 'R'] benchmarks have high level of regressions.
[ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/b51831c1b58a4de7a872dc36d59d7a91...65dba3eeb9494382b7d4e0ef9ece4d7d/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1224054150
Thanks @raulcd @kou, will do 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r954589032
##########
dev/tasks/linux-packages/apache-arrow/yum/arrow.spec.in:
##########
@@ -506,86 +506,6 @@ Libraries and header files for Gandiva.
%{_libdir}/pkgconfig/gandiva.pc
%endif
-%if %{use_python}
Review Comment:
There are still `%if %{use_python}` in this file. Could you also remove them?
Could you also remove `%define use_python (%{rhel} >= 8)`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1184571263
Revision: 11a6d47284c03da5371a1fc499b20d64ca4afbdb
Submitted crossbow builds: [ursacomputing/crossbow @ actions-144a32a245](https://github.com/ursacomputing/crossbow/branches/all?query=actions-144a32a245)
|Task|Status|
|----|------|
|python-sdist|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-144a32a245-github-python-sdist)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r911698925
##########
python/setup.py:
##########
@@ -93,6 +93,7 @@ def build_extensions(self):
_build_ext.build_extensions(self)
def run(self):
+ self._run_cmake_cpyarrow()
Review Comment:
How about calling this `arrow_python` not `cpyarrow`?
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,467 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+
+# normalize ARROW_HOME path
+cmake_path(CONVERT "$ENV{ARROW_HOME}" TO_CMAKE_PATH_LIST ARROW_HOME)
Review Comment:
We can't use `cmake_path` because it requires CMake 3.20 or later: https://cmake.org/cmake/help/latest/command/cmake_path.html
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,467 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+
+# normalize ARROW_HOME path
+cmake_path(CONVERT "$ENV{ARROW_HOME}" TO_CMAKE_PATH_LIST ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+find_package(Python3Alt 3.7 REQUIRED)
+
+# Needed due to
+# CMake Error: INSTALL(EXPORT) given unknown export "arrow_python_targets"
+option(ARROW_BUILD_SHARED "Link to the Arrow shared library" ON)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+if(ARROW_USE_XSIMD)
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_INSTALL_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS} # Defined in line 95
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
+ include_directories("${ARROW_SOURCE_DIR}/src" "${ARROW_SOURCE_DIR}/build/src")
+
+ # Using arrow_flight_shared, see example
+ # https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt
+ set(ARROW_GRPC_USE_SHARED ON)
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,--no-as-needed gRPC::grpc++_reflection
+ -Wl,--as-needed)
+
+ set(FLIGHT_PROTO_PATH "${ARROW_SOURCE}/format")
+ set(FLIGHT_PROTO "${ARROW_SOURCE}/format/Flight.proto")
+
+ set(FLIGHT_GENERATED_PROTO_FILES
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.h"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.h")
+
+ set(PROTO_DEPENDS ${FLIGHT_PROTO} gRPC::grpc_cpp_plugin)
+
+ add_custom_command(OUTPUT ${FLIGHT_GENERATED_PROTO_FILES}
+ COMMAND ${ARROW_PROTOBUF_PROTOC} "-I${FLIGHT_PROTO_PATH}"
+ "--cpp_out=${${ARROW_SOURCE_DIR}/build/src/arrow/flight/}" "${FLIGHT_PROTO}"
+ DEPENDS ${PROTO_DEPENDS} ARGS
+ COMMAND ${ARROW_PROTOBUF_PROTOC} "-I${FLIGHT_PROTO_PATH}"
+ "--grpc_out=${${ARROW_SOURCE_DIR}/build/src/arrow/flight/}"
+ "--plugin=protoc-gen-grpc=$<TARGET_FILE:gRPC::grpc_cpp_plugin>"
+ "${FLIGHT_PROTO}")
+
+ set_source_files_properties(${FLIGHT_GENERATED_PROTO_FILES} PROPERTIES GENERATED TRUE)
+ add_custom_target(flight_grpc_gen ALL DEPENDS ${FLIGHT_GENERATED_PROTO_FILES})
+
+ add_arrow_lib(arrow_python_flight
+ CMAKE_PACKAGE_NAME
+ ArrowPythonFlight
+ PKG_CONFIG_NAME
+ arrow-python-flight
+ SOURCES
+ flight.cc
+ OUTPUTS
+ ARROW_PYFLIGHT_LIBRARIES
+ DEPENDENCIES
+ flight_grpc_gen
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS} # Defined in line 95
+ SHARED_LINK_LIBS
+ arrow_python_shared
+ arrow_flight_shared
+ STATIC_LINK_LIBS
+ ${PYTHON_OTHER_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}"
+ PRIVATE_INCLUDES
+ "${Protobuf_INCLUDE_DIRS}")
+
+ add_dependencies(arrow_python ${ARROW_PYFLIGHT_LIBRARIES})
+
+ foreach(LIB_TARGET ${ARROW_PYFLIGHT_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYFLIGHT_EXPORTING)
+ endforeach()
+
+ if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_flight_static PUBLIC ARROW_STATIC)
+ endif()
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ # Clang, be quiet. Python C API has lots of macros
+ set_property(SOURCE ${ARROW_PYTHON_SRCS}
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS -Wno-parentheses-equality)
+endif()
+
+arrow_install_all_headers("arrow/python")
+
+# ----------------------------------------------------------------------
+
+if(ARROW_BUILD_TESTS)
+ enable_testing()
+
+ # Set necessary paths for cmake to find GTest
+ set(GTEST_ROOT ${ARROW_SOURCE_DIR}/build/googletest_ep-prefix)
Review Comment:
We can't assume that all users use `${ARROW_SOURCE_DIR}/build` as a build directory.
(For example, I always use `cpp.build` as a build directory.)
How about accepting `ARROW_BUILD_DIR` CMake variable from users to detect build directory for Apache Arrow C++?
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,467 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
Review Comment:
How about the following names?
```suggestion
get_filename_component(ARROW_PYTHON_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
get_filename_component(PYARROW_SOURCE_DIR ${ARROW_PYTHON_SOURCE_DIR} DIRECTORY)
get_filename_component(ARROW_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
```
##########
cpp/cmake_modules/FindArrowPython.cmake:
##########
@@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.
-# - Find Arrow Python (arrow/python/api.h, libarrow_python.a, libarrow_python.so)
+# - Find Arrow Python (python/pyarrrow/src_arrow/api.h, libarrow_python.a, libarrow_python.so)
Review Comment:
This is a needless change.
The path shows what value can be used for `#include` not where `api.h` is placed in source directory.
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,467 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+
+# normalize ARROW_HOME path
+cmake_path(CONVERT "$ENV{ARROW_HOME}" TO_CMAKE_PATH_LIST ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+find_package(Python3Alt 3.7 REQUIRED)
+
+# Needed due to
+# CMake Error: INSTALL(EXPORT) given unknown export "arrow_python_targets"
+option(ARROW_BUILD_SHARED "Link to the Arrow shared library" ON)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+if(ARROW_USE_XSIMD)
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_INSTALL_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
Review Comment:
FYI: I will rename this to `ARROW_CMAKE_DIR` in #13477.
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,467 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+
+# normalize ARROW_HOME path
+cmake_path(CONVERT "$ENV{ARROW_HOME}" TO_CMAKE_PATH_LIST ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+find_package(Python3Alt 3.7 REQUIRED)
+
+# Needed due to
+# CMake Error: INSTALL(EXPORT) given unknown export "arrow_python_targets"
+option(ARROW_BUILD_SHARED "Link to the Arrow shared library" ON)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+if(ARROW_USE_XSIMD)
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_INSTALL_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS} # Defined in line 95
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
+ include_directories("${ARROW_SOURCE_DIR}/src" "${ARROW_SOURCE_DIR}/build/src")
+
+ # Using arrow_flight_shared, see example
+ # https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt
+ set(ARROW_GRPC_USE_SHARED ON)
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,--no-as-needed gRPC::grpc++_reflection
+ -Wl,--as-needed)
+
+ set(FLIGHT_PROTO_PATH "${ARROW_SOURCE}/format")
+ set(FLIGHT_PROTO "${ARROW_SOURCE}/format/Flight.proto")
+
+ set(FLIGHT_GENERATED_PROTO_FILES
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.h"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.h")
Review Comment:
Why do you want to use `build/...` in source directory instead of `${CMAKE_CURRENT_BINARY_DIR}` or something?
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,467 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+
+# normalize ARROW_HOME path
+cmake_path(CONVERT "$ENV{ARROW_HOME}" TO_CMAKE_PATH_LIST ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+find_package(Python3Alt 3.7 REQUIRED)
+
+# Needed due to
+# CMake Error: INSTALL(EXPORT) given unknown export "arrow_python_targets"
+option(ARROW_BUILD_SHARED "Link to the Arrow shared library" ON)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+if(ARROW_USE_XSIMD)
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_INSTALL_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS} # Defined in line 95
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
+ include_directories("${ARROW_SOURCE_DIR}/src" "${ARROW_SOURCE_DIR}/build/src")
+
+ # Using arrow_flight_shared, see example
+ # https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt
+ set(ARROW_GRPC_USE_SHARED ON)
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,--no-as-needed gRPC::grpc++_reflection
+ -Wl,--as-needed)
+
+ set(FLIGHT_PROTO_PATH "${ARROW_SOURCE}/format")
+ set(FLIGHT_PROTO "${ARROW_SOURCE}/format/Flight.proto")
+
+ set(FLIGHT_GENERATED_PROTO_FILES
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.h"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.h")
+
+ set(PROTO_DEPENDS ${FLIGHT_PROTO} gRPC::grpc_cpp_plugin)
+
+ add_custom_command(OUTPUT ${FLIGHT_GENERATED_PROTO_FILES}
+ COMMAND ${ARROW_PROTOBUF_PROTOC} "-I${FLIGHT_PROTO_PATH}"
+ "--cpp_out=${${ARROW_SOURCE_DIR}/build/src/arrow/flight/}" "${FLIGHT_PROTO}"
Review Comment:
Is `${${...}...}` intentional?
##########
cpp/src/arrow/public_api_test.cc:
##########
@@ -50,9 +50,9 @@
#include "arrow/json/api.h" // IWYU pragma: keep
#endif
-#ifdef ARROW_PYTHON
-#include "arrow/python/api.h" // IWYU pragma: keep
-#endif
+// #ifdef ARROW_PYTHON
+// #include "arrow/python/api.h" // IWYU pragma: keep
+// #endif
Review Comment:
We can remove this.
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,467 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+
+# normalize ARROW_HOME path
+cmake_path(CONVERT "$ENV{ARROW_HOME}" TO_CMAKE_PATH_LIST ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+find_package(Python3Alt 3.7 REQUIRED)
+
+# Needed due to
+# CMake Error: INSTALL(EXPORT) given unknown export "arrow_python_targets"
+option(ARROW_BUILD_SHARED "Link to the Arrow shared library" ON)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+if(ARROW_USE_XSIMD)
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_INSTALL_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS} # Defined in line 95
Review Comment:
"95" is outdated... :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r911783775
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,467 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+
+# normalize ARROW_HOME path
+cmake_path(CONVERT "$ENV{ARROW_HOME}" TO_CMAKE_PATH_LIST ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+find_package(Python3Alt 3.7 REQUIRED)
+
+# Needed due to
+# CMake Error: INSTALL(EXPORT) given unknown export "arrow_python_targets"
+option(ARROW_BUILD_SHARED "Link to the Arrow shared library" ON)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+if(ARROW_USE_XSIMD)
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_INSTALL_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS} # Defined in line 95
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
+ include_directories("${ARROW_SOURCE_DIR}/src" "${ARROW_SOURCE_DIR}/build/src")
+
+ # Using arrow_flight_shared, see example
+ # https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt
+ set(ARROW_GRPC_USE_SHARED ON)
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,--no-as-needed gRPC::grpc++_reflection
+ -Wl,--as-needed)
+
+ set(FLIGHT_PROTO_PATH "${ARROW_SOURCE}/format")
+ set(FLIGHT_PROTO "${ARROW_SOURCE}/format/Flight.proto")
+
+ set(FLIGHT_GENERATED_PROTO_FILES
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.h"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.h")
Review Comment:
The use of `ARROW_BUILD_DIR` CMake variable (that you suggested) made this nicer, I think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1173534025
Is there something similar I can do with this part:
https://github.com/apache/arrow/blob/38918ef0d59387895759223bee2d704c293bcfb9/cpp/src/arrow/python/flight.cc#L385-L393
that also uses `arrow::flight::internal::SchemaToString`? @lidavidm
I have successfully changed the construction of `FlightInfo` for `Status CreateFlightInfo` in `python/flight.cc`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r911747666
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,467 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+
+# normalize ARROW_HOME path
+cmake_path(CONVERT "$ENV{ARROW_HOME}" TO_CMAKE_PATH_LIST ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+find_package(Python3Alt 3.7 REQUIRED)
+
+# Needed due to
+# CMake Error: INSTALL(EXPORT) given unknown export "arrow_python_targets"
+option(ARROW_BUILD_SHARED "Link to the Arrow shared library" ON)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+if(ARROW_USE_XSIMD)
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_INSTALL_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS} # Defined in line 95
Review Comment:
Yeah, will remove the comment :D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r918624037
##########
python/setup.py:
##########
@@ -227,6 +228,94 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_arrow_python(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
+ raise ValueError("--build-type (or PYARROW_BUILD_TYPE) needs to "
+ "be 'release' or 'debug'")
+
+ # The directory containing this setup.py
+ source = os.path.dirname(os.path.abspath(__file__))
+ # The directory containing this C PyArrow CMakeLists.txt
+ source_cpyarrow = pjoin(source, "pyarrow/src_arrow")
+
+ # The directory for the module being built
+ build_cmd = self.get_finalized_command('build')
+ saved_cwd = os.getcwd()
+ build_temp = pjoin(saved_cwd, 'build/dist/temp')
+ build_include = pjoin(saved_cwd, 'build/dist/include')
+ build_lib = pjoin(os.getcwd(), build_cmd.build_lib)
+
+ # The directory containing Arrow C++ build
+ arrow_build_dir = os.environ.get('ARROW_BUILD_DIR', 'build')
+
+ if self.inplace:
+ # a bit hacky
+ build_lib = saved_cwd
+
+ if not os.path.isdir(build_temp):
+ self.mkpath(build_temp)
+ if not os.path.isdir(build_lib):
+ self.mkpath(build_lib)
+ if not os.path.isdir(build_include):
+ self.mkpath(build_include)
+
+ # Change to the build directory
+ with changed_dir(build_temp):
+ # cmake args
+ cmake_options = [
+ '-DCMAKE_INSTALL_PREFIX=' +
+ str(pjoin(saved_cwd, 'build/dist')),
+ '-DCMAKE_BUILD_TYPE={0}'.format(self.build_type.lower()),
+ '-DARROW_BUILD_DIR=' + str(arrow_build_dir),
+ '-DPYTHON_EXECUTABLE=%s' % sys.executable,
+ '-DPython3_EXECUTABLE=%s' % sys.executable,
+ ]
+
+ # Check for specific options
+ def append_cmake_bool(value, varname):
+ cmake_options.append('-D{0}={1}'.format(
+ varname, 'on' if value else 'off'))
+
+ append_cmake_bool(self.with_dataset, 'PYARROW_WITH_DATASET')
+ append_cmake_bool(self.with_parquet_encryption,
+ 'PYARROW_WITH_PARQUET_ENCRYPTION')
+ append_cmake_bool(self.with_hdfs,
+ 'PYARROW_WITH_HDFS')
+
+ # run cmake
+ print("-- Running cmake for C pyarrow")
+ self.spawn(['cmake'] + cmake_options + [source_cpyarrow])
+ print("-- Finished cmake for C pyarrow")
+ # run make & install
+ print("-- Running make build and install for C pyarrow")
+ self.spawn(['make', '-j4'])
Review Comment:
(I believe that's `cmake --build`?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r918614322
##########
python/setup.py:
##########
@@ -245,6 +334,10 @@ def _run_cmake(self):
if not os.path.isdir(build_temp):
self.mkpath(build_temp)
+ if self.inplace:
+ # a bit hacky
+ build_lib = saved_cwd
Review Comment:
This is now repeated a bit below (line 429), so it can be removed there
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r921934176
##########
cpp/cmake_modules/FindArrowPython.cmake:
##########
@@ -46,8 +46,9 @@ endif()
find_package(Arrow ${find_package_arguments})
if(ARROW_FOUND)
+ message(STATUS "CPYARROW_HOME: ${CPYARROW_HOME}")
arrow_find_package(ARROW_PYTHON
- "${ARROW_HOME}"
+ "${CPYARROW_HOME}"
Review Comment:
PYARROW_CPP sounds good to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r921990917
##########
python/pyarrow/tests/test_gdb.py:
##########
@@ -154,7 +154,7 @@ def select_frame(self, func_name):
# but it's not available on old GDB versions (such as 8.1.1),
# so instead parse the stack trace for a matching frame number.
out = self.run_command("info stack")
- pat = r"(?mi)^#(\d+)\s+.* in " + re.escape(func_name) + " "
+ pat = r"(?mi)^#(\d+)\s+.* in " + re.escape(func_name)
Review Comment:
I see. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] amol- commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
amol- commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r894518980
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -50,6 +79,14 @@ set(ARROW_PYTHON_SRCS
set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
SKIP_UNITY_BUILD_INCLUSION ON)
+# Set necessary components on (from arrow/cpp/CMakeLists.txt, end of the file)
+set(ARROW_COMPUTE ON)
+set(ARROW_CSV ON)
+set(ARROW_DATASET ON)
+set(ARROW_FILESYSTEM ON)
+set(ARROW_HDFS ON)
+set(ARROW_JSON ON)
Review Comment:
is this temporary? We shouldn't be forcefully enabling those as they are optional and PyArrow can be build without things like DATASETS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1174977797
@lidavidm I have changed the code in `flight.cc`
https://github.com/apache/arrow/pull/13311/files#diff-4b08b8c4f93b3a02a872894b39e2d35e60af11669f4dcad6c212f85a41343e95
`flight/types.h`
https://github.com/apache/arrow/pull/13311/files#diff-8b9efe1f8818751a9c88f12fe6c81138bfa673f965bbf8ee781353fc834fd49c
and `flight/types.cc`
https://github.com/apache/arrow/pull/13311/files#diff-b4f0e7ff0c555ddec5cb880a8b636cdcce25453ec3fb998d0026f2b8bd5625f7
Do you think this is OK?
Asking as I guess there is something wrong which causes Fatal Python error when running `pytest`:
https://github.com/apache/arrow/runs/7194408350?check_suite_focus=true
<details>
<summary>Error output</summary>
<br>
```
Thread 0x00007f90800b6740 (most recent call first):
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/tests/test_flight.py", line 917 in test_flight_list_flights
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/_pytest/python.py", line 192 in pytest_pyfunc_call
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pluggy/_callers.py", line 39 in _multicall
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pluggy/_manager.py", line 80 in _hookexec
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pluggy/_hooks.py", line 265 in __call__
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/_pytest/python.py", line 1761 in runtest
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/_pytest/runner.py", line 166 in pytest_runtest_call
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pluggy/_callers.py", line 39 in _multicall
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pluggy/_manager.py", line 80 in _hookexec
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pluggy/_hooks.py", line 265 in __call__
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/_pytest/runner.py", line 259 in <lambda>
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/_pytest/runner.py", line 338 in from_call
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/_pytest/runner.py", line 259 in call_runtest_hook
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/_pytest/runner.py", line 219 in call_and_report
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/_pytest/runner.py", line 130 in runtestprotocol
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/_pytest/runner.py", line 111 in pytest_runtest_protocol
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pluggy/_callers.py", line 39 in _multicall
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pluggy/_manager.py", line 80 in _hookexec
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pluggy/_hooks.py", line 265 in __call__
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/_pytest/main.py", line 347 in pytest_runtestloop
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pluggy/_callers.py", line 39 in _multicall
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pluggy/_manager.py", line 80 in _hookexec
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pluggy/_hooks.py", line 265 in __call__
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/_pytest/main.py", line 322 in _main
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/_pytest/main.py", line 268 in wrap_session
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/_pytest/main.py", line 315 in pytest_cmdline_main
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pluggy/_callers.py", line 39 in _multicall
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pluggy/_manager.py", line 80 in _hookexec
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pluggy/_hooks.py", line 265 in __call__
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/_pytest/config/__init__.py", line 165 in main
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/_pytest/config/__init__.py", line 187 in console_main
File "/opt/conda/envs/arrow/bin/pytest", line 11 in <module>
/arrow/ci/scripts/python_test.sh: line 57: 12744 Segmentation fault
```
</details>
Do you maybe have an idea what could be wrong from first glance?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
lidavidm commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r916814844
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,573 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+
+# compiler flags for different build types (run 'cmake -DCMAKE_BUILD_TYPE=<type> .')
+if(ARROW_GGDB_DEBUG)
+ set(ARROW_DEBUG_SYMBOL_TYPE "gdb")
+ set(C_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(C_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+ set(CXX_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+else()
+ set(C_FLAGS_DEBUG "-g -O0")
+ set(C_FLAGS_FASTDEBUG "-g -O1")
+ set(CXX_FLAGS_DEBUG "-g -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g -O1")
+endif()
+
+set(C_FLAGS_RELEASE "-O3 -DNDEBUG")
+set(CXX_FLAGS_RELEASE "-O3 -DNDEBUG")
+
+set(C_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(C_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+set(CXX_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(CXX_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+
+string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE)
+# Set compile flags based on the build type.
+message(STATUS "Configured for ${CMAKE_BUILD_TYPE} build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})"
+)
+
+if("${CMAKE_BUILD_TYPE}" STREQUAL "DEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_DEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_DEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELWITHDEBINFO")
+
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "FASTDEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_FASTDEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_FASTDEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELEASE")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_RELEASE}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_RELEASE}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_GEN")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_GEN}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_GEN}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_BUILD")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_BUILD}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_BUILD}")
+else()
+ message(FATAL_ERROR "Unknown build type: ${CMAKE_BUILD_TYPE}")
+endif()
+
+message(STATUS "Build Type: ${CMAKE_BUILD_TYPE}")
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+
+if((NOT ARROW_SIMD_LEVEL STREQUAL "NONE") OR (NOT ARROW_RUNTIME_SIMD_LEVEL STREQUAL "NONE"
+ ))
+ set(ARROW_USE_XSIMD TRUE)
+else()
+ set(ARROW_USE_XSIMD FALSE)
+endif()
+
+if(ARROW_USE_XSIMD)
+ set(XSIMD_PREFIX "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/xsimd_ep/src/xsimd_ep-install")
+ if(EXISTS ${XSIMD_PREFIX}/include)
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ else()
+ set(XSIMD_PREFIX "$ENV{CONDA_PREFIX}")
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ endif()
+
+ add_library(xsimd INTERFACE IMPORTED)
+ if(CMAKE_VERSION VERSION_LESS 3.11)
+ set_target_properties(xsimd PROPERTIES INTERFACE_INCLUDE_DIRECTORIES
+ "${XSIMD_INCLUDE_DIR}")
+ else()
+ target_include_directories(xsimd INTERFACE "${XSIMD_INCLUDE_DIR}")
+ endif()
+
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS}
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ # Static gRPC means we cannot link to shared Arrow, since then
+ # we'll violate ODR for gRPC symbols
+ if(NOT ARROW_GRPC_USE_SHARED AND NOT ARROW_BUILD_STATIC)
+ message(FATAL_ERROR "Statically built gRPC requires ARROW_BUILD_STATIC=ON")
+ endif()
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
+ include_directories("${ARROW_CPP_SOURCE_DIR}/src" "${ARROW_CPP_SOURCE_DIR}/}/${ARROW_BUILD_DIR}/src")
+
+ # Using example
+ # https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt
+
+ if(ARROW_GRPC_USE_SHARED)
+ set(FLIGHT_EXAMPLES_LINK_LIBS arrow_flight_shared)
+ # We don't directly use symbols from the reflection library, so
+ # ensure the linker still links to it
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,--no-as-needed gRPC::grpc++_reflection
+ -Wl,--as-needed)
Review Comment:
```suggestion
```
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,573 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+
+# compiler flags for different build types (run 'cmake -DCMAKE_BUILD_TYPE=<type> .')
+if(ARROW_GGDB_DEBUG)
+ set(ARROW_DEBUG_SYMBOL_TYPE "gdb")
+ set(C_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(C_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+ set(CXX_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+else()
+ set(C_FLAGS_DEBUG "-g -O0")
+ set(C_FLAGS_FASTDEBUG "-g -O1")
+ set(CXX_FLAGS_DEBUG "-g -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g -O1")
+endif()
+
+set(C_FLAGS_RELEASE "-O3 -DNDEBUG")
+set(CXX_FLAGS_RELEASE "-O3 -DNDEBUG")
+
+set(C_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(C_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+set(CXX_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(CXX_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+
+string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE)
+# Set compile flags based on the build type.
+message(STATUS "Configured for ${CMAKE_BUILD_TYPE} build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})"
+)
+
+if("${CMAKE_BUILD_TYPE}" STREQUAL "DEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_DEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_DEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELWITHDEBINFO")
+
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "FASTDEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_FASTDEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_FASTDEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELEASE")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_RELEASE}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_RELEASE}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_GEN")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_GEN}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_GEN}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_BUILD")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_BUILD}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_BUILD}")
+else()
+ message(FATAL_ERROR "Unknown build type: ${CMAKE_BUILD_TYPE}")
+endif()
+
+message(STATUS "Build Type: ${CMAKE_BUILD_TYPE}")
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+
+if((NOT ARROW_SIMD_LEVEL STREQUAL "NONE") OR (NOT ARROW_RUNTIME_SIMD_LEVEL STREQUAL "NONE"
+ ))
+ set(ARROW_USE_XSIMD TRUE)
+else()
+ set(ARROW_USE_XSIMD FALSE)
+endif()
+
+if(ARROW_USE_XSIMD)
+ set(XSIMD_PREFIX "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/xsimd_ep/src/xsimd_ep-install")
+ if(EXISTS ${XSIMD_PREFIX}/include)
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ else()
+ set(XSIMD_PREFIX "$ENV{CONDA_PREFIX}")
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ endif()
+
+ add_library(xsimd INTERFACE IMPORTED)
+ if(CMAKE_VERSION VERSION_LESS 3.11)
+ set_target_properties(xsimd PROPERTIES INTERFACE_INCLUDE_DIRECTORIES
+ "${XSIMD_INCLUDE_DIR}")
+ else()
+ target_include_directories(xsimd INTERFACE "${XSIMD_INCLUDE_DIR}")
+ endif()
+
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS}
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ # Static gRPC means we cannot link to shared Arrow, since then
+ # we'll violate ODR for gRPC symbols
+ if(NOT ARROW_GRPC_USE_SHARED AND NOT ARROW_BUILD_STATIC)
+ message(FATAL_ERROR "Statically built gRPC requires ARROW_BUILD_STATIC=ON")
+ endif()
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
+ include_directories("${ARROW_CPP_SOURCE_DIR}/src" "${ARROW_CPP_SOURCE_DIR}/}/${ARROW_BUILD_DIR}/src")
+
+ # Using example
+ # https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt
+
+ if(ARROW_GRPC_USE_SHARED)
+ set(FLIGHT_EXAMPLES_LINK_LIBS arrow_flight_shared)
+ # We don't directly use symbols from the reflection library, so
+ # ensure the linker still links to it
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,--no-as-needed gRPC::grpc++_reflection
+ -Wl,--as-needed)
+ else()
+ set(FLIGHT_EXAMPLES_LINK_LIBS arrow_flight_static)
+ if(APPLE)
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,-force_load gRPC::grpc++_reflection)
+ else()
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,--whole-archive gRPC::grpc++_reflection
+ -Wl,--no-whole-archive)
+ endif()
Review Comment:
```suggestion
```
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,573 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+
+# compiler flags for different build types (run 'cmake -DCMAKE_BUILD_TYPE=<type> .')
+if(ARROW_GGDB_DEBUG)
+ set(ARROW_DEBUG_SYMBOL_TYPE "gdb")
+ set(C_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(C_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+ set(CXX_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+else()
+ set(C_FLAGS_DEBUG "-g -O0")
+ set(C_FLAGS_FASTDEBUG "-g -O1")
+ set(CXX_FLAGS_DEBUG "-g -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g -O1")
+endif()
+
+set(C_FLAGS_RELEASE "-O3 -DNDEBUG")
+set(CXX_FLAGS_RELEASE "-O3 -DNDEBUG")
+
+set(C_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(C_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+set(CXX_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(CXX_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+
+string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE)
+# Set compile flags based on the build type.
+message(STATUS "Configured for ${CMAKE_BUILD_TYPE} build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})"
+)
+
+if("${CMAKE_BUILD_TYPE}" STREQUAL "DEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_DEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_DEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELWITHDEBINFO")
+
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "FASTDEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_FASTDEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_FASTDEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELEASE")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_RELEASE}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_RELEASE}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_GEN")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_GEN}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_GEN}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_BUILD")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_BUILD}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_BUILD}")
+else()
+ message(FATAL_ERROR "Unknown build type: ${CMAKE_BUILD_TYPE}")
+endif()
+
+message(STATUS "Build Type: ${CMAKE_BUILD_TYPE}")
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+
+if((NOT ARROW_SIMD_LEVEL STREQUAL "NONE") OR (NOT ARROW_RUNTIME_SIMD_LEVEL STREQUAL "NONE"
+ ))
+ set(ARROW_USE_XSIMD TRUE)
+else()
+ set(ARROW_USE_XSIMD FALSE)
+endif()
+
+if(ARROW_USE_XSIMD)
+ set(XSIMD_PREFIX "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/xsimd_ep/src/xsimd_ep-install")
+ if(EXISTS ${XSIMD_PREFIX}/include)
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ else()
+ set(XSIMD_PREFIX "$ENV{CONDA_PREFIX}")
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ endif()
+
+ add_library(xsimd INTERFACE IMPORTED)
+ if(CMAKE_VERSION VERSION_LESS 3.11)
+ set_target_properties(xsimd PROPERTIES INTERFACE_INCLUDE_DIRECTORIES
+ "${XSIMD_INCLUDE_DIR}")
+ else()
+ target_include_directories(xsimd INTERFACE "${XSIMD_INCLUDE_DIR}")
+ endif()
+
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS}
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ # Static gRPC means we cannot link to shared Arrow, since then
+ # we'll violate ODR for gRPC symbols
+ if(NOT ARROW_GRPC_USE_SHARED AND NOT ARROW_BUILD_STATIC)
+ message(FATAL_ERROR "Statically built gRPC requires ARROW_BUILD_STATIC=ON")
+ endif()
Review Comment:
Don't think we need this (as explained below)
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,573 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+
+# compiler flags for different build types (run 'cmake -DCMAKE_BUILD_TYPE=<type> .')
+if(ARROW_GGDB_DEBUG)
+ set(ARROW_DEBUG_SYMBOL_TYPE "gdb")
+ set(C_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(C_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+ set(CXX_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+else()
+ set(C_FLAGS_DEBUG "-g -O0")
+ set(C_FLAGS_FASTDEBUG "-g -O1")
+ set(CXX_FLAGS_DEBUG "-g -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g -O1")
+endif()
+
+set(C_FLAGS_RELEASE "-O3 -DNDEBUG")
+set(CXX_FLAGS_RELEASE "-O3 -DNDEBUG")
+
+set(C_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(C_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+set(CXX_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(CXX_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+
+string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE)
+# Set compile flags based on the build type.
+message(STATUS "Configured for ${CMAKE_BUILD_TYPE} build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})"
+)
+
+if("${CMAKE_BUILD_TYPE}" STREQUAL "DEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_DEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_DEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELWITHDEBINFO")
+
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "FASTDEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_FASTDEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_FASTDEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELEASE")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_RELEASE}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_RELEASE}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_GEN")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_GEN}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_GEN}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_BUILD")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_BUILD}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_BUILD}")
+else()
+ message(FATAL_ERROR "Unknown build type: ${CMAKE_BUILD_TYPE}")
+endif()
+
+message(STATUS "Build Type: ${CMAKE_BUILD_TYPE}")
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+
+if((NOT ARROW_SIMD_LEVEL STREQUAL "NONE") OR (NOT ARROW_RUNTIME_SIMD_LEVEL STREQUAL "NONE"
+ ))
+ set(ARROW_USE_XSIMD TRUE)
+else()
+ set(ARROW_USE_XSIMD FALSE)
+endif()
+
+if(ARROW_USE_XSIMD)
+ set(XSIMD_PREFIX "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/xsimd_ep/src/xsimd_ep-install")
+ if(EXISTS ${XSIMD_PREFIX}/include)
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ else()
+ set(XSIMD_PREFIX "$ENV{CONDA_PREFIX}")
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ endif()
+
+ add_library(xsimd INTERFACE IMPORTED)
+ if(CMAKE_VERSION VERSION_LESS 3.11)
+ set_target_properties(xsimd PROPERTIES INTERFACE_INCLUDE_DIRECTORIES
+ "${XSIMD_INCLUDE_DIR}")
+ else()
+ target_include_directories(xsimd INTERFACE "${XSIMD_INCLUDE_DIR}")
+ endif()
+
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS}
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
Review Comment:
Just `if(ARROW_FLIGHT)`?
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,573 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+
+# compiler flags for different build types (run 'cmake -DCMAKE_BUILD_TYPE=<type> .')
+if(ARROW_GGDB_DEBUG)
+ set(ARROW_DEBUG_SYMBOL_TYPE "gdb")
+ set(C_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(C_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+ set(CXX_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+else()
+ set(C_FLAGS_DEBUG "-g -O0")
+ set(C_FLAGS_FASTDEBUG "-g -O1")
+ set(CXX_FLAGS_DEBUG "-g -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g -O1")
+endif()
+
+set(C_FLAGS_RELEASE "-O3 -DNDEBUG")
+set(CXX_FLAGS_RELEASE "-O3 -DNDEBUG")
+
+set(C_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(C_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+set(CXX_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(CXX_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+
+string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE)
+# Set compile flags based on the build type.
+message(STATUS "Configured for ${CMAKE_BUILD_TYPE} build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})"
+)
+
+if("${CMAKE_BUILD_TYPE}" STREQUAL "DEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_DEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_DEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELWITHDEBINFO")
+
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "FASTDEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_FASTDEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_FASTDEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELEASE")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_RELEASE}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_RELEASE}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_GEN")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_GEN}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_GEN}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_BUILD")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_BUILD}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_BUILD}")
+else()
+ message(FATAL_ERROR "Unknown build type: ${CMAKE_BUILD_TYPE}")
+endif()
+
+message(STATUS "Build Type: ${CMAKE_BUILD_TYPE}")
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+
+if((NOT ARROW_SIMD_LEVEL STREQUAL "NONE") OR (NOT ARROW_RUNTIME_SIMD_LEVEL STREQUAL "NONE"
+ ))
+ set(ARROW_USE_XSIMD TRUE)
+else()
+ set(ARROW_USE_XSIMD FALSE)
+endif()
+
+if(ARROW_USE_XSIMD)
+ set(XSIMD_PREFIX "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/xsimd_ep/src/xsimd_ep-install")
+ if(EXISTS ${XSIMD_PREFIX}/include)
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ else()
+ set(XSIMD_PREFIX "$ENV{CONDA_PREFIX}")
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ endif()
+
+ add_library(xsimd INTERFACE IMPORTED)
+ if(CMAKE_VERSION VERSION_LESS 3.11)
+ set_target_properties(xsimd PROPERTIES INTERFACE_INCLUDE_DIRECTORIES
+ "${XSIMD_INCLUDE_DIR}")
+ else()
+ target_include_directories(xsimd INTERFACE "${XSIMD_INCLUDE_DIR}")
+ endif()
+
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS}
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ # Static gRPC means we cannot link to shared Arrow, since then
+ # we'll violate ODR for gRPC symbols
+ if(NOT ARROW_GRPC_USE_SHARED AND NOT ARROW_BUILD_STATIC)
+ message(FATAL_ERROR "Statically built gRPC requires ARROW_BUILD_STATIC=ON")
+ endif()
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
Review Comment:
Should be no need for us to link to gRPC
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,573 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+
+# compiler flags for different build types (run 'cmake -DCMAKE_BUILD_TYPE=<type> .')
+if(ARROW_GGDB_DEBUG)
+ set(ARROW_DEBUG_SYMBOL_TYPE "gdb")
+ set(C_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(C_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+ set(CXX_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+else()
+ set(C_FLAGS_DEBUG "-g -O0")
+ set(C_FLAGS_FASTDEBUG "-g -O1")
+ set(CXX_FLAGS_DEBUG "-g -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g -O1")
+endif()
+
+set(C_FLAGS_RELEASE "-O3 -DNDEBUG")
+set(CXX_FLAGS_RELEASE "-O3 -DNDEBUG")
+
+set(C_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(C_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+set(CXX_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(CXX_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+
+string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE)
+# Set compile flags based on the build type.
+message(STATUS "Configured for ${CMAKE_BUILD_TYPE} build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})"
+)
+
+if("${CMAKE_BUILD_TYPE}" STREQUAL "DEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_DEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_DEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELWITHDEBINFO")
+
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "FASTDEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_FASTDEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_FASTDEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELEASE")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_RELEASE}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_RELEASE}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_GEN")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_GEN}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_GEN}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_BUILD")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_BUILD}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_BUILD}")
+else()
+ message(FATAL_ERROR "Unknown build type: ${CMAKE_BUILD_TYPE}")
+endif()
+
+message(STATUS "Build Type: ${CMAKE_BUILD_TYPE}")
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+
+if((NOT ARROW_SIMD_LEVEL STREQUAL "NONE") OR (NOT ARROW_RUNTIME_SIMD_LEVEL STREQUAL "NONE"
+ ))
+ set(ARROW_USE_XSIMD TRUE)
+else()
+ set(ARROW_USE_XSIMD FALSE)
+endif()
+
+if(ARROW_USE_XSIMD)
+ set(XSIMD_PREFIX "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/xsimd_ep/src/xsimd_ep-install")
+ if(EXISTS ${XSIMD_PREFIX}/include)
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ else()
+ set(XSIMD_PREFIX "$ENV{CONDA_PREFIX}")
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ endif()
+
+ add_library(xsimd INTERFACE IMPORTED)
+ if(CMAKE_VERSION VERSION_LESS 3.11)
+ set_target_properties(xsimd PROPERTIES INTERFACE_INCLUDE_DIRECTORIES
+ "${XSIMD_INCLUDE_DIR}")
+ else()
+ target_include_directories(xsimd INTERFACE "${XSIMD_INCLUDE_DIR}")
+ endif()
+
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS}
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ # Static gRPC means we cannot link to shared Arrow, since then
+ # we'll violate ODR for gRPC symbols
+ if(NOT ARROW_GRPC_USE_SHARED AND NOT ARROW_BUILD_STATIC)
+ message(FATAL_ERROR "Statically built gRPC requires ARROW_BUILD_STATIC=ON")
+ endif()
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
+ include_directories("${ARROW_CPP_SOURCE_DIR}/src" "${ARROW_CPP_SOURCE_DIR}/}/${ARROW_BUILD_DIR}/src")
+
+ # Using example
+ # https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt
+
+ if(ARROW_GRPC_USE_SHARED)
+ set(FLIGHT_EXAMPLES_LINK_LIBS arrow_flight_shared)
+ # We don't directly use symbols from the reflection library, so
+ # ensure the linker still links to it
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,--no-as-needed gRPC::grpc++_reflection
+ -Wl,--as-needed)
+ else()
+ set(FLIGHT_EXAMPLES_LINK_LIBS arrow_flight_static)
+ if(APPLE)
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,-force_load gRPC::grpc++_reflection)
+ else()
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,--whole-archive gRPC::grpc++_reflection
+ -Wl,--no-whole-archive)
+ endif()
+ endif()
+
+ set(FLIGHT_PROTO_PATH "${ARROW_SOURCE}/format")
+ set(FLIGHT_PROTO "${ARROW_SOURCE}/format/Flight.proto")
+
+ set(FLIGHT_GENERATED_PROTO_FILES
+ "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/src/arrow/flight/Flight.pb.cc"
+ "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/src/arrow/flight/Flight.pb.h"
+ "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/src/arrow/flight/Flight.grpc.pb.cc"
+ "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/src/arrow/flight/Flight.grpc.pb.h")
+
+ set(PROTO_DEPENDS ${FLIGHT_PROTO} gRPC::grpc_cpp_plugin)
+
+ add_custom_command(OUTPUT ${FLIGHT_GENERATED_PROTO_FILES}
+ COMMAND ${ARROW_PROTOBUF_PROTOC} "-I${FLIGHT_PROTO_PATH}"
+ "--cpp_out=${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/src/arrow/flight/"
+ "${FLIGHT_PROTO}"
+ DEPENDS ${PROTO_DEPENDS} ARGS
+ COMMAND ${ARROW_PROTOBUF_PROTOC} "-I${FLIGHT_PROTO_PATH}"
+ "--grpc_out=${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/src/arrow/flight/"
+ "--plugin=protoc-gen-grpc=$<TARGET_FILE:gRPC::grpc_cpp_plugin>"
+ "${FLIGHT_PROTO}")
+
+ set_source_files_properties(${FLIGHT_GENERATED_PROTO_FILES} PROPERTIES GENERATED TRUE)
+ add_custom_target(flight_grpc_gen ALL DEPENDS ${FLIGHT_GENERATED_PROTO_FILES})
+
+ add_arrow_lib(arrow_python_flight
+ CMAKE_PACKAGE_NAME
+ ArrowPythonFlight
+ PKG_CONFIG_NAME
+ arrow-python-flight
+ SOURCES
+ flight.cc
+ OUTPUTS
+ ARROW_PYFLIGHT_LIBRARIES
+ DEPENDENCIES
+ flight_grpc_gen
Review Comment:
```suggestion
```
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,573 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+
+# compiler flags for different build types (run 'cmake -DCMAKE_BUILD_TYPE=<type> .')
+if(ARROW_GGDB_DEBUG)
+ set(ARROW_DEBUG_SYMBOL_TYPE "gdb")
+ set(C_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(C_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+ set(CXX_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+else()
+ set(C_FLAGS_DEBUG "-g -O0")
+ set(C_FLAGS_FASTDEBUG "-g -O1")
+ set(CXX_FLAGS_DEBUG "-g -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g -O1")
+endif()
+
+set(C_FLAGS_RELEASE "-O3 -DNDEBUG")
+set(CXX_FLAGS_RELEASE "-O3 -DNDEBUG")
+
+set(C_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(C_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+set(CXX_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(CXX_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+
+string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE)
+# Set compile flags based on the build type.
+message(STATUS "Configured for ${CMAKE_BUILD_TYPE} build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})"
+)
+
+if("${CMAKE_BUILD_TYPE}" STREQUAL "DEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_DEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_DEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELWITHDEBINFO")
+
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "FASTDEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_FASTDEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_FASTDEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELEASE")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_RELEASE}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_RELEASE}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_GEN")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_GEN}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_GEN}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_BUILD")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_BUILD}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_BUILD}")
+else()
+ message(FATAL_ERROR "Unknown build type: ${CMAKE_BUILD_TYPE}")
+endif()
+
+message(STATUS "Build Type: ${CMAKE_BUILD_TYPE}")
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+
+if((NOT ARROW_SIMD_LEVEL STREQUAL "NONE") OR (NOT ARROW_RUNTIME_SIMD_LEVEL STREQUAL "NONE"
+ ))
+ set(ARROW_USE_XSIMD TRUE)
+else()
+ set(ARROW_USE_XSIMD FALSE)
+endif()
+
+if(ARROW_USE_XSIMD)
+ set(XSIMD_PREFIX "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/xsimd_ep/src/xsimd_ep-install")
+ if(EXISTS ${XSIMD_PREFIX}/include)
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ else()
+ set(XSIMD_PREFIX "$ENV{CONDA_PREFIX}")
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ endif()
+
+ add_library(xsimd INTERFACE IMPORTED)
+ if(CMAKE_VERSION VERSION_LESS 3.11)
+ set_target_properties(xsimd PROPERTIES INTERFACE_INCLUDE_DIRECTORIES
+ "${XSIMD_INCLUDE_DIR}")
+ else()
+ target_include_directories(xsimd INTERFACE "${XSIMD_INCLUDE_DIR}")
+ endif()
+
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS}
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ # Static gRPC means we cannot link to shared Arrow, since then
+ # we'll violate ODR for gRPC symbols
+ if(NOT ARROW_GRPC_USE_SHARED AND NOT ARROW_BUILD_STATIC)
+ message(FATAL_ERROR "Statically built gRPC requires ARROW_BUILD_STATIC=ON")
+ endif()
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
+ include_directories("${ARROW_CPP_SOURCE_DIR}/src" "${ARROW_CPP_SOURCE_DIR}/}/${ARROW_BUILD_DIR}/src")
+
+ # Using example
+ # https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt
+
+ if(ARROW_GRPC_USE_SHARED)
+ set(FLIGHT_EXAMPLES_LINK_LIBS arrow_flight_shared)
Review Comment:
this should probably be renamed something like `FLIGHT_LINK_LIBS` since this isn't an example
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,573 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+
+# compiler flags for different build types (run 'cmake -DCMAKE_BUILD_TYPE=<type> .')
+if(ARROW_GGDB_DEBUG)
+ set(ARROW_DEBUG_SYMBOL_TYPE "gdb")
+ set(C_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(C_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+ set(CXX_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+else()
+ set(C_FLAGS_DEBUG "-g -O0")
+ set(C_FLAGS_FASTDEBUG "-g -O1")
+ set(CXX_FLAGS_DEBUG "-g -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g -O1")
+endif()
+
+set(C_FLAGS_RELEASE "-O3 -DNDEBUG")
+set(CXX_FLAGS_RELEASE "-O3 -DNDEBUG")
+
+set(C_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(C_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+set(CXX_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(CXX_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+
+string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE)
+# Set compile flags based on the build type.
+message(STATUS "Configured for ${CMAKE_BUILD_TYPE} build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})"
+)
+
+if("${CMAKE_BUILD_TYPE}" STREQUAL "DEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_DEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_DEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELWITHDEBINFO")
+
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "FASTDEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_FASTDEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_FASTDEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELEASE")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_RELEASE}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_RELEASE}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_GEN")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_GEN}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_GEN}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_BUILD")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_BUILD}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_BUILD}")
+else()
+ message(FATAL_ERROR "Unknown build type: ${CMAKE_BUILD_TYPE}")
+endif()
+
+message(STATUS "Build Type: ${CMAKE_BUILD_TYPE}")
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+
+if((NOT ARROW_SIMD_LEVEL STREQUAL "NONE") OR (NOT ARROW_RUNTIME_SIMD_LEVEL STREQUAL "NONE"
+ ))
+ set(ARROW_USE_XSIMD TRUE)
+else()
+ set(ARROW_USE_XSIMD FALSE)
+endif()
+
+if(ARROW_USE_XSIMD)
+ set(XSIMD_PREFIX "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/xsimd_ep/src/xsimd_ep-install")
+ if(EXISTS ${XSIMD_PREFIX}/include)
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ else()
+ set(XSIMD_PREFIX "$ENV{CONDA_PREFIX}")
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ endif()
+
+ add_library(xsimd INTERFACE IMPORTED)
+ if(CMAKE_VERSION VERSION_LESS 3.11)
+ set_target_properties(xsimd PROPERTIES INTERFACE_INCLUDE_DIRECTORIES
+ "${XSIMD_INCLUDE_DIR}")
+ else()
+ target_include_directories(xsimd INTERFACE "${XSIMD_INCLUDE_DIR}")
+ endif()
+
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS}
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ # Static gRPC means we cannot link to shared Arrow, since then
+ # we'll violate ODR for gRPC symbols
+ if(NOT ARROW_GRPC_USE_SHARED AND NOT ARROW_BUILD_STATIC)
+ message(FATAL_ERROR "Statically built gRPC requires ARROW_BUILD_STATIC=ON")
+ endif()
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
+ include_directories("${ARROW_CPP_SOURCE_DIR}/src" "${ARROW_CPP_SOURCE_DIR}/}/${ARROW_BUILD_DIR}/src")
+
+ # Using example
+ # https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt
+
+ if(ARROW_GRPC_USE_SHARED)
Review Comment:
This should just be `if(ARROW_BUILD_SHARED)` I think
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,573 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+
+# compiler flags for different build types (run 'cmake -DCMAKE_BUILD_TYPE=<type> .')
+if(ARROW_GGDB_DEBUG)
+ set(ARROW_DEBUG_SYMBOL_TYPE "gdb")
+ set(C_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(C_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+ set(CXX_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+else()
+ set(C_FLAGS_DEBUG "-g -O0")
+ set(C_FLAGS_FASTDEBUG "-g -O1")
+ set(CXX_FLAGS_DEBUG "-g -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g -O1")
+endif()
+
+set(C_FLAGS_RELEASE "-O3 -DNDEBUG")
+set(CXX_FLAGS_RELEASE "-O3 -DNDEBUG")
+
+set(C_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(C_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+set(CXX_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(CXX_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+
+string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE)
+# Set compile flags based on the build type.
+message(STATUS "Configured for ${CMAKE_BUILD_TYPE} build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})"
+)
+
+if("${CMAKE_BUILD_TYPE}" STREQUAL "DEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_DEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_DEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELWITHDEBINFO")
+
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "FASTDEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_FASTDEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_FASTDEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELEASE")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_RELEASE}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_RELEASE}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_GEN")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_GEN}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_GEN}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_BUILD")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_BUILD}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_BUILD}")
+else()
+ message(FATAL_ERROR "Unknown build type: ${CMAKE_BUILD_TYPE}")
+endif()
+
+message(STATUS "Build Type: ${CMAKE_BUILD_TYPE}")
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+
+if((NOT ARROW_SIMD_LEVEL STREQUAL "NONE") OR (NOT ARROW_RUNTIME_SIMD_LEVEL STREQUAL "NONE"
+ ))
+ set(ARROW_USE_XSIMD TRUE)
+else()
+ set(ARROW_USE_XSIMD FALSE)
+endif()
+
+if(ARROW_USE_XSIMD)
+ set(XSIMD_PREFIX "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/xsimd_ep/src/xsimd_ep-install")
+ if(EXISTS ${XSIMD_PREFIX}/include)
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ else()
+ set(XSIMD_PREFIX "$ENV{CONDA_PREFIX}")
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ endif()
+
+ add_library(xsimd INTERFACE IMPORTED)
+ if(CMAKE_VERSION VERSION_LESS 3.11)
+ set_target_properties(xsimd PROPERTIES INTERFACE_INCLUDE_DIRECTORIES
+ "${XSIMD_INCLUDE_DIR}")
+ else()
+ target_include_directories(xsimd INTERFACE "${XSIMD_INCLUDE_DIR}")
+ endif()
+
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS}
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ # Static gRPC means we cannot link to shared Arrow, since then
+ # we'll violate ODR for gRPC symbols
+ if(NOT ARROW_GRPC_USE_SHARED AND NOT ARROW_BUILD_STATIC)
+ message(FATAL_ERROR "Statically built gRPC requires ARROW_BUILD_STATIC=ON")
+ endif()
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
+ include_directories("${ARROW_CPP_SOURCE_DIR}/src" "${ARROW_CPP_SOURCE_DIR}/}/${ARROW_BUILD_DIR}/src")
+
+ # Using example
+ # https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt
+
+ if(ARROW_GRPC_USE_SHARED)
+ set(FLIGHT_EXAMPLES_LINK_LIBS arrow_flight_shared)
+ # We don't directly use symbols from the reflection library, so
+ # ensure the linker still links to it
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,--no-as-needed gRPC::grpc++_reflection
+ -Wl,--as-needed)
+ else()
+ set(FLIGHT_EXAMPLES_LINK_LIBS arrow_flight_static)
+ if(APPLE)
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,-force_load gRPC::grpc++_reflection)
+ else()
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,--whole-archive gRPC::grpc++_reflection
+ -Wl,--no-whole-archive)
+ endif()
+ endif()
+
+ set(FLIGHT_PROTO_PATH "${ARROW_SOURCE}/format")
+ set(FLIGHT_PROTO "${ARROW_SOURCE}/format/Flight.proto")
+
+ set(FLIGHT_GENERATED_PROTO_FILES
+ "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/src/arrow/flight/Flight.pb.cc"
+ "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/src/arrow/flight/Flight.pb.h"
+ "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/src/arrow/flight/Flight.grpc.pb.cc"
+ "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/src/arrow/flight/Flight.grpc.pb.h")
+
+ set(PROTO_DEPENDS ${FLIGHT_PROTO} gRPC::grpc_cpp_plugin)
+
+ add_custom_command(OUTPUT ${FLIGHT_GENERATED_PROTO_FILES}
+ COMMAND ${ARROW_PROTOBUF_PROTOC} "-I${FLIGHT_PROTO_PATH}"
+ "--cpp_out=${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/src/arrow/flight/"
+ "${FLIGHT_PROTO}"
+ DEPENDS ${PROTO_DEPENDS} ARGS
+ COMMAND ${ARROW_PROTOBUF_PROTOC} "-I${FLIGHT_PROTO_PATH}"
+ "--grpc_out=${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/src/arrow/flight/"
+ "--plugin=protoc-gen-grpc=$<TARGET_FILE:gRPC::grpc_cpp_plugin>"
+ "${FLIGHT_PROTO}")
+
+ set_source_files_properties(${FLIGHT_GENERATED_PROTO_FILES} PROPERTIES GENERATED TRUE)
+ add_custom_target(flight_grpc_gen ALL DEPENDS ${FLIGHT_GENERATED_PROTO_FILES})
Review Comment:
None of this should be needed - we don't need to regenerate the Protobuf sources here.
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,573 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+
+# compiler flags for different build types (run 'cmake -DCMAKE_BUILD_TYPE=<type> .')
+if(ARROW_GGDB_DEBUG)
+ set(ARROW_DEBUG_SYMBOL_TYPE "gdb")
+ set(C_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(C_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+ set(CXX_FLAGS_DEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g${ARROW_DEBUG_SYMBOL_TYPE} -O1")
+else()
+ set(C_FLAGS_DEBUG "-g -O0")
+ set(C_FLAGS_FASTDEBUG "-g -O1")
+ set(CXX_FLAGS_DEBUG "-g -O0")
+ set(CXX_FLAGS_FASTDEBUG "-g -O1")
+endif()
+
+set(C_FLAGS_RELEASE "-O3 -DNDEBUG")
+set(CXX_FLAGS_RELEASE "-O3 -DNDEBUG")
+
+set(C_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(C_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+set(CXX_FLAGS_PROFILE_GEN "${CXX_FLAGS_RELEASE} -fprofile-generate")
+set(CXX_FLAGS_PROFILE_BUILD "${CXX_FLAGS_RELEASE} -fprofile-use")
+
+string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE)
+# Set compile flags based on the build type.
+message(STATUS "Configured for ${CMAKE_BUILD_TYPE} build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})"
+)
+
+if("${CMAKE_BUILD_TYPE}" STREQUAL "DEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_DEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_DEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELWITHDEBINFO")
+
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "FASTDEBUG")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_FASTDEBUG}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_FASTDEBUG}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "RELEASE")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_RELEASE}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_RELEASE}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_GEN")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_GEN}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_GEN}")
+elseif("${CMAKE_BUILD_TYPE}" STREQUAL "PROFILE_BUILD")
+ set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${C_FLAGS_PROFILE_BUILD}")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${CXX_FLAGS_PROFILE_BUILD}")
+else()
+ message(FATAL_ERROR "Unknown build type: ${CMAKE_BUILD_TYPE}")
+endif()
+
+message(STATUS "Build Type: ${CMAKE_BUILD_TYPE}")
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_CPP_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+
+if((NOT ARROW_SIMD_LEVEL STREQUAL "NONE") OR (NOT ARROW_RUNTIME_SIMD_LEVEL STREQUAL "NONE"
+ ))
+ set(ARROW_USE_XSIMD TRUE)
+else()
+ set(ARROW_USE_XSIMD FALSE)
+endif()
+
+if(ARROW_USE_XSIMD)
+ set(XSIMD_PREFIX "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/xsimd_ep/src/xsimd_ep-install")
+ if(EXISTS ${XSIMD_PREFIX}/include)
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ else()
+ set(XSIMD_PREFIX "$ENV{CONDA_PREFIX}")
+ set(XSIMD_INCLUDE_DIR "${XSIMD_PREFIX}/include")
+ endif()
+
+ add_library(xsimd INTERFACE IMPORTED)
+ if(CMAKE_VERSION VERSION_LESS 3.11)
+ set_target_properties(xsimd PROPERTIES INTERFACE_INCLUDE_DIRECTORIES
+ "${XSIMD_INCLUDE_DIR}")
+ else()
+ target_include_directories(xsimd INTERFACE "${XSIMD_INCLUDE_DIR}")
+ endif()
+
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS}
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ # Static gRPC means we cannot link to shared Arrow, since then
+ # we'll violate ODR for gRPC symbols
+ if(NOT ARROW_GRPC_USE_SHARED AND NOT ARROW_BUILD_STATIC)
+ message(FATAL_ERROR "Statically built gRPC requires ARROW_BUILD_STATIC=ON")
+ endif()
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
+ include_directories("${ARROW_CPP_SOURCE_DIR}/src" "${ARROW_CPP_SOURCE_DIR}/}/${ARROW_BUILD_DIR}/src")
+
+ # Using example
+ # https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt
Review Comment:
The example is a little overcomplicated because it's trying to link in gRPC and Flight and you have to be careful there. But if you are just trying to link Flight then you don't need to do anything 'special'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1179973704
Removing `flight_grpc_gen` from the dependencies makes the whole flight part much much easier, thank you @lidavidm!
Just for the record, I was building on top of the existing code that links to gRPC:
https://github.com/apache/arrow/blob/88b42ef66fe664043c5ee5274b2982a3858b414e/cpp/src/arrow/python/CMakeLists.txt#L124-L151
Due to the note in this section I linked python flight only to shared flight library and also kept `if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)`. Let me know if you still think this should be changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
raulcd commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1183003834
@github-actions crossbow submit *python*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1184565708
@github-actions crossbow submit python-sdist
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r921833223
##########
cpp/cmake_modules/FindArrowPython.cmake:
##########
@@ -46,8 +46,9 @@ endif()
find_package(Arrow ${find_package_arguments})
if(ARROW_FOUND)
+ message(STATUS "CPYARROW_HOME: ${CPYARROW_HOME}")
arrow_find_package(ARROW_PYTHON
- "${ARROW_HOME}"
+ "${CPYARROW_HOME}"
Review Comment:
How about `ARROW_PYTHON_HOME` because we use `ArrowPython` as package name?
##########
docs/source/developers/python.rst:
##########
@@ -131,6 +131,29 @@ for ``.py`` files or
for ``.pyx`` and ``.pxi`` files. In this case you will also need to
install the `pytest-cython <https://github.com/lgpage/pytest-cython>`_ plugin.
+Testing Arrow Python (C++ code)
+-------------------------------
+If you want to run ctest for the tests that are included in the Arrow Python
+module, you will need to build Arrow with ``DARROW_BUILD_TESTS=ON``.
Review Comment:
```suggestion
module, you will need to build Arrow with ``-DARROW_BUILD_TESTS=ON``.
```
##########
cpp/cmake_modules/FindArrowPythonFlight.cmake:
##########
@@ -50,7 +50,7 @@ find_package(ArrowPython ${find_package_arguments})
if(ARROW_PYTHON_FOUND AND ARROW_FLIGHT_FOUND)
arrow_find_package(ARROW_PYTHON_FLIGHT
- "${ARROW_HOME}"
+ "${CPYARROW_HOME}"
Review Comment:
```suggestion
"${ARROW_PYTHON_HOME}"
```
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
Review Comment:
We need to update this in release process by `dev/release/utils-prepare.sh`.
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+include(SetupCxxFlags)
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+# Changing ARROW_SOURCE_DIR for sdist build
+# In this case cpp/cmake_modules doesn't exist
+if(NOT EXISTS "${ARROW_SOURCE_DIR}/cmake_modules/Find${MODULE}.cmake")
+ set(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR})
+endif()
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS}
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+ # Must link to shared libarrow_flight: we don't want to link more than one
+ # copy of gRPC into the eventual Cython shared object, otherwise gRPC calls
+ # fail with weird errors due to multiple copies of global static state (The
+ # other solution is to link gRPC shared everywhere instead of statically only
+ # in Flight)
+ find_package(ArrowFlight REQUIRED)
+ include_directories("${ARROW_CPP_SOURCE_DIR}/src" "${ARROW_CPP_SOURCE_DIR}/}/${ARROW_BUILD_DIR}/src")
Review Comment:
Could you use `EXTRA_INCLUDES` in the below `add_arrow_lib()` instead of global `include_directories()`?
BTW, why is this needed? I think `arrow_flight_shared` adds needed include directories.
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+include(SetupCxxFlags)
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+# Changing ARROW_SOURCE_DIR for sdist build
+# In this case cpp/cmake_modules doesn't exist
+if(NOT EXISTS "${ARROW_SOURCE_DIR}/cmake_modules/Find${MODULE}.cmake")
+ set(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR})
+endif()
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS}
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+ # Must link to shared libarrow_flight: we don't want to link more than one
+ # copy of gRPC into the eventual Cython shared object, otherwise gRPC calls
+ # fail with weird errors due to multiple copies of global static state (The
+ # other solution is to link gRPC shared everywhere instead of statically only
+ # in Flight)
+ find_package(ArrowFlight REQUIRED)
+ include_directories("${ARROW_CPP_SOURCE_DIR}/src" "${ARROW_CPP_SOURCE_DIR}/}/${ARROW_BUILD_DIR}/src")
+
+ set(FLIGHT_LINK_LIBS arrow_flight_shared)
+
+ add_arrow_lib(arrow_python_flight
+ CMAKE_PACKAGE_NAME
+ ArrowPythonFlight
+ PKG_CONFIG_NAME
+ arrow-python-flight
+ SOURCES
+ flight.cc
+ OUTPUTS
+ ARROW_PYFLIGHT_LIBRARIES
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS}
+ SHARED_LINK_LIBS
+ arrow_python_shared
+ arrow_flight_shared
+ STATIC_LINK_LIBS
+ ${PYTHON_OTHER_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}"
+ PRIVATE_INCLUDES
+ "${Protobuf_INCLUDE_DIRS}")
+
+ add_dependencies(arrow_python ${ARROW_PYFLIGHT_LIBRARIES})
+
+ foreach(LIB_TARGET ${ARROW_PYFLIGHT_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYFLIGHT_EXPORTING)
+ endforeach()
+
+ if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_flight_static PUBLIC ARROW_STATIC)
+ endif()
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ # Clang, be quiet. Python C API has lots of macros
+ set_property(SOURCE ${ARROW_PYTHON_SRCS}
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS -Wno-parentheses-equality)
+endif()
+
+arrow_install_all_headers("arrow/python")
+
+# ----------------------------------------------------------------------
+
+#
+# Tests
+# The tests will be moved to Cython and are currently supported for bundled GTest
+# Follow-up: https://issues.apache.org/jira/browse/ARROW-17016?filter=-1
Review Comment:
```suggestion
# Follow-up: https://issues.apache.org/jira/browse/ARROW-17016
```
##########
python/setup.py:
##########
@@ -227,6 +228,105 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_arrow_python(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
+ raise ValueError("--build-type (or PYARROW_BUILD_TYPE) needs to "
+ "be 'release' or 'debug'")
+
+ # The directory containing this setup.py
+ source = os.path.dirname(os.path.abspath(__file__))
+ # The directory containing this C PyArrow CMakeLists.txt
+ source_cpyarrow = pjoin(source, "pyarrow/src_arrow")
+
+ # The directory for the module being built
+ build_cmd = self.get_finalized_command('build')
+ saved_cwd = os.getcwd()
+ build_temp = pjoin(saved_cwd, 'build', 'dist', 'temp')
+ build_include = pjoin(saved_cwd, 'build', 'dist', 'include')
+ build_lib = pjoin(os.getcwd(), build_cmd.build_lib)
+
+ # The directory containing Arrow C++ build
+ arrow_build_dir = os.environ.get('ARROW_BUILD_DIR', 'build')
+
+ if self.inplace:
+ # a bit hacky
+ build_lib = saved_cwd
+
+ if not os.path.isdir(build_temp):
+ self.mkpath(build_temp)
+ if not os.path.isdir(build_lib):
+ self.mkpath(build_lib)
+ if not os.path.isdir(build_include):
+ self.mkpath(build_include)
+
+ # Change to the build directory
+ with changed_dir(build_temp):
+ # cmake args
+ cmake_options = [
+ '-DCMAKE_INSTALL_PREFIX=' +
+ str(pjoin(saved_cwd, 'build/dist')),
+ '-DCMAKE_BUILD_TYPE={0}'.format(self.build_type.lower()),
+ '-DARROW_BUILD_DIR=' + str(arrow_build_dir),
+ '-DPYTHON_EXECUTABLE=%s' % sys.executable,
+ '-DPython3_EXECUTABLE=%s' % sys.executable,
+ ]
+
+ # Check for specific options
+ def append_cmake_bool(value, varname):
+ cmake_options.append('-D{0}={1}'.format(
+ varname, 'on' if value else 'off'))
+
+ append_cmake_bool(self.with_dataset, 'PYARROW_WITH_DATASET')
+ append_cmake_bool(self.with_parquet_encryption,
+ 'PYARROW_WITH_PARQUET_ENCRYPTION')
+ append_cmake_bool(self.with_hdfs,
+ 'PYARROW_WITH_HDFS')
+
+ # Windows
+ if self.cmake_generator:
+ cmake_options += ['-G', self.cmake_generator]
+
+ # build args
+ build_tool_args = []
+ if os.environ.get('PYARROW_PARALLEL'):
+ build_tool_args.append('--')
+ build_tool_args.append(
+ '-j{0}'.format(os.environ['PYARROW_PARALLEL']))
+
+ # run cmake
+ print("-- Running cmake for arrow python")
+ self.spawn(['cmake'] + cmake_options + [source_cpyarrow])
+ print("-- Finished cmake for arrow python")
+ # run make & install
+ print("-- Running make build and install for arrow python")
Review Comment:
```suggestion
print("-- Running cmake build and install for arrow python")
```
##########
docs/source/developers/python.rst:
##########
@@ -131,6 +131,29 @@ for ``.py`` files or
for ``.pyx`` and ``.pxi`` files. In this case you will also need to
install the `pytest-cython <https://github.com/lgpage/pytest-cython>`_ plugin.
+Testing Arrow Python (C++ code)
+-------------------------------
+If you want to run ctest for the tests that are included in the Arrow Python
+module, you will need to build Arrow with ``DARROW_BUILD_TESTS=ON``.
+
+.. note::
+
+ Currently building the C++ unit tests does not work with googletest
+ from conda-forge, so we must use the ``BUNDLED`` source for building that
+ dependency
+
+ In case you use conda add ``-DGTest_SOURCE=BUNDLED`` to the cmake flags
Review Comment:
```suggestion
In case you use conda add ``-DGTest_SOURCE=BUNDLED`` to the CMake flags
```
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
Review Comment:
```suggestion
get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
```
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
Review Comment:
Why do we need to set `Python3_FIND_*` variables? I think that they should be specified by users if they need.
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
Review Comment:
```suggestion
# Check all the options from Arrow and ArrowPython to be in line
```
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
Review Comment:
Can we remove this because we have `ARROW_PYTHON_INCLUDES`?
##########
python/pyarrow/tests/test_gdb.py:
##########
@@ -154,7 +154,7 @@ def select_frame(self, func_name):
# but it's not available on old GDB versions (such as 8.1.1),
# so instead parse the stack trace for a matching frame number.
out = self.run_command("info stack")
- pat = r"(?mi)^#(\d+)\s+.* in " + re.escape(func_name) + " "
+ pat = r"(?mi)^#(\d+)\s+.* in " + re.escape(func_name)
Review Comment:
Why is this change needed?
##########
python/setup.py:
##########
@@ -227,6 +228,105 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_arrow_python(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
+ raise ValueError("--build-type (or PYARROW_BUILD_TYPE) needs to "
+ "be 'release' or 'debug'")
+
+ # The directory containing this setup.py
+ source = os.path.dirname(os.path.abspath(__file__))
+ # The directory containing this C PyArrow CMakeLists.txt
+ source_cpyarrow = pjoin(source, "pyarrow/src_arrow")
+
+ # The directory for the module being built
+ build_cmd = self.get_finalized_command('build')
+ saved_cwd = os.getcwd()
+ build_temp = pjoin(saved_cwd, 'build', 'dist', 'temp')
+ build_include = pjoin(saved_cwd, 'build', 'dist', 'include')
+ build_lib = pjoin(os.getcwd(), build_cmd.build_lib)
+
+ # The directory containing Arrow C++ build
+ arrow_build_dir = os.environ.get('ARROW_BUILD_DIR', 'build')
+
+ if self.inplace:
+ # a bit hacky
+ build_lib = saved_cwd
+
+ if not os.path.isdir(build_temp):
+ self.mkpath(build_temp)
+ if not os.path.isdir(build_lib):
+ self.mkpath(build_lib)
+ if not os.path.isdir(build_include):
+ self.mkpath(build_include)
+
+ # Change to the build directory
+ with changed_dir(build_temp):
+ # cmake args
+ cmake_options = [
+ '-DCMAKE_INSTALL_PREFIX=' +
+ str(pjoin(saved_cwd, 'build/dist')),
+ '-DCMAKE_BUILD_TYPE={0}'.format(self.build_type.lower()),
+ '-DARROW_BUILD_DIR=' + str(arrow_build_dir),
+ '-DPYTHON_EXECUTABLE=%s' % sys.executable,
+ '-DPython3_EXECUTABLE=%s' % sys.executable,
+ ]
+
+ # Check for specific options
+ def append_cmake_bool(value, varname):
+ cmake_options.append('-D{0}={1}'.format(
+ varname, 'on' if value else 'off'))
+
+ append_cmake_bool(self.with_dataset, 'PYARROW_WITH_DATASET')
+ append_cmake_bool(self.with_parquet_encryption,
+ 'PYARROW_WITH_PARQUET_ENCRYPTION')
+ append_cmake_bool(self.with_hdfs,
+ 'PYARROW_WITH_HDFS')
+
+ # Windows
+ if self.cmake_generator:
+ cmake_options += ['-G', self.cmake_generator]
+
+ # build args
+ build_tool_args = []
+ if os.environ.get('PYARROW_PARALLEL'):
+ build_tool_args.append('--')
+ build_tool_args.append(
+ '-j{0}'.format(os.environ['PYARROW_PARALLEL']))
+
+ # run cmake
+ print("-- Running cmake for arrow python")
+ self.spawn(['cmake'] + cmake_options + [source_cpyarrow])
+ print("-- Finished cmake for arrow python")
+ # run make & install
+ print("-- Running make build and install for arrow python")
+ self.spawn(['cmake', '--build', '.', '--target', 'install'] +
+ build_tool_args)
+ print("-- Finished make build and install for arrow python")
Review Comment:
```suggestion
print("-- Finished cmake build and install for arrow python")
```
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
Review Comment:
```suggestion
# Arrow vs ArrowPython options
```
##########
python/setup.py:
##########
@@ -227,6 +228,105 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_arrow_python(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
+ raise ValueError("--build-type (or PYARROW_BUILD_TYPE) needs to "
+ "be 'release' or 'debug'")
+
+ # The directory containing this setup.py
+ source = os.path.dirname(os.path.abspath(__file__))
+ # The directory containing this C PyArrow CMakeLists.txt
Review Comment:
```suggestion
# The directory containing this ArrowPython CMakeLists.txt
```
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
Review Comment:
Why do we need this alias?
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
Review Comment:
How about `ARROW_PYTHON_SOURCE_DIR`?
```suggestion
get_filename_component(ARROW_PYTHON_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
get_filename_component(PYTHON_SOURCE_DIR ${ARROW_PYTHON_SOURCE_DIR} DIRECTORY)
```
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+include(SetupCxxFlags)
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
Review Comment:
It seems that `PARQUET_REQUIRE_ENCRYPTION` is redundant:
```suggestion
if(PYARROW_WITH_PARQUET_ENCRYPTION)
```
##########
ci/scripts/python_test.sh:
##########
@@ -54,4 +55,11 @@ export PYARROW_TEST_ORC
export PYARROW_TEST_PARQUET
export PYARROW_TEST_S3
+# Testing Arrow Python
+if [ "${ARROW_BUILD_TESTS}" == "ON" ]; then
+ pushd ${test_dir}
+ ctest
Review Comment:
```suggestion
ctest --output-on-failure --timeout 300
```
I want to add `--parallel ${n_jobs}` too like `ci/scripts/cpp_test.sh`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1184041988
@github-actions crossbow submit python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1184422044
@github-actions crossbow submit python-sdist
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1191996473
Revision: 8947536858335b567d1c1790f1a82fe2152e13a9
Submitted crossbow builds: [ursacomputing/crossbow @ actions-d00562d0b2](https://github.com/ursacomputing/crossbow/branches/all?query=actions-d00562d0b2)
|Task|Status|
|----|------|
|example-python-minimal-build-fedora-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-example-python-minimal-build-fedora-conda)|
|example-python-minimal-build-ubuntu-venv|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-example-python-minimal-build-ubuntu-venv)|
|python-sdist|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-python-sdist)|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.10)|
|test-conda-python-3.7|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.7)|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.7-hdfs-3.2.1)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-0.24|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.7-pandas-0.24)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-spark-v3.1.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.7-spark-v3.1.2)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.8)|
|test-conda-python-3.8-hypothesis|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.8-hypothesis)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.8-pandas-nightly)|
|test-conda-python-3.8-spark-v3.2.0|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.8-spark-v3.2.0)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.9)|
|test-conda-python-3.9-dask-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.9-dask-latest)|
|test-conda-python-3.9-dask-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.9-dask-master)|
|test-conda-python-3.9-pandas-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.9-pandas-master)|
|test-conda-python-3.9-spark-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-test-conda-python-3.9-spark-master)|
|test-debian-11-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-d00562d0b2-azure-test-debian-11-python-3)|
|test-fedora-35-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-d00562d0b2-azure-test-fedora-35-python-3)|
|test-ubuntu-20.04-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-d00562d0b2-azure-test-ubuntu-20.04-python-3)|
|verify-rc-source-python-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-verify-rc-source-python-linux-almalinux-8-amd64)|
|verify-rc-source-python-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-verify-rc-source-python-linux-conda-latest-amd64)|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
|verify-rc-source-python-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-verify-rc-source-python-linux-ubuntu-20.04-amd64)|
|verify-rc-source-python-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-verify-rc-source-python-linux-ubuntu-22.04-amd64)|
|verify-rc-source-python-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-verify-rc-source-python-macos-amd64)|
|verify-rc-source-python-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-verify-rc-source-python-macos-arm64)|
|verify-rc-source-python-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-d00562d0b2-github-verify-rc-source-python-macos-conda-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926292425
##########
python/setup.py:
##########
@@ -354,6 +468,19 @@ def append_cmake_bool(value, varname):
shutil.move(pjoin(build_prefix, 'include'),
pjoin(build_lib, 'pyarrow'))
+ # We need to, again, add the PyArrow cpp include folder
+ build_pyarrow_cpp_include = pjoin(
+ saved_cwd, 'build/dist/include')
+ if not os.path.isdir(
+ pjoin(build_pyarrow_cpp_include, "arrow", "python")):
+ self.mkpath(
Review Comment:
Hope I understood correctly, the changes I made (together with updated comment from https://github.com/apache/arrow/pull/13311#discussion_r926243308):
```python
- # We need to, again, add the PyArrow cpp include folder
+ # As the pyarrow/include file has been deleted in the previous step
+ # we need to, again, add the PyArrow cpp include folder also
build_pyarrow_cpp_include = pjoin(
saved_cwd, 'build/dist/include')
- if not os.path.isdir(
- pjoin(build_pyarrow_cpp_include, "arrow", "python")):
- self.mkpath(
- pjoin(build_pyarrow_cpp_include, "arrow", "python"))
shutil.move(pjoin(
build_pyarrow_cpp_include, "arrow", "python"),
pjoin(
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1191092857
@github-actions crossbow submit verify-rc-source-python-linux-ubuntu-18.04-amd64
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r930333095
##########
python/setup.py:
##########
@@ -227,6 +228,118 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_pyarrow_cpp(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
+ raise ValueError("--build-type (or PYARROW_BUILD_TYPE) needs to "
+ "be 'release' or 'debug'")
+
+ # The directory containing this setup.py
+ source = os.path.dirname(os.path.abspath(__file__))
+ # The directory containing this PyArrow cpp CMakeLists.txt
+ source_pyarrow_cpp = pjoin(source, "pyarrow/src")
+
+ # The directory for the module being built
+ build_cmd = self.get_finalized_command('build')
+ saved_cwd = os.getcwd()
+ build_dir = pjoin(saved_cwd, 'build', 'dist')
+ build_include = pjoin(saved_cwd, 'build', 'dist', 'include')
+ build_lib = pjoin(os.getcwd(), build_cmd.build_lib)
+
+ # The directory containing Arrow C++ build
+ arrow_build_dir = os.environ.get('ARROW_BUILD_DIR', 'build')
+ if self.inplace:
+ # a bit hacky
+ build_lib = saved_cwd
+ if not os.path.isdir(build_dir):
+ self.mkpath(build_dir)
+ if not os.path.isdir(build_lib):
+ self.mkpath(build_lib)
+ if not os.path.isdir(build_include):
+ self.mkpath(build_include)
+
+ # Change to the build directory
+ with changed_dir(build_dir):
+ # cmake args
+ cmake_options = [
+ '-DCMAKE_INSTALL_PREFIX=' +
+ str(pjoin(saved_cwd, 'build/dist')),
+ '-DCMAKE_BUILD_TYPE={0}'.format(self.build_type.lower()),
+ '-DARROW_BUILD_DIR=' + str(arrow_build_dir),
+ '-DPYTHON_EXECUTABLE=%s' % sys.executable,
+ '-DPython3_EXECUTABLE=%s' % sys.executable,
+ ]
+
+ # Check for specific options
+ def append_cmake_bool(value, varname):
+ cmake_options.append('-D{0}={1}'.format(
+ varname, 'on' if value else 'off'))
+
+ append_cmake_bool(self.with_dataset, 'PYARROW_WITH_DATASET')
+ append_cmake_bool(self.with_parquet_encryption,
+ 'PYARROW_WITH_PARQUET_ENCRYPTION')
+ append_cmake_bool(self.with_hdfs,
+ 'PYARROW_WITH_HDFS')
+
+ # Windows
+ if self.cmake_generator:
+ cmake_options += ['-G', self.cmake_generator]
+
+ # build args
+ build_tool_args = []
+ if os.environ.get('PYARROW_PARALLEL'):
+ build_tool_args.append('--')
+ build_tool_args.append(
+ '-j{0}'.format(os.environ['PYARROW_PARALLEL']))
+
+ # run cmake
+ print("-- Running cmake for pyarrow cpp")
+ self.spawn(['cmake'] + cmake_options + [source_pyarrow_cpp])
+ print("-- Finished cmake for pyarrow cpp")
+ # run make & install
+ print("-- Running cmake build and install for pyarrow cpp")
+ self.spawn(['cmake', '--build', '.', '--target', 'install'] +
+ build_tool_args)
+ print("-- Finished cmake build and install for pyarrow cpp")
+
+ # Move the libraries to the place expected by the Python build
+ try:
+ os.makedirs(pjoin(build_lib, 'pyarrow'))
+ except OSError:
+ pass
+
+ # helper function
+ def copy_libs(folder_name):
+ for libname in os.listdir(pjoin(build_dir, folder_name)):
+ if "python" in libname:
+ libname_path = pjoin(build_lib, "pyarrow", libname)
+ if os.path.exists(libname_path):
+ os.remove(libname_path)
+ print(
+ f"Copying {pjoin(build_dir, folder_name, libname)}"
+ f" to {pjoin(build_lib, 'pyarrow', libname)}")
+ shutil.copy(pjoin(build_dir, folder_name, libname),
+ pjoin(build_lib, "pyarrow"))
+
+ # Move libraries to python/pyarrow
+ # For windows builds, move dll from bin
+ for folder in ['lib', 'lib64', 'bin']:
+ try:
+ copy_libs(folder)
+ except OSError:
Review Comment:
Oh, sorry, I missed this comment. Yes, I will try 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r922967854
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
Review Comment:
I added these lines together with `include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)` in https://github.com/apache/arrow/pull/13311/commits/28f2b536027783e45ac05575e2468a2a47846254 to fix a CI fail due to `Could NOT find Python3` due to missing `NumPy` module. I suspected all these lines of code are needed as they are included in arrow/python `CMakeLists.txt` also:
https://github.com/apache/arrow/blob/ca204f541626e69f722047729ac76bd1446665da/python/CMakeLists.txt#L42-L47
https://github.com/apache/arrow/blob/ca204f541626e69f722047729ac76bd1446665da/python/CMakeLists.txt#L217-L221
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926198302
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
Review Comment:
CMake's project name isn't related to the final library name. It's safe to change this without breaking compatibility.
I think that all `python/pyarrow/src/` are rewritten in Cython eventually. So `libarrow_python.so` will be gone. (I think that we don't need to rename it.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926431607
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs PyArrow cpp options
+#
+
+# Check all the options from Arrow and PyArrow cpp to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+include(SetupCxxFlags)
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
Review Comment:
AFAIU in the PyArrow cpp `parquet_shared` is only needed if Parquet encryption is enabled (because of `parquet_encryption.cc`). At least that is how it is currently done:
https://github.com/apache/arrow/blob/9c422a2011404ee0c5c01eeb2a6a1d5333816cad/cpp/src/arrow/python/CMakeLists.txt#L80-L82
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1168527583
@kou there is one more issue I am struggling with and would be happy to get your opinion.
On the CI I get linker errors for python flight module (not locally on M1):
https://github.com/apache/arrow/runs/7087113174?check_suite_focus=true
I am facing similar issue locally when trying to build the tests for C++ part of PyArrow with GTest:
```
[100%] Linking CXX executable arrow-python-test
ld: library not found for -larrow_testing_shared
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [arrow-python-test] Error 1
make[1]: *** [CMakeFiles/arrow-python-test.dir/all] Error 2
make: *** [all] Error 2
error: command '/usr/bin/make' failed with exit code 2
~/repos/arrow-cmake
```
What could be the setup work I am missing so that the libraries would be linked correctly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
pitrou commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1172175875
I mean in general, though that could be done as part of this PR (unless I'm missing something that makes internal headers mandatory here, but that would be surprising).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1220513437
@github-actions crossbow submit *wheel*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1223687082
Thank you @raulcd for the feedback! 🙏
I am looking at the failures in the nightlies, thanks for running them @kou. Some of them seem to be related to this PR, will add a summary shortly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1184463352
Revision: b19a0b20fba6644ec41d5ea934fc8fbadf5509c7
Submitted crossbow builds: [ursacomputing/crossbow @ actions-e0ace050a0](https://github.com/ursacomputing/crossbow/branches/all?query=actions-e0ace050a0)
|Task|Status|
|----|------|
|python-sdist|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-e0ace050a0-github-python-sdist)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r921925091
##########
cpp/cmake_modules/FindArrowPython.cmake:
##########
@@ -46,8 +46,9 @@ endif()
find_package(Arrow ${find_package_arguments})
if(ARROW_FOUND)
+ message(STATUS "CPYARROW_HOME: ${CPYARROW_HOME}")
arrow_find_package(ARROW_PYTHON
- "${ARROW_HOME}"
+ "${CPYARROW_HOME}"
Review Comment:
How about being explicit, so e.g. `PYARROW_CPP_HOME`? `CPYARROW` isn't very descriptive (could be some Cython code, or some CPython-specific code).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r921928949
##########
cpp/cmake_modules/FindArrowPython.cmake:
##########
@@ -46,8 +46,9 @@ endif()
find_package(Arrow ${find_package_arguments})
if(ARROW_FOUND)
+ message(STATUS "CPYARROW_HOME: ${CPYARROW_HOME}")
arrow_find_package(ARROW_PYTHON
- "${ARROW_HOME}"
+ "${CPYARROW_HOME}"
Review Comment:
Hmm. If others like this name, I'm OK with this. But I think that at least `CPP` is better than `C`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1184044973
Revision: 3d7f9b320bc2f2a10b82b990c9efd368c646cace
Submitted crossbow builds: [ursacomputing/crossbow @ actions-3e57291765](https://github.com/ursacomputing/crossbow/branches/all?query=actions-3e57291765)
|Task|Status|
|----|------|
|example-python-minimal-build-fedora-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-example-python-minimal-build-fedora-conda)|
|example-python-minimal-build-ubuntu-venv|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-example-python-minimal-build-ubuntu-venv)|
|python-sdist|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-python-sdist)|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.10)|
|test-conda-python-3.7|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.7)|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.7-hdfs-3.2.1)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-0.24|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.7-pandas-0.24)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-spark-v3.1.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.7-spark-v3.1.2)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.8)|
|test-conda-python-3.8-hypothesis|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.8-hypothesis)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.8-pandas-nightly)|
|test-conda-python-3.8-spark-v3.2.0|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.8-spark-v3.2.0)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.9)|
|test-conda-python-3.9-dask-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.9-dask-latest)|
|test-conda-python-3.9-dask-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.9-dask-master)|
|test-conda-python-3.9-pandas-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.9-pandas-master)|
|test-conda-python-3.9-spark-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-test-conda-python-3.9-spark-master)|
|test-debian-11-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-3e57291765-azure-test-debian-11-python-3)|
|test-fedora-35-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-3e57291765-azure-test-fedora-35-python-3)|
|test-ubuntu-20.04-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-3e57291765-azure-test-ubuntu-20.04-python-3)|
|verify-rc-source-python-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-verify-rc-source-python-linux-almalinux-8-amd64)|
|verify-rc-source-python-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-verify-rc-source-python-linux-conda-latest-amd64)|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
|verify-rc-source-python-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-verify-rc-source-python-linux-ubuntu-20.04-amd64)|
|verify-rc-source-python-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-verify-rc-source-python-linux-ubuntu-22.04-amd64)|
|verify-rc-source-python-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-verify-rc-source-python-macos-amd64)|
|verify-rc-source-python-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-verify-rc-source-python-macos-arm64)|
|verify-rc-source-python-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e57291765-github-verify-rc-source-python-macos-conda-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1184042696
```
Unable to match any tasks for `python`
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/2668349951
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r911755346
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,467 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+
+# normalize ARROW_HOME path
+cmake_path(CONVERT "$ENV{ARROW_HOME}" TO_CMAKE_PATH_LIST ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+find_package(Python3Alt 3.7 REQUIRED)
+
+# Needed due to
+# CMake Error: INSTALL(EXPORT) given unknown export "arrow_python_targets"
+option(ARROW_BUILD_SHARED "Link to the Arrow shared library" ON)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+if(ARROW_USE_XSIMD)
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_INSTALL_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS} # Defined in line 95
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
+ include_directories("${ARROW_SOURCE_DIR}/src" "${ARROW_SOURCE_DIR}/build/src")
+
+ # Using arrow_flight_shared, see example
+ # https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt
+ set(ARROW_GRPC_USE_SHARED ON)
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,--no-as-needed gRPC::grpc++_reflection
+ -Wl,--as-needed)
+
+ set(FLIGHT_PROTO_PATH "${ARROW_SOURCE}/format")
+ set(FLIGHT_PROTO "${ARROW_SOURCE}/format/Flight.proto")
+
+ set(FLIGHT_GENERATED_PROTO_FILES
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.h"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.h")
Review Comment:
AFAIU when building C PyArrow `${CMAKE_CURRENT_BINARY_DIR}` points to the build dir of the C PyArrow build which is in arrow/python. But here I need files from Arrow C++ build. I had no other idea than to use relative path like so.
I will try to compile Python Flight bindings without including some internal headers, as @pitrou suggested, and then maybe this will be redundant.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1172280393
Thanks for the comments!
In that case I will try to remove the internal headers for Python Flight and see how it goes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
lidavidm commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r913722006
##########
cpp/src/arrow/flight/types.cc:
##########
@@ -150,6 +150,12 @@ arrow::Result<std::shared_ptr<Schema>> SchemaResult::GetSchema(
return ipc::ReadSchema(&schema_reader, dictionary_memo);
}
+arrow::Result<SchemaResult> SchemaResult::Make(const Schema& schema) {
+ std::string schema_in;
+ RETURN_NOT_OK(internal::SchemaToString(schema, &schema_in));
+ return SchemaResult(schema_in);
Review Comment:
nit
```suggestion
return SchemaResult(std::move(schema_in));
```
##########
python/pyarrow/src_arrow/flight.cc:
##########
@@ -371,24 +370,21 @@ Status CreateFlightInfo(const std::shared_ptr<arrow::Schema>& schema,
const std::vector<arrow::flight::FlightEndpoint>& endpoints,
int64_t total_records, int64_t total_bytes,
std::unique_ptr<arrow::flight::FlightInfo>* out) {
- arrow::flight::FlightInfo::Data flight_data;
- RETURN_NOT_OK(arrow::flight::internal::SchemaToString(*schema, &flight_data.schema));
- flight_data.descriptor = descriptor;
- flight_data.endpoints = endpoints;
- flight_data.total_records = total_records;
- flight_data.total_bytes = total_bytes;
- arrow::flight::FlightInfo value(flight_data);
- *out = std::unique_ptr<arrow::flight::FlightInfo>(new arrow::flight::FlightInfo(value));
+ auto result =
+ arrow::flight::FlightInfo::Make(*schema,
+ descriptor,
+ endpoints,
+ total_records,
+ total_bytes);
+ *out = std::unique_ptr<arrow::flight::FlightInfo>(&(*result));
Review Comment:
These should use ARROW_RETURN_NOT_OK
```suggestion
ARROW_ASSIGN_OR_RAISE(auto result,
arrow::flight::FlightInfo::Make(*schema,
descriptor,
endpoints,
total_records,
total_bytes));
*out = std::unique_ptr<arrow::flight::FlightInfo>(std::move(result));
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1187403642
Revision: dcda33bc1ee5fd8b56415b77f13b061c73e61361
Submitted crossbow builds: [ursacomputing/crossbow @ actions-cc8db4fbed](https://github.com/ursacomputing/crossbow/branches/all?query=actions-cc8db4fbed)
|Task|Status|
|----|------|
|example-python-minimal-build-fedora-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-example-python-minimal-build-fedora-conda)|
|example-python-minimal-build-ubuntu-venv|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-example-python-minimal-build-ubuntu-venv)|
|python-sdist|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-python-sdist)|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.10)|
|test-conda-python-3.7|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.7)|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.7-hdfs-3.2.1)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-0.24|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.7-pandas-0.24)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-spark-v3.1.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.7-spark-v3.1.2)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.8)|
|test-conda-python-3.8-hypothesis|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.8-hypothesis)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.8-pandas-nightly)|
|test-conda-python-3.8-spark-v3.2.0|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.8-spark-v3.2.0)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.9)|
|test-conda-python-3.9-dask-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.9-dask-latest)|
|test-conda-python-3.9-dask-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.9-dask-master)|
|test-conda-python-3.9-pandas-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.9-pandas-master)|
|test-conda-python-3.9-spark-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-test-conda-python-3.9-spark-master)|
|test-debian-11-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-cc8db4fbed-azure-test-debian-11-python-3)|
|test-fedora-35-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-cc8db4fbed-azure-test-fedora-35-python-3)|
|test-ubuntu-20.04-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-cc8db4fbed-azure-test-ubuntu-20.04-python-3)|
|verify-rc-source-python-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-verify-rc-source-python-linux-almalinux-8-amd64)|
|verify-rc-source-python-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-verify-rc-source-python-linux-conda-latest-amd64)|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
|verify-rc-source-python-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-verify-rc-source-python-linux-ubuntu-20.04-amd64)|
|verify-rc-source-python-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-verify-rc-source-python-linux-ubuntu-22.04-amd64)|
|verify-rc-source-python-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-verify-rc-source-python-macos-amd64)|
|verify-rc-source-python-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-verify-rc-source-python-macos-arm64)|
|verify-rc-source-python-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-cc8db4fbed-github-verify-rc-source-python-macos-conda-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1191093664
Revision: b0ea5ddfaceb9a56f60b1ed85fbf720c6baedc47
Submitted crossbow builds: [ursacomputing/crossbow @ actions-05c026ba58](https://github.com/ursacomputing/crossbow/branches/all?query=actions-05c026ba58)
|Task|Status|
|----|------|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-05c026ba58-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926438185
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs PyArrow cpp options
+#
+
+# Check all the options from Arrow and PyArrow cpp to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+include(SetupCxxFlags)
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
Review Comment:
Ah, I see. Yes, you're right.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926340236
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
Review Comment:
> I think that all `python/pyarrow/src/` are rewritten in Cython eventually.
That's basically impossible. Cython does not allow you to write C++ classes, for example.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r925393624
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
Review Comment:
Oh, didn't think about that.
It also implies a lot of changes in this PR alone and so I would actually be in favour of not changing the name of the final lib.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r949065096
##########
ci/scripts/python_test.sh:
##########
@@ -54,4 +55,14 @@ export PYARROW_TEST_ORC
export PYARROW_TEST_PARQUET
export PYARROW_TEST_S3
+# Testing PyArrow CPP
Review Comment:
```suggestion
# Testing PyArrow C++
```
##########
docs/source/developers/python.rst:
##########
@@ -131,6 +131,30 @@ for ``.py`` files or
for ``.pyx`` and ``.pxi`` files. In this case you will also need to
install the `pytest-cython <https://github.com/lgpage/pytest-cython>`_ plugin.
+Testing PyArrow CPP
Review Comment:
```suggestion
Testing PyArrow C++
```
##########
docs/source/developers/python.rst:
##########
@@ -131,6 +131,30 @@ for ``.py`` files or
for ``.pyx`` and ``.pxi`` files. In this case you will also need to
install the `pytest-cython <https://github.com/lgpage/pytest-cython>`_ plugin.
+Testing PyArrow CPP
+-------------------
+
+If you want to run ctest for the tests that are included in the PyArrow CPP
+module, you will need to build Arrow with ``-DARROW_BUILD_TESTS=ON``.
+
+.. note::
+
+ Currently building the C++ unit tests does not work with googletest
+ from conda-forge, so we must use the ``BUNDLED`` source for building that
+ dependency
+
+ In case you use conda add ``-DGTest_SOURCE=BUNDLED`` to the CMake flags
+ when building Arrow.
Review Comment:
```suggestion
Currently, building the PyArrow C++ unit tests does not work with the
googletest package from conda-forge. If you are in this situation, please
add ``-DGTest_SOURCE=BUNDLED`` to the CMake flags
when building Arrow C++.
```
##########
docs/source/developers/python.rst:
##########
@@ -131,6 +131,30 @@ for ``.py`` files or
for ``.pyx`` and ``.pxi`` files. In this case you will also need to
install the `pytest-cython <https://github.com/lgpage/pytest-cython>`_ plugin.
+Testing PyArrow CPP
+-------------------
+
+If you want to run ctest for the tests that are included in the PyArrow CPP
+module, you will need to build Arrow with ``-DARROW_BUILD_TESTS=ON``.
Review Comment:
```suggestion
Most of the tests for PyArrow are part of the ``pytest``-based test suite mentioned above,
but a few low-level tests are written directly in C++ for historical reasons.
Those tests can be run using ``ctest``, but you first will need to build Arrow C++
with ``-DARROW_BUILD_TESTS=ON``.
```
##########
docs/source/developers/python.rst:
##########
@@ -131,6 +131,30 @@ for ``.py`` files or
for ``.pyx`` and ``.pxi`` files. In this case you will also need to
install the `pytest-cython <https://github.com/lgpage/pytest-cython>`_ plugin.
+Testing PyArrow CPP
+-------------------
+
+If you want to run ctest for the tests that are included in the PyArrow CPP
+module, you will need to build Arrow with ``-DARROW_BUILD_TESTS=ON``.
+
+.. note::
+
+ Currently building the C++ unit tests does not work with googletest
+ from conda-forge, so we must use the ``BUNDLED`` source for building that
+ dependency
+
+ In case you use conda add ``-DGTest_SOURCE=BUNDLED`` to the CMake flags
+ when building Arrow.
+
+After Arrow C++ and PyArrow are built, navigate to ``python/build/dist``
+folder and run ctest:
Review Comment:
```suggestion
After Arrow C++ and PyArrow are built, you can navigate to the ``python/build/dist``
folder and run ``ctest``:
```
##########
docs/source/developers/python.rst:
##########
@@ -391,6 +415,12 @@ variable to 1.
To set the number of threads used to compile PyArrow's C++/Cython components,
set the ``PYARROW_PARALLEL`` environment variable.
+.. note::
+
+ If you used a different directory name for building Arrow C++ (by default it is
+ named "build"), then also set `ARROW_BUILD_DIR='name_of_build_dir'`. This way
+ pyarrow can find Arrow C++ built files.
Review Comment:
```suggestion
If you used a different directory name for building Arrow C++ (by default it is
named "build"), then you should also set the environment variable
``ARROW_BUILD_DIR='name_of_build_dir'``. This way
PyArrow can find the Arrow C++ built files.
```
##########
python/setup.py:
##########
@@ -227,6 +228,126 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_pyarrow_cpp(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
Review Comment:
For the record, can we open a JIRA to also support `RelWithDebInfo`?
##########
python/pyarrow/tests/test_cython.py:
##########
@@ -136,9 +138,9 @@ def test_cython_api(tmpdir):
delim, var = ':', 'LD_LIBRARY_PATH'
subprocess_env[var] = delim.join(
- pa.get_library_dirs() + [subprocess_env.get(var, '')]
+ pa.get_library_dirs() +
+ [subprocess_env.get(var, '')] + [str(tmpdir)]
Review Comment:
I'm curious, why was it necessary to add `tmpdir` here? Ideally, `pa.get_library_dirs()` is sufficient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [C++][Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1228124010
@kou Description is updated and the JIRA issue for the `centos-9-stream-amd64` failure can be found here: https://issues.apache.org/jira/browse/ARROW-17536.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1224007723
Yes. Python related entries should be removed from `arrow.spec.in`.
Python Flight related entries https://github.com/apache/arrow/blob/master/dev/tasks/linux-packages/apache-arrow/yum/arrow.spec.in#L548-L587 and `use_python` related codes such as https://github.com/apache/arrow/blob/master/dev/tasks/linux-packages/apache-arrow/yum/arrow.spec.in#L548-L587 should be removed too.
We also need to remove Python related patterns from `dev/tasks/tasks.yml`: https://github.com/apache/arrow/blob/master/dev/tasks/tasks.yml#L751-L760
We also need to do similar thing for .deb:
* Remove lines that are started with `@ENABLE_PYTHON@` in https://github.com/apache/arrow/blob/master/dev/tasks/linux-packages/apache-arrow/debian/control.in
* Remove `*python*` in https://github.com/apache/arrow/tree/master/dev/tasks/linux-packages/apache-arrow/debian
* Remove `dev/tasks/linux-packages/apache-arrow/debian/*python*` in https://github.com/apache/arrow/blob/master/dev/release/rat_exclude_files.txt
* Remove `python3-numpy` from `dev/tasks/linux-packages/apache-arrow/apt/*/Dockerfile` such as https://github.com/apache/arrow/tree/master/dev/tasks/linux-packages/apache-arrow/debian
* Remove Python related options in https://github.com/apache/arrow/blob/master/dev/tasks/linux-packages/apache-arrow/debian/rules
* https://github.com/apache/arrow/blob/master/dev/tasks/tasks.yml#L630-L637
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r930505957
##########
python/setup.py:
##########
@@ -227,6 +228,118 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_pyarrow_cpp(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
+ raise ValueError("--build-type (or PYARROW_BUILD_TYPE) needs to "
+ "be 'release' or 'debug'")
+
+ # The directory containing this setup.py
+ source = os.path.dirname(os.path.abspath(__file__))
+ # The directory containing this PyArrow cpp CMakeLists.txt
+ source_pyarrow_cpp = pjoin(source, "pyarrow/src")
+
+ # The directory for the module being built
+ build_cmd = self.get_finalized_command('build')
+ saved_cwd = os.getcwd()
+ build_dir = pjoin(saved_cwd, 'build', 'dist')
+ build_include = pjoin(saved_cwd, 'build', 'dist', 'include')
+ build_lib = pjoin(os.getcwd(), build_cmd.build_lib)
+
+ # The directory containing Arrow C++ build
+ arrow_build_dir = os.environ.get('ARROW_BUILD_DIR', 'build')
+ if self.inplace:
+ # a bit hacky
+ build_lib = saved_cwd
+ if not os.path.isdir(build_dir):
+ self.mkpath(build_dir)
+ if not os.path.isdir(build_lib):
+ self.mkpath(build_lib)
+ if not os.path.isdir(build_include):
+ self.mkpath(build_include)
+
+ # Change to the build directory
+ with changed_dir(build_dir):
+ # cmake args
+ cmake_options = [
+ '-DCMAKE_INSTALL_PREFIX=' +
+ str(pjoin(saved_cwd, 'build/dist')),
+ '-DCMAKE_BUILD_TYPE={0}'.format(self.build_type.lower()),
+ '-DARROW_BUILD_DIR=' + str(arrow_build_dir),
+ '-DPYTHON_EXECUTABLE=%s' % sys.executable,
+ '-DPython3_EXECUTABLE=%s' % sys.executable,
+ ]
+
+ # Check for specific options
+ def append_cmake_bool(value, varname):
+ cmake_options.append('-D{0}={1}'.format(
+ varname, 'on' if value else 'off'))
+
+ append_cmake_bool(self.with_dataset, 'PYARROW_WITH_DATASET')
+ append_cmake_bool(self.with_parquet_encryption,
+ 'PYARROW_WITH_PARQUET_ENCRYPTION')
+ append_cmake_bool(self.with_hdfs,
+ 'PYARROW_WITH_HDFS')
+
+ # Windows
+ if self.cmake_generator:
+ cmake_options += ['-G', self.cmake_generator]
+
+ # build args
+ build_tool_args = []
+ if os.environ.get('PYARROW_PARALLEL'):
+ build_tool_args.append('--')
+ build_tool_args.append(
+ '-j{0}'.format(os.environ['PYARROW_PARALLEL']))
+
+ # run cmake
+ print("-- Running cmake for pyarrow cpp")
+ self.spawn(['cmake'] + cmake_options + [source_pyarrow_cpp])
+ print("-- Finished cmake for pyarrow cpp")
+ # run make & install
+ print("-- Running cmake build and install for pyarrow cpp")
+ self.spawn(['cmake', '--build', '.', '--target', 'install'] +
+ build_tool_args)
+ print("-- Finished cmake build and install for pyarrow cpp")
+
+ # Move the libraries to the place expected by the Python build
+ try:
+ os.makedirs(pjoin(build_lib, 'pyarrow'))
+ except OSError:
+ pass
+
+ # helper function
+ def copy_libs(folder_name):
+ for libname in os.listdir(pjoin(build_dir, folder_name)):
+ if "python" in libname:
+ libname_path = pjoin(build_lib, "pyarrow", libname)
+ if os.path.exists(libname_path):
+ os.remove(libname_path)
+ print(
+ f"Copying {pjoin(build_dir, folder_name, libname)}"
+ f" to {pjoin(build_lib, 'pyarrow', libname)}")
+ shutil.copy(pjoin(build_dir, folder_name, libname),
+ pjoin(build_lib, "pyarrow"))
+
+ # Move libraries to python/pyarrow
+ # For windows builds, move dll from bin
+ for folder in ['lib', 'lib64', 'bin']:
+ try:
+ copy_libs(folder)
+ except OSError:
Review Comment:
Oh, actually in the Windows build there is a `lib` and `bin` folder and the later one is the one that needs to get copied as it includes `bin/arrow_python.dll` :/
I guess I will make the windows copying separate ...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1196162388
@github-actions crossbow submit *python*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1193333029
Revision: a9a3e912adf70b18c8cd7c572221f103414dcea8
Submitted crossbow builds: [ursacomputing/crossbow @ actions-167817167a](https://github.com/ursacomputing/crossbow/branches/all?query=actions-167817167a)
|Task|Status|
|----|------|
|example-python-minimal-build-fedora-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-example-python-minimal-build-fedora-conda)|
|example-python-minimal-build-ubuntu-venv|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-example-python-minimal-build-ubuntu-venv)|
|python-sdist|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-python-sdist)|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.10)|
|test-conda-python-3.7|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.7)|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.7-hdfs-3.2.1)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-0.24|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.7-pandas-0.24)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-spark-v3.1.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.7-spark-v3.1.2)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.8)|
|test-conda-python-3.8-hypothesis|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.8-hypothesis)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.8-pandas-nightly)|
|test-conda-python-3.8-spark-v3.2.0|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.8-spark-v3.2.0)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.9)|
|test-conda-python-3.9-dask-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.9-dask-latest)|
|test-conda-python-3.9-dask-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.9-dask-master)|
|test-conda-python-3.9-pandas-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.9-pandas-master)|
|test-conda-python-3.9-spark-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-test-conda-python-3.9-spark-master)|
|test-debian-11-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-167817167a-azure-test-debian-11-python-3)|
|test-fedora-35-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-167817167a-azure-test-fedora-35-python-3)|
|test-ubuntu-20.04-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-167817167a-azure-test-ubuntu-20.04-python-3)|
|verify-rc-source-python-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-verify-rc-source-python-linux-almalinux-8-amd64)|
|verify-rc-source-python-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-verify-rc-source-python-linux-conda-latest-amd64)|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
|verify-rc-source-python-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-verify-rc-source-python-linux-ubuntu-20.04-amd64)|
|verify-rc-source-python-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-verify-rc-source-python-linux-ubuntu-22.04-amd64)|
|verify-rc-source-python-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-verify-rc-source-python-macos-amd64)|
|verify-rc-source-python-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-verify-rc-source-python-macos-arm64)|
|verify-rc-source-python-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-167817167a-github-verify-rc-source-python-macos-conda-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1178777287
BTW, something I ran into that we should maybe also document / might be a typical gotcha: in a typical development workflow, after making some change, you might first rebuild the C++ library, and then rebuild pyarrow (both inplace, incrementally / with caching, without doing a full clean rebuild).
Previously, because the Python C++ lived in the main C++ library, it was taken care of that the Python C++ code was rebuilt when necessary because of other changes in the C++ code base. But now, because this Python C++ code is now a separate build step, this isn't done automatically (or is there a way this could still be done automatically?). So very often when doing a change in the C++ code, you will need to force a clean rebuild of the Python C++ code (i.e. remove the python/build directory, which isn't done by default at the moment) to avoid running into compilation issues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1178736346
Comment and a question about Arrow Flight build in this PR.
I bumped into a situation where the build has `ARROW_GRPC_USE_SHARED=off` and `ARROW_BUILD_STATIC=off`:
https://github.com/apache/arrow/runs/7247952478?check_suite_focus=true
https://github.com/apache/arrow/runs/7247952610?check_suite_focus=true
But if I understand correctly (from:) https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt#L32-L51 this case should produce an error? I added next lines in this PR:
https://github.com/apache/arrow/blob/7fec5fd83c38b5ff8f991e3f17017941bdb1d64d/python/pyarrow/src_arrow/CMakeLists.txt#L336-L340
so because of it some builds will always fail.
Am I correct to suggest changing the build flags or am I missing something?
@lidavidm @kou @raulcd
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r918615021
##########
python/setup.py:
##########
@@ -227,6 +228,94 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_arrow_python(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
+ raise ValueError("--build-type (or PYARROW_BUILD_TYPE) needs to "
+ "be 'release' or 'debug'")
+
+ # The directory containing this setup.py
+ source = os.path.dirname(os.path.abspath(__file__))
+ # The directory containing this C PyArrow CMakeLists.txt
+ source_cpyarrow = pjoin(source, "pyarrow/src_arrow")
+
+ # The directory for the module being built
+ build_cmd = self.get_finalized_command('build')
+ saved_cwd = os.getcwd()
+ build_temp = pjoin(saved_cwd, 'build/dist/temp')
+ build_include = pjoin(saved_cwd, 'build/dist/include')
+ build_lib = pjoin(os.getcwd(), build_cmd.build_lib)
+
+ # The directory containing Arrow C++ build
+ arrow_build_dir = os.environ.get('ARROW_BUILD_DIR', 'build')
+
+ if self.inplace:
+ # a bit hacky
+ build_lib = saved_cwd
+
+ if not os.path.isdir(build_temp):
+ self.mkpath(build_temp)
+ if not os.path.isdir(build_lib):
+ self.mkpath(build_lib)
+ if not os.path.isdir(build_include):
+ self.mkpath(build_include)
+
+ # Change to the build directory
+ with changed_dir(build_temp):
+ # cmake args
+ cmake_options = [
+ '-DCMAKE_INSTALL_PREFIX=' +
+ str(pjoin(saved_cwd, 'build/dist')),
+ '-DCMAKE_BUILD_TYPE={0}'.format(self.build_type.lower()),
+ '-DARROW_BUILD_DIR=' + str(arrow_build_dir),
+ '-DPYTHON_EXECUTABLE=%s' % sys.executable,
+ '-DPython3_EXECUTABLE=%s' % sys.executable,
+ ]
+
+ # Check for specific options
+ def append_cmake_bool(value, varname):
+ cmake_options.append('-D{0}={1}'.format(
+ varname, 'on' if value else 'off'))
+
+ append_cmake_bool(self.with_dataset, 'PYARROW_WITH_DATASET')
+ append_cmake_bool(self.with_parquet_encryption,
+ 'PYARROW_WITH_PARQUET_ENCRYPTION')
+ append_cmake_bool(self.with_hdfs,
+ 'PYARROW_WITH_HDFS')
+
+ # run cmake
+ print("-- Running cmake for C pyarrow")
+ self.spawn(['cmake'] + cmake_options + [source_cpyarrow])
+ print("-- Finished cmake for C pyarrow")
+ # run make & install
+ print("-- Running make build and install for C pyarrow")
+ self.spawn(['make', '-j4'])
Review Comment:
Instead of hard-coding the `j4`, we should probably make this consistent with how `_run_cmake` (for cython) does this, based on `PYARROW_PARALLEL`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r918623819
##########
python/setup.py:
##########
@@ -227,6 +228,94 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_arrow_python(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
+ raise ValueError("--build-type (or PYARROW_BUILD_TYPE) needs to "
+ "be 'release' or 'debug'")
+
+ # The directory containing this setup.py
+ source = os.path.dirname(os.path.abspath(__file__))
+ # The directory containing this C PyArrow CMakeLists.txt
+ source_cpyarrow = pjoin(source, "pyarrow/src_arrow")
+
+ # The directory for the module being built
+ build_cmd = self.get_finalized_command('build')
+ saved_cwd = os.getcwd()
+ build_temp = pjoin(saved_cwd, 'build/dist/temp')
+ build_include = pjoin(saved_cwd, 'build/dist/include')
+ build_lib = pjoin(os.getcwd(), build_cmd.build_lib)
+
+ # The directory containing Arrow C++ build
+ arrow_build_dir = os.environ.get('ARROW_BUILD_DIR', 'build')
+
+ if self.inplace:
+ # a bit hacky
+ build_lib = saved_cwd
+
+ if not os.path.isdir(build_temp):
+ self.mkpath(build_temp)
+ if not os.path.isdir(build_lib):
+ self.mkpath(build_lib)
+ if not os.path.isdir(build_include):
+ self.mkpath(build_include)
+
+ # Change to the build directory
+ with changed_dir(build_temp):
+ # cmake args
+ cmake_options = [
+ '-DCMAKE_INSTALL_PREFIX=' +
+ str(pjoin(saved_cwd, 'build/dist')),
+ '-DCMAKE_BUILD_TYPE={0}'.format(self.build_type.lower()),
+ '-DARROW_BUILD_DIR=' + str(arrow_build_dir),
+ '-DPYTHON_EXECUTABLE=%s' % sys.executable,
+ '-DPython3_EXECUTABLE=%s' % sys.executable,
+ ]
+
+ # Check for specific options
+ def append_cmake_bool(value, varname):
+ cmake_options.append('-D{0}={1}'.format(
+ varname, 'on' if value else 'off'))
+
+ append_cmake_bool(self.with_dataset, 'PYARROW_WITH_DATASET')
+ append_cmake_bool(self.with_parquet_encryption,
+ 'PYARROW_WITH_PARQUET_ENCRYPTION')
+ append_cmake_bool(self.with_hdfs,
+ 'PYARROW_WITH_HDFS')
+
+ # run cmake
+ print("-- Running cmake for C pyarrow")
+ self.spawn(['cmake'] + cmake_options + [source_cpyarrow])
+ print("-- Finished cmake for C pyarrow")
+ # run make & install
+ print("-- Running make build and install for C pyarrow")
+ self.spawn(['make', '-j4'])
Review Comment:
Yes, and we should not invoke `make` directly but let `cmake` do the interface so that the right generator is invoked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r918630852
##########
python/setup.py:
##########
@@ -227,6 +228,94 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_arrow_python(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
+ raise ValueError("--build-type (or PYARROW_BUILD_TYPE) needs to "
+ "be 'release' or 'debug'")
+
+ # The directory containing this setup.py
+ source = os.path.dirname(os.path.abspath(__file__))
+ # The directory containing this C PyArrow CMakeLists.txt
+ source_cpyarrow = pjoin(source, "pyarrow/src_arrow")
+
+ # The directory for the module being built
+ build_cmd = self.get_finalized_command('build')
+ saved_cwd = os.getcwd()
+ build_temp = pjoin(saved_cwd, 'build/dist/temp')
+ build_include = pjoin(saved_cwd, 'build/dist/include')
+ build_lib = pjoin(os.getcwd(), build_cmd.build_lib)
+
+ # The directory containing Arrow C++ build
+ arrow_build_dir = os.environ.get('ARROW_BUILD_DIR', 'build')
+
+ if self.inplace:
+ # a bit hacky
+ build_lib = saved_cwd
+
+ if not os.path.isdir(build_temp):
+ self.mkpath(build_temp)
+ if not os.path.isdir(build_lib):
+ self.mkpath(build_lib)
+ if not os.path.isdir(build_include):
+ self.mkpath(build_include)
+
+ # Change to the build directory
+ with changed_dir(build_temp):
+ # cmake args
+ cmake_options = [
+ '-DCMAKE_INSTALL_PREFIX=' +
+ str(pjoin(saved_cwd, 'build/dist')),
+ '-DCMAKE_BUILD_TYPE={0}'.format(self.build_type.lower()),
+ '-DARROW_BUILD_DIR=' + str(arrow_build_dir),
+ '-DPYTHON_EXECUTABLE=%s' % sys.executable,
+ '-DPython3_EXECUTABLE=%s' % sys.executable,
+ ]
+
+ # Check for specific options
+ def append_cmake_bool(value, varname):
+ cmake_options.append('-D{0}={1}'.format(
+ varname, 'on' if value else 'off'))
+
+ append_cmake_bool(self.with_dataset, 'PYARROW_WITH_DATASET')
+ append_cmake_bool(self.with_parquet_encryption,
+ 'PYARROW_WITH_PARQUET_ENCRYPTION')
+ append_cmake_bool(self.with_hdfs,
+ 'PYARROW_WITH_HDFS')
+
+ # run cmake
+ print("-- Running cmake for C pyarrow")
+ self.spawn(['cmake'] + cmake_options + [source_cpyarrow])
+ print("-- Finished cmake for C pyarrow")
+ # run make & install
+ print("-- Running make build and install for C pyarrow")
+ self.spawn(['make', '-j4'])
Review Comment:
Ah, I was playing around with `cmake --build` a bit but didn't follow through. Will do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1145975582
:warning: Ticket **has not been started in JIRA**, please click 'Start Progress'.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r911784446
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,467 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+
+# normalize ARROW_HOME path
+cmake_path(CONVERT "$ENV{ARROW_HOME}" TO_CMAKE_PATH_LIST ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+find_package(Python3Alt 3.7 REQUIRED)
+
+# Needed due to
+# CMake Error: INSTALL(EXPORT) given unknown export "arrow_python_targets"
+option(ARROW_BUILD_SHARED "Link to the Arrow shared library" ON)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+#
+# Linker flags
+#
+
+# Localize thirdparty symbols using a linker version script. This hides them
+# from the client application. The OS X linker does not support the
+# version-script option.
+if(CMAKE_VERSION VERSION_LESS 3.18)
+ if(APPLE OR WIN32)
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT FALSE)
+ else()
+ set(CXX_LINKER_SUPPORTS_VERSION_SCRIPT TRUE)
+ endif()
+else()
+ include(CheckLinkerFlag)
+ check_linker_flag(CXX
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map"
+ CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+endif()
+
+# Need to set ARROW_VERSION_SCRIPT_FLAGS for add_arrow_lib
+if(CXX_LINKER_SUPPORTS_VERSION_SCRIPT)
+ set(ARROW_VERSION_SCRIPT_FLAGS
+ "-Wl,--version-script=${ARROW_SOURCE_DIR}/src/arrow/symbols.map")
+endif()
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+if(ARROW_USE_XSIMD)
+ list(APPEND ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS xsimd)
+ list(APPEND ARROW_PYTHON_STATIC_LINK_LIBS xsimd)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_INSTALL_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS} # Defined in line 95
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+
+ find_package(ArrowFlight REQUIRED)
+ find_package(gRPC CONFIG REQUIRED)
+ include_directories("${ARROW_SOURCE_DIR}/src" "${ARROW_SOURCE_DIR}/build/src")
+
+ # Using arrow_flight_shared, see example
+ # https://github.com/apache/arrow/blob/7a0f00c16e084d194ae53d209b33b809cfc8f2d5/cpp/examples/arrow/CMakeLists.txt
+ set(ARROW_GRPC_USE_SHARED ON)
+ set(GRPC_REFLECTION_LINK_LIBS -Wl,--no-as-needed gRPC::grpc++_reflection
+ -Wl,--as-needed)
+
+ set(FLIGHT_PROTO_PATH "${ARROW_SOURCE}/format")
+ set(FLIGHT_PROTO "${ARROW_SOURCE}/format/Flight.proto")
+
+ set(FLIGHT_GENERATED_PROTO_FILES
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.pb.h"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.cc"
+ "${ARROW_SOURCE_DIR}/build/src/arrow/flight/Flight.grpc.pb.h")
Review Comment:
The use of `ARROW_BUILD_DIR` CMake variable (that you suggested) made this nicer, I think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r908152379
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -19,8 +19,45 @@
# arrow_python
#
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+# Add cmake_modules and lib/cmake/arrow folder to CMAKE_MODULE_PATH
+# TODO: remove lib/cmake/arrow folder as it should not be necessary as find_package(Arrow)
+# should load all the cmake variables (ex ARROW_DATASET)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_SOURCE}/dist/lib/cmake/arrow")
+message(STATUS "CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH}")
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+find_package(Arrow REQUIRED)
+# Loading ArrowOptions, they should be loaded by default with find_package(Arrow REQUIRED)
+include(ArrowOptions)
Review Comment:
Setting `CMAKE_PREFIX_PATH=$ARROW_HOME` environment variable (not CMake variable) didn't work for me locally.
Ended up using `ARROW_HOME` with `CMAKE_MODULE_PATH` in the CMakeLists.txt and it works.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1173764304
I will give it a try and mimic the `Make` method of `FlightInfo`. If I get stuck I might give a shout :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r913757882
##########
python/pyarrow/src_arrow/flight.cc:
##########
@@ -371,24 +370,21 @@ Status CreateFlightInfo(const std::shared_ptr<arrow::Schema>& schema,
const std::vector<arrow::flight::FlightEndpoint>& endpoints,
int64_t total_records, int64_t total_bytes,
std::unique_ptr<arrow::flight::FlightInfo>* out) {
- arrow::flight::FlightInfo::Data flight_data;
- RETURN_NOT_OK(arrow::flight::internal::SchemaToString(*schema, &flight_data.schema));
- flight_data.descriptor = descriptor;
- flight_data.endpoints = endpoints;
- flight_data.total_records = total_records;
- flight_data.total_bytes = total_bytes;
- arrow::flight::FlightInfo value(flight_data);
- *out = std::unique_ptr<arrow::flight::FlightInfo>(new arrow::flight::FlightInfo(value));
+ auto result =
+ arrow::flight::FlightInfo::Make(*schema,
+ descriptor,
+ endpoints,
+ total_records,
+ total_bytes);
+ *out = std::unique_ptr<arrow::flight::FlightInfo>(&(*result));
Review Comment:
If I use `std::move` here I get
```cpp
/Users/alenkafrim/repos/arrow-2/python/pyarrow/src_arrow/flight.cc:379:10: error: no matching conversion for functional-style cast from 'typename remove_reference<FlightInfo &>::type' (aka 'arrow::flight::FlightInfo') to 'std::unique_ptr<arrow::flight::FlightInfo>'
*out = std::unique_ptr<arrow::flight::FlightInfo>(std::move(result));
```
For time being I am keeping `&result` so that the code compiles. Can change it later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1184320796
@github-actions crossbow submit verify-rc-source-python-linux-ubuntu-18.04-amd64
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r925264377
##########
python/CMakeLists.txt:
##########
@@ -604,29 +604,27 @@ foreach(module ${CYTHON_EXTENSIONS})
${module_output_directory})
endif()
- if(PYARROW_BUNDLE_ARROW_CPP)
- # In the event that we are bundling the shared libraries (e.g. in a
- # manylinux1 wheel), we need to set the RPATH of the extensions to the
- # root of the pyarrow/ package so that libarrow/libarrow_python are able
- # to be loaded properly
- if(APPLE)
- set(module_install_rpath "@loader_path/")
- else()
- set(module_install_rpath "\$ORIGIN")
- endif()
+ # In the event that we are bundling the shared libraries (e.g. in a
+ # manylinux1 wheel), we need to set the RPATH of the extensions to the
+ # root of the pyarrow/ package so that libarrow/libarrow_python are able
+ # to be loaded properly
Review Comment:
I think this comment is no longer fully up to date? (i.e. we are now always bundling the shared library?)
##########
python/setup.py:
##########
@@ -227,6 +228,118 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_pyarrow_cpp(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
+ raise ValueError("--build-type (or PYARROW_BUILD_TYPE) needs to "
+ "be 'release' or 'debug'")
+
+ # The directory containing this setup.py
+ source = os.path.dirname(os.path.abspath(__file__))
+ # The directory containing this PyArrow cpp CMakeLists.txt
+ source_pyarrow_cpp = pjoin(source, "pyarrow/src")
+
+ # The directory for the module being built
+ build_cmd = self.get_finalized_command('build')
+ saved_cwd = os.getcwd()
+ build_dir = pjoin(saved_cwd, 'build', 'dist')
+ build_include = pjoin(saved_cwd, 'build', 'dist', 'include')
+ build_lib = pjoin(os.getcwd(), build_cmd.build_lib)
+
+ # The directory containing Arrow C++ build
+ arrow_build_dir = os.environ.get('ARROW_BUILD_DIR', 'build')
+ if self.inplace:
+ # a bit hacky
+ build_lib = saved_cwd
+ if not os.path.isdir(build_dir):
+ self.mkpath(build_dir)
+ if not os.path.isdir(build_lib):
+ self.mkpath(build_lib)
+ if not os.path.isdir(build_include):
+ self.mkpath(build_include)
+
+ # Change to the build directory
+ with changed_dir(build_dir):
+ # cmake args
+ cmake_options = [
+ '-DCMAKE_INSTALL_PREFIX=' +
+ str(pjoin(saved_cwd, 'build/dist')),
+ '-DCMAKE_BUILD_TYPE={0}'.format(self.build_type.lower()),
+ '-DARROW_BUILD_DIR=' + str(arrow_build_dir),
+ '-DPYTHON_EXECUTABLE=%s' % sys.executable,
+ '-DPython3_EXECUTABLE=%s' % sys.executable,
Review Comment:
Small style nitpick: we are using 3 different string formatting ways in 3 lines ;) (format, +, %)
(I know this is partly from copied elsewhere)
##########
python/setup.py:
##########
@@ -354,6 +468,19 @@ def append_cmake_bool(value, varname):
shutil.move(pjoin(build_prefix, 'include'),
pjoin(build_lib, 'pyarrow'))
+ # We need to, again, add the PyArrow cpp include folder
+ build_pyarrow_cpp_include = pjoin(
+ saved_cwd, 'build/dist/include')
+ if not os.path.isdir(
+ pjoin(build_pyarrow_cpp_include, "arrow", "python")):
+ self.mkpath(
Review Comment:
We are making the source path we want to copy? (if we need to create something, I would expect the target directory)
##########
python/setup.py:
##########
@@ -354,6 +468,19 @@ def append_cmake_bool(value, varname):
shutil.move(pjoin(build_prefix, 'include'),
pjoin(build_lib, 'pyarrow'))
+ # We need to, again, add the PyArrow cpp include folder
Review Comment:
Why is it needed to do this again? (can we clarify that comment?)
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
Review Comment:
On the other hand, the final library is still called "arrow_python.so", so here this is maybe fine?
##########
python/setup.py:
##########
@@ -227,6 +228,118 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_pyarrow_cpp(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
+ raise ValueError("--build-type (or PYARROW_BUILD_TYPE) needs to "
+ "be 'release' or 'debug'")
+
+ # The directory containing this setup.py
+ source = os.path.dirname(os.path.abspath(__file__))
+ # The directory containing this PyArrow cpp CMakeLists.txt
+ source_pyarrow_cpp = pjoin(source, "pyarrow/src")
+
+ # The directory for the module being built
+ build_cmd = self.get_finalized_command('build')
+ saved_cwd = os.getcwd()
+ build_dir = pjoin(saved_cwd, 'build', 'dist')
+ build_include = pjoin(saved_cwd, 'build', 'dist', 'include')
+ build_lib = pjoin(os.getcwd(), build_cmd.build_lib)
+
+ # The directory containing Arrow C++ build
+ arrow_build_dir = os.environ.get('ARROW_BUILD_DIR', 'build')
+ if self.inplace:
+ # a bit hacky
+ build_lib = saved_cwd
+ if not os.path.isdir(build_dir):
+ self.mkpath(build_dir)
+ if not os.path.isdir(build_lib):
+ self.mkpath(build_lib)
+ if not os.path.isdir(build_include):
+ self.mkpath(build_include)
+
+ # Change to the build directory
+ with changed_dir(build_dir):
+ # cmake args
+ cmake_options = [
+ '-DCMAKE_INSTALL_PREFIX=' +
+ str(pjoin(saved_cwd, 'build/dist')),
+ '-DCMAKE_BUILD_TYPE={0}'.format(self.build_type.lower()),
+ '-DARROW_BUILD_DIR=' + str(arrow_build_dir),
+ '-DPYTHON_EXECUTABLE=%s' % sys.executable,
+ '-DPython3_EXECUTABLE=%s' % sys.executable,
+ ]
+
+ # Check for specific options
+ def append_cmake_bool(value, varname):
+ cmake_options.append('-D{0}={1}'.format(
+ varname, 'on' if value else 'off'))
+
+ append_cmake_bool(self.with_dataset, 'PYARROW_WITH_DATASET')
+ append_cmake_bool(self.with_parquet_encryption,
+ 'PYARROW_WITH_PARQUET_ENCRYPTION')
+ append_cmake_bool(self.with_hdfs,
+ 'PYARROW_WITH_HDFS')
+
+ # Windows
+ if self.cmake_generator:
+ cmake_options += ['-G', self.cmake_generator]
+
+ # build args
+ build_tool_args = []
+ if os.environ.get('PYARROW_PARALLEL'):
+ build_tool_args.append('--')
+ build_tool_args.append(
+ '-j{0}'.format(os.environ['PYARROW_PARALLEL']))
+
+ # run cmake
+ print("-- Running cmake for pyarrow cpp")
+ self.spawn(['cmake'] + cmake_options + [source_pyarrow_cpp])
+ print("-- Finished cmake for pyarrow cpp")
+ # run make & install
+ print("-- Running cmake build and install for pyarrow cpp")
+ self.spawn(['cmake', '--build', '.', '--target', 'install'] +
+ build_tool_args)
+ print("-- Finished cmake build and install for pyarrow cpp")
+
+ # Move the libraries to the place expected by the Python build
+ try:
+ os.makedirs(pjoin(build_lib, 'pyarrow'))
+ except OSError:
+ pass
+
+ # helper function
+ def copy_libs(folder_name):
+ for libname in os.listdir(pjoin(build_dir, folder_name)):
+ if "python" in libname:
+ libname_path = pjoin(build_lib, "pyarrow", libname)
+ if os.path.exists(libname_path):
+ os.remove(libname_path)
+ print(
+ f"Copying {pjoin(build_dir, folder_name, libname)}"
+ f" to {pjoin(build_lib, 'pyarrow', libname)}")
+ shutil.copy(pjoin(build_dir, folder_name, libname),
+ pjoin(build_lib, "pyarrow"))
+
+ # Move libraries to python/pyarrow
+ # For windows builds, move dll from bin
+ for folder in ['lib', 'lib64', 'bin']:
+ try:
+ copy_libs(folder)
+ except OSError:
+ pass
+
+ # Copy headers tp python/pyarrow/include
Review Comment:
```suggestion
# Copy headers to python/pyarrow/include
```
##########
python/setup.py:
##########
@@ -401,15 +528,10 @@ def append_cmake_bool(value, varname):
def _bundle_arrow_cpp(self, build_prefix, build_lib):
print(pjoin(build_lib, 'pyarrow'))
move_shared_libs(build_prefix, build_lib, "arrow")
- move_shared_libs(build_prefix, build_lib, "arrow_python")
if self.with_cuda:
move_shared_libs(build_prefix, build_lib, "arrow_cuda")
if self.with_substrait:
move_shared_libs(build_prefix, build_lib, "arrow_substrait")
- if self.with_flight:
- move_shared_libs(build_prefix, build_lib, "arrow_flight")
Review Comment:
Why is arrow_flight not needed anymore?
##########
python/setup.py:
##########
@@ -227,6 +228,118 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_pyarrow_cpp(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
+ raise ValueError("--build-type (or PYARROW_BUILD_TYPE) needs to "
+ "be 'release' or 'debug'")
+
+ # The directory containing this setup.py
+ source = os.path.dirname(os.path.abspath(__file__))
+ # The directory containing this PyArrow cpp CMakeLists.txt
+ source_pyarrow_cpp = pjoin(source, "pyarrow/src")
+
+ # The directory for the module being built
+ build_cmd = self.get_finalized_command('build')
+ saved_cwd = os.getcwd()
+ build_dir = pjoin(saved_cwd, 'build', 'dist')
+ build_include = pjoin(saved_cwd, 'build', 'dist', 'include')
+ build_lib = pjoin(os.getcwd(), build_cmd.build_lib)
+
+ # The directory containing Arrow C++ build
+ arrow_build_dir = os.environ.get('ARROW_BUILD_DIR', 'build')
+ if self.inplace:
+ # a bit hacky
+ build_lib = saved_cwd
+ if not os.path.isdir(build_dir):
+ self.mkpath(build_dir)
+ if not os.path.isdir(build_lib):
+ self.mkpath(build_lib)
+ if not os.path.isdir(build_include):
+ self.mkpath(build_include)
+
+ # Change to the build directory
+ with changed_dir(build_dir):
+ # cmake args
+ cmake_options = [
+ '-DCMAKE_INSTALL_PREFIX=' +
+ str(pjoin(saved_cwd, 'build/dist')),
Review Comment:
Is this the same as `build_dir` ? (meaning we can use that variable here)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926371603
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
Review Comment:
> That's basically impossible. Cython does not allow you to write C++ classes, for example.
Oh...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926437650
##########
python/setup.py:
##########
@@ -227,6 +228,118 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_pyarrow_cpp(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
+ raise ValueError("--build-type (or PYARROW_BUILD_TYPE) needs to "
+ "be 'release' or 'debug'")
+
+ # The directory containing this setup.py
+ source = os.path.dirname(os.path.abspath(__file__))
+ # The directory containing this PyArrow cpp CMakeLists.txt
+ source_pyarrow_cpp = pjoin(source, "pyarrow/src")
+
+ # The directory for the module being built
+ build_cmd = self.get_finalized_command('build')
+ saved_cwd = os.getcwd()
+ build_dir = pjoin(saved_cwd, 'build', 'dist')
+ build_include = pjoin(saved_cwd, 'build', 'dist', 'include')
+ build_lib = pjoin(os.getcwd(), build_cmd.build_lib)
+
+ # The directory containing Arrow C++ build
+ arrow_build_dir = os.environ.get('ARROW_BUILD_DIR', 'build')
+ if self.inplace:
+ # a bit hacky
+ build_lib = saved_cwd
+ if not os.path.isdir(build_dir):
+ self.mkpath(build_dir)
+ if not os.path.isdir(build_lib):
+ self.mkpath(build_lib)
+ if not os.path.isdir(build_include):
+ self.mkpath(build_include)
+
+ # Change to the build directory
+ with changed_dir(build_dir):
+ # cmake args
+ cmake_options = [
+ '-DCMAKE_INSTALL_PREFIX=' +
+ str(pjoin(saved_cwd, 'build/dist')),
+ '-DCMAKE_BUILD_TYPE={0}'.format(self.build_type.lower()),
+ '-DARROW_BUILD_DIR=' + str(arrow_build_dir),
+ '-DPYTHON_EXECUTABLE=%s' % sys.executable,
+ '-DPython3_EXECUTABLE=%s' % sys.executable,
+ ]
+
+ # Check for specific options
+ def append_cmake_bool(value, varname):
+ cmake_options.append('-D{0}={1}'.format(
+ varname, 'on' if value else 'off'))
+
+ append_cmake_bool(self.with_dataset, 'PYARROW_WITH_DATASET')
+ append_cmake_bool(self.with_parquet_encryption,
+ 'PYARROW_WITH_PARQUET_ENCRYPTION')
+ append_cmake_bool(self.with_hdfs,
+ 'PYARROW_WITH_HDFS')
+
+ # Windows
+ if self.cmake_generator:
+ cmake_options += ['-G', self.cmake_generator]
+
+ # build args
+ build_tool_args = []
+ if os.environ.get('PYARROW_PARALLEL'):
+ build_tool_args.append('--')
+ build_tool_args.append(
+ '-j{0}'.format(os.environ['PYARROW_PARALLEL']))
+
+ # run cmake
+ print("-- Running cmake for pyarrow cpp")
+ self.spawn(['cmake'] + cmake_options + [source_pyarrow_cpp])
+ print("-- Finished cmake for pyarrow cpp")
+ # run make & install
+ print("-- Running cmake build and install for pyarrow cpp")
+ self.spawn(['cmake', '--build', '.', '--target', 'install'] +
+ build_tool_args)
+ print("-- Finished cmake build and install for pyarrow cpp")
+
+ # Move the libraries to the place expected by the Python build
+ try:
+ os.makedirs(pjoin(build_lib, 'pyarrow'))
+ except OSError:
+ pass
+
+ # helper function
+ def copy_libs(folder_name):
+ for libname in os.listdir(pjoin(build_dir, folder_name)):
+ if "python" in libname:
+ libname_path = pjoin(build_lib, "pyarrow", libname)
+ if os.path.exists(libname_path):
+ os.remove(libname_path)
+ print(
+ f"Copying {pjoin(build_dir, folder_name, libname)}"
+ f" to {pjoin(build_lib, 'pyarrow', libname)}")
+ shutil.copy(pjoin(build_dir, folder_name, libname),
+ pjoin(build_lib, "pyarrow"))
+
+ # Move libraries to python/pyarrow
+ # For windows builds, move dll from bin
+ for folder in ['lib', 'lib64', 'bin']:
+ try:
+ copy_libs(folder)
+ except OSError:
Review Comment:
Ah... can we then ensure that exactly one copy (not zero, not several) was done successfully?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926434398
##########
ci/scripts/python_test.sh:
##########
@@ -20,6 +20,7 @@
set -ex
arrow_dir=${1}
+test_dir=${1}/python/build/dist/temp
Review Comment:
Hm ... not sure, can't remember. I think `ctest` selects it, but will check again and see if I can change it (if there is too much work I will leave it as is as we will be changing the tests for PyArrow cpp soon)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
raulcd commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r928571489
##########
ci/appveyor-cpp-build.bat:
##########
@@ -162,7 +162,7 @@ python setup.py develop -q || exit /B
set PYTHONDEVMODE=1
-py.test -r sxX --durations=15 --pyargs pyarrow.tests || exit /B
+py.test -s --durations=15 --pyargs pyarrow.tests || exit /B
Review Comment:
should we revert this back?
```suggestion
py.test -r sxX --durations=15 --pyargs pyarrow.tests || exit /B
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r928748047
##########
ci/appveyor-cpp-build.bat:
##########
@@ -162,7 +162,7 @@ python setup.py develop -q || exit /B
set PYTHONDEVMODE=1
-py.test -r sxX --durations=15 --pyargs pyarrow.tests || exit /B
+py.test -s --durations=15 --pyargs pyarrow.tests || exit /B
Review Comment:
Yes, will do that! 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
raulcd commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r894347131
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -19,8 +19,37 @@
# arrow_python
#
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules")
+
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION 9.0.0)
Review Comment:
Minor thing, I've seen this pattern used to set other versions on CMake which I think works better with our release scripts to update versions on Releases/release candidates/post release tasks. Examples:
https://github.com/apache/arrow/blob/master/cpp/CMakeLists.txt#L50-L52
https://github.com/apache/arrow/blob/master/matlab/CMakeLists.txt#L192-L195
```suggestion
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r951032044
##########
python/setup.py:
##########
@@ -227,6 +228,126 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_pyarrow_cpp(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
Review Comment:
Done: https://issues.apache.org/jira/browse/ARROW-17488
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1225944740
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
raulcd commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1223875213
In regards to the centos and almalinux packaging builds I suspect we want to remove all the python references from:
https://github.com/apache/arrow/blob/master/dev/tasks/linux-packages/apache-arrow/yum/arrow.spec.in#L509-L546
As all those will be shipped with pyarrow now and won't be included on the RPM?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1226118941
Revision: 153a1cb3c30d8774dde4f0da9d805e9ac0724854
Submitted crossbow builds: [ursacomputing/crossbow @ actions-a9994de6ad](https://github.com/ursacomputing/crossbow/branches/all?query=actions-a9994de6ad)
|Task|Status|
|----|------|
|almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-almalinux-8-amd64)|
|almalinux-8-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|almalinux-9-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-almalinux-9-amd64)|
|almalinux-9-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|amazon-linux-2-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-amazon-linux-2-amd64)|
|amazon-linux-2-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|centos-7-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-centos-7-amd64)|
|centos-8-stream-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-centos-8-stream-amd64)|
|centos-8-stream-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|centos-9-stream-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-centos-9-stream-amd64)|
|centos-9-stream-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|conan-maximum|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-conan-maximum)|
|conan-minimum|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-conan-minimum)|
|conda-clean|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-clean)|
|conda-linux-gcc-py310-arm64|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py310-arm64)|
|conda-linux-gcc-py310-cpu|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py310-cpu)|
|conda-linux-gcc-py310-cuda|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py310-cuda)|
|conda-linux-gcc-py310-ppc64le|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py310-ppc64le)|
|conda-linux-gcc-py37-arm64|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py37-arm64)|
|conda-linux-gcc-py37-cpu-r40|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py37-cpu-r40)|
|conda-linux-gcc-py37-cpu-r41|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py37-cpu-r41)|
|conda-linux-gcc-py37-cuda|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py37-cuda)|
|conda-linux-gcc-py37-ppc64le|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py37-ppc64le)|
|conda-linux-gcc-py38-arm64|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py38-arm64)|
|conda-linux-gcc-py38-cpu|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py38-cpu)|
|conda-linux-gcc-py38-cuda|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py38-cuda)|
|conda-linux-gcc-py38-ppc64le|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py38-ppc64le)|
|conda-linux-gcc-py39-arm64|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py39-arm64)|
|conda-linux-gcc-py39-cpu|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py39-cpu)|
|conda-linux-gcc-py39-cuda|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py39-cuda)|
|conda-linux-gcc-py39-ppc64le|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-linux-gcc-py39-ppc64le)|
|conda-osx-arm64-clang-py310|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-osx-arm64-clang-py310)|
|conda-osx-arm64-clang-py38|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-osx-arm64-clang-py38)|
|conda-osx-arm64-clang-py39|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-osx-arm64-clang-py39)|
|conda-osx-clang-py310|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-osx-clang-py310)|
|conda-osx-clang-py37-r40|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-osx-clang-py37-r40)|
|conda-osx-clang-py37-r41|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-osx-clang-py37-r41)|
|conda-osx-clang-py38|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-osx-clang-py38)|
|conda-osx-clang-py39|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-osx-clang-py39)|
|conda-win-vs2017-py310|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-win-vs2017-py310)|
|conda-win-vs2017-py37-r40|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-win-vs2017-py37-r40)|
|conda-win-vs2017-py37-r41|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-win-vs2017-py37-r41)|
|conda-win-vs2017-py38|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-win-vs2017-py38)|
|conda-win-vs2017-py39|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-conda-win-vs2017-py39)|
|debian-bookworm-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-debian-bookworm-amd64)|
|debian-bookworm-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|debian-bullseye-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-debian-bullseye-amd64)|
|debian-bullseye-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|example-cpp-minimal-build-static|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-example-cpp-minimal-build-static)|
|example-cpp-minimal-build-static-system-dependency|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-example-cpp-minimal-build-static-system-dependency)|
|example-python-minimal-build-fedora-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-example-python-minimal-build-fedora-conda)|
|example-python-minimal-build-ubuntu-venv|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-example-python-minimal-build-ubuntu-venv)|
|homebrew-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-homebrew-cpp)|
|homebrew-r-autobrew|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-homebrew-r-autobrew)|
|homebrew-r-brew|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-homebrew-r-brew)|
|java-jars|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-java-jars)|
|nuget|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-nuget)|
|python-sdist|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-python-sdist)|
|r-binary-packages|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-r-binary-packages)|
|test-alpine-linux-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-alpine-linux-cpp)|
|test-build-cpp-fuzz|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-build-cpp-fuzz)|
|test-build-vcpkg-win|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-build-vcpkg-win)|
|test-conda-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-cpp)|
|test-conda-cpp-valgrind|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-conda-cpp-valgrind)|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.10)|
|test-conda-python-3.7|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.7)|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.7-hdfs-3.2.1)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-0.24|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.7-pandas-0.24)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-spark-v3.1.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.7-spark-v3.1.2)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.8)|
|test-conda-python-3.8-hypothesis|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.8-hypothesis)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.8-pandas-nightly)|
|test-conda-python-3.8-spark-v3.2.0|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.8-spark-v3.2.0)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.9)|
|test-conda-python-3.9-dask-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.9-dask-latest)|
|test-conda-python-3.9-dask-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.9-dask-master)|
|test-conda-python-3.9-pandas-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.9-pandas-master)|
|test-conda-python-3.9-spark-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-conda-python-3.9-spark-master)|
|test-debian-10-cpp-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-debian-10-cpp-amd64)|
|test-debian-10-cpp-i386|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-debian-10-cpp-i386)|
|test-debian-11-cpp-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-debian-11-cpp-amd64)|
|test-debian-11-cpp-i386|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-debian-11-cpp-i386)|
|test-debian-11-go-1.16|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-debian-11-go-1.16)|
|test-debian-11-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-debian-11-python-3)|
|test-debian-c-glib|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-debian-c-glib)|
|test-debian-ruby|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-debian-ruby)|
|test-fedora-35-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-fedora-35-cpp)|
|test-fedora-35-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-fedora-35-python-3)|
|test-fedora-r-clang-sanitizer|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-fedora-r-clang-sanitizer)|
|test-r-arrow-backwards-compatibility|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-r-arrow-backwards-compatibility)|
|test-r-depsource-bundled|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-r-depsource-bundled)|
|test-r-depsource-system|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-r-depsource-system)|
|test-r-dev-duckdb|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-r-dev-duckdb)|
|test-r-devdocs|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-r-devdocs)|
|test-r-gcc-11|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-r-gcc-11)|
|test-r-gcc-12|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-r-gcc-12)|
|test-r-install-local|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-r-install-local)|
|test-r-linux-as-cran|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-r-linux-as-cran)|
|test-r-linux-rchk|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-r-linux-rchk)|
|test-r-linux-valgrind|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-r-linux-valgrind)|
|test-r-minimal-build|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-r-minimal-build)|
|test-r-offline-maximal|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-r-offline-maximal)|
|test-r-offline-minimal|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-r-offline-minimal)|
|test-r-rhub-debian-gcc-devel-lto-latest|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-r-rhub-debian-gcc-devel-lto-latest)|
|test-r-rhub-debian-gcc-release-custom-ccache|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-r-rhub-debian-gcc-release-custom-ccache)|
|test-r-rhub-ubuntu-gcc-release-latest|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-r-rhub-ubuntu-gcc-release-latest)|
|test-r-rocker-r-base-latest|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-r-rocker-r-base-latest)|
|test-r-rstudio-r-base-4.1-opensuse153|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-r-rstudio-r-base-4.1-opensuse153)|
|test-r-rstudio-r-base-4.2-centos7-devtoolset-8|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-r-rstudio-r-base-4.2-centos7-devtoolset-8)|
|test-r-rstudio-r-base-4.2-focal|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-r-rstudio-r-base-4.2-focal)|
|test-r-ubuntu-22.04|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-r-ubuntu-22.04)|
|test-r-versions|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-r-versions)|
|test-skyhook-integration|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-skyhook-integration)|
|test-ubuntu-18.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-ubuntu-18.04-cpp)|
|test-ubuntu-18.04-cpp-release|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-ubuntu-18.04-cpp-release)|
|test-ubuntu-18.04-cpp-static|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-ubuntu-18.04-cpp-static)|
|test-ubuntu-18.04-r-sanitizer|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-ubuntu-18.04-r-sanitizer)|
|test-ubuntu-20.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-ubuntu-20.04-cpp)|
|test-ubuntu-20.04-cpp-14|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-ubuntu-20.04-cpp-14)|
|test-ubuntu-20.04-cpp-17|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-ubuntu-20.04-cpp-17)|
|test-ubuntu-20.04-cpp-bundled|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-ubuntu-20.04-cpp-bundled)|
|test-ubuntu-20.04-cpp-thread-sanitizer|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-ubuntu-20.04-cpp-thread-sanitizer)|
|test-ubuntu-20.04-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-ubuntu-20.04-python-3)|
|test-ubuntu-22.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-ubuntu-22.04-cpp)|
|test-ubuntu-c-glib|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-ubuntu-c-glib)|
|test-ubuntu-default-docs|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-a9994de6ad-azure-test-ubuntu-default-docs)|
|test-ubuntu-ruby|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-test-ubuntu-ruby)|
|ubuntu-bionic-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-ubuntu-bionic-amd64)|
|ubuntu-bionic-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|ubuntu-focal-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-ubuntu-focal-amd64)|
|ubuntu-focal-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|ubuntu-jammy-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-ubuntu-jammy-amd64)|
|ubuntu-jammy-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|verify-rc-source-cpp-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-cpp-linux-almalinux-8-amd64)|
|verify-rc-source-cpp-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-cpp-linux-conda-latest-amd64)|
|verify-rc-source-cpp-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-cpp-linux-ubuntu-18.04-amd64)|
|verify-rc-source-cpp-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-cpp-linux-ubuntu-20.04-amd64)|
|verify-rc-source-cpp-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-cpp-linux-ubuntu-22.04-amd64)|
|verify-rc-source-cpp-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-cpp-macos-amd64)|
|verify-rc-source-cpp-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-cpp-macos-arm64)|
|verify-rc-source-cpp-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-cpp-macos-conda-amd64)|
|verify-rc-source-csharp-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-csharp-linux-almalinux-8-amd64)|
|verify-rc-source-csharp-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-csharp-linux-conda-latest-amd64)|
|verify-rc-source-csharp-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-csharp-linux-ubuntu-18.04-amd64)|
|verify-rc-source-csharp-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-csharp-linux-ubuntu-20.04-amd64)|
|verify-rc-source-csharp-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-csharp-linux-ubuntu-22.04-amd64)|
|verify-rc-source-csharp-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-csharp-macos-amd64)|
|verify-rc-source-csharp-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-csharp-macos-arm64)|
|verify-rc-source-go-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-go-linux-almalinux-8-amd64)|
|verify-rc-source-go-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-go-linux-conda-latest-amd64)|
|verify-rc-source-go-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-go-linux-ubuntu-18.04-amd64)|
|verify-rc-source-go-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-go-linux-ubuntu-20.04-amd64)|
|verify-rc-source-go-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-go-linux-ubuntu-22.04-amd64)|
|verify-rc-source-go-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-go-macos-amd64)|
|verify-rc-source-go-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-go-macos-arm64)|
|verify-rc-source-integration-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-integration-linux-almalinux-8-amd64)|
|verify-rc-source-integration-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-integration-linux-conda-latest-amd64)|
|verify-rc-source-integration-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-integration-linux-ubuntu-18.04-amd64)|
|verify-rc-source-integration-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-integration-linux-ubuntu-20.04-amd64)|
|verify-rc-source-integration-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-integration-linux-ubuntu-22.04-amd64)|
|verify-rc-source-integration-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-integration-macos-amd64)|
|verify-rc-source-integration-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-integration-macos-arm64)|
|verify-rc-source-integration-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-integration-macos-conda-amd64)|
|verify-rc-source-java-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-java-linux-almalinux-8-amd64)|
|verify-rc-source-java-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-java-linux-conda-latest-amd64)|
|verify-rc-source-java-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-java-linux-ubuntu-18.04-amd64)|
|verify-rc-source-java-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-java-linux-ubuntu-20.04-amd64)|
|verify-rc-source-java-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-java-linux-ubuntu-22.04-amd64)|
|verify-rc-source-java-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-java-macos-amd64)|
|verify-rc-source-js-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-js-linux-almalinux-8-amd64)|
|verify-rc-source-js-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-js-linux-conda-latest-amd64)|
|verify-rc-source-js-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-js-linux-ubuntu-18.04-amd64)|
|verify-rc-source-js-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-js-linux-ubuntu-20.04-amd64)|
|verify-rc-source-js-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-js-linux-ubuntu-22.04-amd64)|
|verify-rc-source-js-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-js-macos-amd64)|
|verify-rc-source-js-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-js-macos-arm64)|
|verify-rc-source-python-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-python-linux-almalinux-8-amd64)|
|verify-rc-source-python-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-python-linux-conda-latest-amd64)|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
|verify-rc-source-python-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-python-linux-ubuntu-20.04-amd64)|
|verify-rc-source-python-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-python-linux-ubuntu-22.04-amd64)|
|verify-rc-source-python-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-python-macos-amd64)|
|verify-rc-source-python-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-python-macos-arm64)|
|verify-rc-source-python-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-python-macos-conda-amd64)|
|verify-rc-source-ruby-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-ruby-linux-almalinux-8-amd64)|
|verify-rc-source-ruby-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-ruby-linux-conda-latest-amd64)|
|verify-rc-source-ruby-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-ruby-linux-ubuntu-18.04-amd64)|
|verify-rc-source-ruby-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-ruby-linux-ubuntu-20.04-amd64)|
|verify-rc-source-ruby-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-ruby-linux-ubuntu-22.04-amd64)|
|verify-rc-source-ruby-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-ruby-macos-amd64)|
|verify-rc-source-ruby-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-ruby-macos-arm64)|
|verify-rc-source-windows|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-verify-rc-source-windows)|
|wheel-macos-big-sur-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-macos-big-sur-cp310-arm64)|
|wheel-macos-big-sur-cp310-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-macos-big-sur-cp310-universal2)|
|wheel-macos-big-sur-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-macos-big-sur-cp38-arm64)|
|wheel-macos-big-sur-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-macos-big-sur-cp39-arm64)|
|wheel-macos-big-sur-cp39-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-macos-big-sur-cp39-universal2)|
|wheel-macos-high-sierra-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-macos-high-sierra-cp310-amd64)|
|wheel-macos-high-sierra-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-macos-high-sierra-cp37-amd64)|
|wheel-macos-high-sierra-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-macos-high-sierra-cp38-amd64)|
|wheel-macos-high-sierra-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-macos-high-sierra-cp39-amd64)|
|wheel-macos-mavericks-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-macos-mavericks-cp310-amd64)|
|wheel-macos-mavericks-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-macos-mavericks-cp37-amd64)|
|wheel-macos-mavericks-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-macos-mavericks-cp38-amd64)|
|wheel-macos-mavericks-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-macos-mavericks-cp39-amd64)|
|wheel-manylinux2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-manylinux2014-cp310-amd64)|
|wheel-manylinux2014-cp310-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-manylinux2014-cp37-amd64)|
|wheel-manylinux2014-cp37-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-manylinux2014-cp38-amd64)|
|wheel-manylinux2014-cp38-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-manylinux2014-cp39-amd64)|
|wheel-manylinux2014-cp39-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-windows-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-windows-cp310-amd64)|
|wheel-windows-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-windows-cp37-amd64)|
|wheel-windows-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-windows-cp38-amd64)|
|wheel-windows-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-a9994de6ad-github-wheel-windows-cp39-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1225224335
```
Invalid group(s) {'centos', 'almalinux'}. Must be one of {'python', 'test', 'example', 'r', 'verify-rc', 'linux-amd64', 'verify-rc-jars', 'nightly', 'conda', 'nightly-release', 'nightly-packaging', 'homebrew', 'verify-rc-source-macos', 'verify-rc-source-linux', 'fuzz', 'linux-arm64', 'verify-rc-source', 'nightly-tests', 'verify-rc-binaries', 'ruby', 'wheel', 'example-cpp', 'example-python', 'integration', 'conan', 'c-glib', 'packaging', 'verify-rc-wheels', 'vcpkg', 'linux', 'cpp'}
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/2916640822
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1226948035
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1220514241
Revision: f416f2ee644de25c7931ed88c581b39f93e71b01
Submitted crossbow builds: [ursacomputing/crossbow @ actions-fece2a76cc](https://github.com/ursacomputing/crossbow/branches/all?query=actions-fece2a76cc)
|Task|Status|
|----|------|
|example-python-minimal-build-fedora-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-example-python-minimal-build-fedora-conda)|
|example-python-minimal-build-ubuntu-venv|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-example-python-minimal-build-ubuntu-venv)|
|python-sdist|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-python-sdist)|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.10)|
|test-conda-python-3.7|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.7)|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.7-hdfs-3.2.1)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-0.24|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.7-pandas-0.24)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-spark-v3.1.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.7-spark-v3.1.2)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.8)|
|test-conda-python-3.8-hypothesis|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.8-hypothesis)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.8-pandas-nightly)|
|test-conda-python-3.8-spark-v3.2.0|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.8-spark-v3.2.0)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.9)|
|test-conda-python-3.9-dask-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.9-dask-latest)|
|test-conda-python-3.9-dask-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.9-dask-master)|
|test-conda-python-3.9-pandas-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.9-pandas-master)|
|test-conda-python-3.9-spark-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-test-conda-python-3.9-spark-master)|
|test-debian-11-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-fece2a76cc-azure-test-debian-11-python-3)|
|test-fedora-35-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-fece2a76cc-azure-test-fedora-35-python-3)|
|test-ubuntu-20.04-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-fece2a76cc-azure-test-ubuntu-20.04-python-3)|
|verify-rc-source-python-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-verify-rc-source-python-linux-almalinux-8-amd64)|
|verify-rc-source-python-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-verify-rc-source-python-linux-conda-latest-amd64)|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
|verify-rc-source-python-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-verify-rc-source-python-linux-ubuntu-20.04-amd64)|
|verify-rc-source-python-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-verify-rc-source-python-linux-ubuntu-22.04-amd64)|
|verify-rc-source-python-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-verify-rc-source-python-macos-amd64)|
|verify-rc-source-python-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-verify-rc-source-python-macos-arm64)|
|verify-rc-source-python-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-fece2a76cc-github-verify-rc-source-python-macos-conda-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1220514874
Revision: f416f2ee644de25c7931ed88c581b39f93e71b01
Submitted crossbow builds: [ursacomputing/crossbow @ actions-3e88808590](https://github.com/ursacomputing/crossbow/branches/all?query=actions-3e88808590)
|Task|Status|
|----|------|
|verify-rc-binaries-wheels-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-verify-rc-binaries-wheels-linux-almalinux-8-amd64)|
|verify-rc-binaries-wheels-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-verify-rc-binaries-wheels-linux-conda-latest-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64)|
|verify-rc-binaries-wheels-macos-11-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-verify-rc-binaries-wheels-macos-11-amd64)|
|verify-rc-binaries-wheels-macos-11-amd64-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-verify-rc-binaries-wheels-macos-11-amd64-conda)|
|verify-rc-binaries-wheels-macos-11-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-verify-rc-binaries-wheels-macos-11-arm64)|
|verify-rc-binaries-wheels-macos-12-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-verify-rc-binaries-wheels-macos-12-amd64)|
|verify-rc-binaries-wheels-macos-12-amd64-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-verify-rc-binaries-wheels-macos-12-amd64-conda)|
|verify-rc-binaries-wheels-windows|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-verify-rc-binaries-wheels-windows)|
|wheel-macos-big-sur-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-macos-big-sur-cp310-arm64)|
|wheel-macos-big-sur-cp310-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-macos-big-sur-cp310-universal2)|
|wheel-macos-big-sur-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-macos-big-sur-cp38-arm64)|
|wheel-macos-big-sur-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-macos-big-sur-cp39-arm64)|
|wheel-macos-big-sur-cp39-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-macos-big-sur-cp39-universal2)|
|wheel-macos-high-sierra-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-macos-high-sierra-cp310-amd64)|
|wheel-macos-high-sierra-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-macos-high-sierra-cp37-amd64)|
|wheel-macos-high-sierra-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-macos-high-sierra-cp38-amd64)|
|wheel-macos-high-sierra-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-macos-high-sierra-cp39-amd64)|
|wheel-macos-mavericks-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-macos-mavericks-cp310-amd64)|
|wheel-macos-mavericks-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-macos-mavericks-cp37-amd64)|
|wheel-macos-mavericks-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-macos-mavericks-cp38-amd64)|
|wheel-macos-mavericks-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-macos-mavericks-cp39-amd64)|
|wheel-manylinux2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-manylinux2014-cp310-amd64)|
|wheel-manylinux2014-cp310-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-manylinux2014-cp37-amd64)|
|wheel-manylinux2014-cp37-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-manylinux2014-cp38-amd64)|
|wheel-manylinux2014-cp38-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-manylinux2014-cp39-amd64)|
|wheel-manylinux2014-cp39-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-windows-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-windows-cp310-amd64)|
|wheel-windows-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-windows-cp37-amd64)|
|wheel-windows-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-windows-cp38-amd64)|
|wheel-windows-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-3e88808590-github-wheel-windows-cp39-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1219292089
@github-actions crossbow submit *python*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1219292420
@github-actions crossbow submit *wheel*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1219346991
Revision: e0cb6e1f22d7c5f3f7741ff079bb7d63ed44cb10
Submitted crossbow builds: [ursacomputing/crossbow @ actions-c4b9f1b1d6](https://github.com/ursacomputing/crossbow/branches/all?query=actions-c4b9f1b1d6)
|Task|Status|
|----|------|
|verify-rc-binaries-wheels-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-verify-rc-binaries-wheels-linux-almalinux-8-amd64)|
|verify-rc-binaries-wheels-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-verify-rc-binaries-wheels-linux-conda-latest-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64)|
|verify-rc-binaries-wheels-macos-11-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-verify-rc-binaries-wheels-macos-11-amd64)|
|verify-rc-binaries-wheels-macos-11-amd64-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-verify-rc-binaries-wheels-macos-11-amd64-conda)|
|verify-rc-binaries-wheels-macos-11-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-verify-rc-binaries-wheels-macos-11-arm64)|
|verify-rc-binaries-wheels-macos-12-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-verify-rc-binaries-wheels-macos-12-amd64)|
|verify-rc-binaries-wheels-macos-12-amd64-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-verify-rc-binaries-wheels-macos-12-amd64-conda)|
|verify-rc-binaries-wheels-windows|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-verify-rc-binaries-wheels-windows)|
|wheel-macos-big-sur-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-macos-big-sur-cp310-arm64)|
|wheel-macos-big-sur-cp310-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-macos-big-sur-cp310-universal2)|
|wheel-macos-big-sur-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-macos-big-sur-cp38-arm64)|
|wheel-macos-big-sur-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-macos-big-sur-cp39-arm64)|
|wheel-macos-big-sur-cp39-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-macos-big-sur-cp39-universal2)|
|wheel-macos-high-sierra-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-macos-high-sierra-cp310-amd64)|
|wheel-macos-high-sierra-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-macos-high-sierra-cp37-amd64)|
|wheel-macos-high-sierra-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-macos-high-sierra-cp38-amd64)|
|wheel-macos-high-sierra-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-macos-high-sierra-cp39-amd64)|
|wheel-macos-mavericks-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-macos-mavericks-cp310-amd64)|
|wheel-macos-mavericks-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-macos-mavericks-cp37-amd64)|
|wheel-macos-mavericks-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-macos-mavericks-cp38-amd64)|
|wheel-macos-mavericks-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-macos-mavericks-cp39-amd64)|
|wheel-manylinux2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-manylinux2014-cp310-amd64)|
|wheel-manylinux2014-cp310-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-manylinux2014-cp37-amd64)|
|wheel-manylinux2014-cp37-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-manylinux2014-cp38-amd64)|
|wheel-manylinux2014-cp38-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-manylinux2014-cp39-amd64)|
|wheel-manylinux2014-cp39-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-windows-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-windows-cp310-amd64)|
|wheel-windows-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-windows-cp37-amd64)|
|wheel-windows-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-windows-cp38-amd64)|
|wheel-windows-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c4b9f1b1d6-github-wheel-windows-cp39-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r949087408
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,457 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
Review Comment:
Yes, correct! Will change it manually, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1145975530
https://issues.apache.org/jira/browse/ARROW-16340
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r949086432
##########
python/pyarrow/tests/test_cython.py:
##########
@@ -136,9 +138,9 @@ def test_cython_api(tmpdir):
delim, var = ':', 'LD_LIBRARY_PATH'
subprocess_env[var] = delim.join(
- pa.get_library_dirs() + [subprocess_env.get(var, '')]
+ pa.get_library_dirs() +
+ [subprocess_env.get(var, '')] + [str(tmpdir)]
Review Comment:
This is a leftover from debugging a Win CI failure. Will remove it 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1191834244
@github-actions crossbow submit *wheel*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1193324689
Revision: a9a3e912adf70b18c8cd7c572221f103414dcea8
Submitted crossbow builds: [ursacomputing/crossbow @ actions-b43d962138](https://github.com/ursacomputing/crossbow/branches/all?query=actions-b43d962138)
|Task|Status|
|----|------|
|verify-rc-binaries-wheels-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-verify-rc-binaries-wheels-linux-almalinux-8-amd64)|
|verify-rc-binaries-wheels-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-verify-rc-binaries-wheels-linux-conda-latest-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64)|
|verify-rc-binaries-wheels-macos-10.15-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-verify-rc-binaries-wheels-macos-10.15-amd64)|
|verify-rc-binaries-wheels-macos-10.15-amd64-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-verify-rc-binaries-wheels-macos-10.15-amd64-conda)|
|verify-rc-binaries-wheels-macos-11-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-verify-rc-binaries-wheels-macos-11-amd64)|
|verify-rc-binaries-wheels-macos-11-amd64-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-verify-rc-binaries-wheels-macos-11-amd64-conda)|
|verify-rc-binaries-wheels-macos-11-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-verify-rc-binaries-wheels-macos-11-arm64)|
|verify-rc-binaries-wheels-windows|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-verify-rc-binaries-wheels-windows)|
|wheel-macos-big-sur-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-macos-big-sur-cp310-arm64)|
|wheel-macos-big-sur-cp310-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-macos-big-sur-cp310-universal2)|
|wheel-macos-big-sur-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-macos-big-sur-cp38-arm64)|
|wheel-macos-big-sur-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-macos-big-sur-cp39-arm64)|
|wheel-macos-big-sur-cp39-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-macos-big-sur-cp39-universal2)|
|wheel-macos-high-sierra-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-macos-high-sierra-cp310-amd64)|
|wheel-macos-high-sierra-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-macos-high-sierra-cp37-amd64)|
|wheel-macos-high-sierra-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-macos-high-sierra-cp38-amd64)|
|wheel-macos-high-sierra-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-macos-high-sierra-cp39-amd64)|
|wheel-macos-mavericks-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-macos-mavericks-cp310-amd64)|
|wheel-macos-mavericks-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-macos-mavericks-cp37-amd64)|
|wheel-macos-mavericks-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-macos-mavericks-cp38-amd64)|
|wheel-macos-mavericks-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-macos-mavericks-cp39-amd64)|
|wheel-manylinux2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-manylinux2014-cp310-amd64)|
|wheel-manylinux2014-cp310-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-manylinux2014-cp37-amd64)|
|wheel-manylinux2014-cp37-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-manylinux2014-cp38-amd64)|
|wheel-manylinux2014-cp38-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-manylinux2014-cp39-amd64)|
|wheel-manylinux2014-cp39-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-windows-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-windows-cp310-amd64)|
|wheel-windows-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-windows-cp37-amd64)|
|wheel-windows-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-windows-cp38-amd64)|
|wheel-windows-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-b43d962138-github-wheel-windows-cp39-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1223348186
Revision: f416f2ee644de25c7931ed88c581b39f93e71b01
Submitted crossbow builds: [ursacomputing/crossbow @ actions-c36c9be408](https://github.com/ursacomputing/crossbow/branches/all?query=actions-c36c9be408)
|Task|Status|
|----|------|
|almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-almalinux-8-amd64)|
|almalinux-8-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|almalinux-9-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-almalinux-9-amd64)|
|almalinux-9-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|amazon-linux-2-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-amazon-linux-2-amd64)|
|amazon-linux-2-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|centos-7-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-centos-7-amd64)|
|centos-8-stream-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-centos-8-stream-amd64)|
|centos-8-stream-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|centos-9-stream-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-centos-9-stream-amd64)|
|centos-9-stream-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|conan-maximum|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-conan-maximum)|
|conan-minimum|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-conan-minimum)|
|conda-clean|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-clean)|
|conda-linux-gcc-py310-arm64|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py310-arm64)|
|conda-linux-gcc-py310-cpu|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py310-cpu)|
|conda-linux-gcc-py310-cuda|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py310-cuda)|
|conda-linux-gcc-py310-ppc64le|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py310-ppc64le)|
|conda-linux-gcc-py37-arm64|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py37-arm64)|
|conda-linux-gcc-py37-cpu-r40|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py37-cpu-r40)|
|conda-linux-gcc-py37-cpu-r41|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py37-cpu-r41)|
|conda-linux-gcc-py37-cuda|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py37-cuda)|
|conda-linux-gcc-py37-ppc64le|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py37-ppc64le)|
|conda-linux-gcc-py38-arm64|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py38-arm64)|
|conda-linux-gcc-py38-cpu|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py38-cpu)|
|conda-linux-gcc-py38-cuda|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py38-cuda)|
|conda-linux-gcc-py38-ppc64le|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py38-ppc64le)|
|conda-linux-gcc-py39-arm64|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py39-arm64)|
|conda-linux-gcc-py39-cpu|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py39-cpu)|
|conda-linux-gcc-py39-cuda|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py39-cuda)|
|conda-linux-gcc-py39-ppc64le|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-linux-gcc-py39-ppc64le)|
|conda-osx-arm64-clang-py310|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-osx-arm64-clang-py310)|
|conda-osx-arm64-clang-py38|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-osx-arm64-clang-py38)|
|conda-osx-arm64-clang-py39|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-osx-arm64-clang-py39)|
|conda-osx-clang-py310|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-osx-clang-py310)|
|conda-osx-clang-py37-r40|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-osx-clang-py37-r40)|
|conda-osx-clang-py37-r41|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-osx-clang-py37-r41)|
|conda-osx-clang-py38|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-osx-clang-py38)|
|conda-osx-clang-py39|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-osx-clang-py39)|
|conda-win-vs2017-py310|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-win-vs2017-py310)|
|conda-win-vs2017-py37-r40|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-win-vs2017-py37-r40)|
|conda-win-vs2017-py37-r41|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-win-vs2017-py37-r41)|
|conda-win-vs2017-py38|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-win-vs2017-py38)|
|conda-win-vs2017-py39|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-conda-win-vs2017-py39)|
|debian-bookworm-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-debian-bookworm-amd64)|
|debian-bookworm-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|debian-bullseye-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-debian-bullseye-amd64)|
|debian-bullseye-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|example-cpp-minimal-build-static|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-example-cpp-minimal-build-static)|
|example-cpp-minimal-build-static-system-dependency|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-example-cpp-minimal-build-static-system-dependency)|
|example-python-minimal-build-fedora-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-example-python-minimal-build-fedora-conda)|
|example-python-minimal-build-ubuntu-venv|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-example-python-minimal-build-ubuntu-venv)|
|homebrew-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-homebrew-cpp)|
|homebrew-r-autobrew|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-homebrew-r-autobrew)|
|homebrew-r-brew|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-homebrew-r-brew)|
|java-jars|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-java-jars)|
|nuget|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-nuget)|
|python-sdist|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-python-sdist)|
|r-binary-packages|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-r-binary-packages)|
|test-alpine-linux-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-alpine-linux-cpp)|
|test-build-cpp-fuzz|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-build-cpp-fuzz)|
|test-build-vcpkg-win|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-build-vcpkg-win)|
|test-conda-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-cpp)|
|test-conda-cpp-valgrind|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-conda-cpp-valgrind)|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.10)|
|test-conda-python-3.7|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.7)|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.7-hdfs-3.2.1)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-0.24|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.7-pandas-0.24)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-spark-v3.1.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.7-spark-v3.1.2)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.8)|
|test-conda-python-3.8-hypothesis|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.8-hypothesis)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.8-pandas-nightly)|
|test-conda-python-3.8-spark-v3.2.0|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.8-spark-v3.2.0)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.9)|
|test-conda-python-3.9-dask-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.9-dask-latest)|
|test-conda-python-3.9-dask-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.9-dask-master)|
|test-conda-python-3.9-pandas-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.9-pandas-master)|
|test-conda-python-3.9-spark-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-conda-python-3.9-spark-master)|
|test-debian-10-cpp-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-debian-10-cpp-amd64)|
|test-debian-10-cpp-i386|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-debian-10-cpp-i386)|
|test-debian-11-cpp-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-debian-11-cpp-amd64)|
|test-debian-11-cpp-i386|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-debian-11-cpp-i386)|
|test-debian-11-go-1.16|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-debian-11-go-1.16)|
|test-debian-11-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-debian-11-python-3)|
|test-debian-c-glib|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-debian-c-glib)|
|test-debian-ruby|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-debian-ruby)|
|test-fedora-35-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-fedora-35-cpp)|
|test-fedora-35-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-fedora-35-python-3)|
|test-fedora-r-clang-sanitizer|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-fedora-r-clang-sanitizer)|
|test-r-arrow-backwards-compatibility|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-r-arrow-backwards-compatibility)|
|test-r-depsource-bundled|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-r-depsource-bundled)|
|test-r-depsource-system|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-r-depsource-system)|
|test-r-dev-duckdb|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-r-dev-duckdb)|
|test-r-devdocs|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-r-devdocs)|
|test-r-gcc-11|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-r-gcc-11)|
|test-r-gcc-12|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-r-gcc-12)|
|test-r-install-local|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-r-install-local)|
|test-r-linux-as-cran|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-r-linux-as-cran)|
|test-r-linux-rchk|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-r-linux-rchk)|
|test-r-linux-valgrind|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-r-linux-valgrind)|
|test-r-minimal-build|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-r-minimal-build)|
|test-r-offline-maximal|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-r-offline-maximal)|
|test-r-offline-minimal|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-r-offline-minimal)|
|test-r-rhub-debian-gcc-devel-lto-latest|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-r-rhub-debian-gcc-devel-lto-latest)|
|test-r-rhub-debian-gcc-release-custom-ccache|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-r-rhub-debian-gcc-release-custom-ccache)|
|test-r-rhub-ubuntu-gcc-release-latest|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-r-rhub-ubuntu-gcc-release-latest)|
|test-r-rocker-r-base-latest|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-r-rocker-r-base-latest)|
|test-r-rstudio-r-base-4.1-opensuse153|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-r-rstudio-r-base-4.1-opensuse153)|
|test-r-rstudio-r-base-4.2-centos7-devtoolset-8|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-r-rstudio-r-base-4.2-centos7-devtoolset-8)|
|test-r-rstudio-r-base-4.2-focal|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-r-rstudio-r-base-4.2-focal)|
|test-r-ubuntu-22.04|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-r-ubuntu-22.04)|
|test-r-versions|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-r-versions)|
|test-skyhook-integration|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-skyhook-integration)|
|test-ubuntu-18.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-ubuntu-18.04-cpp)|
|test-ubuntu-18.04-cpp-release|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-ubuntu-18.04-cpp-release)|
|test-ubuntu-18.04-cpp-static|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-ubuntu-18.04-cpp-static)|
|test-ubuntu-18.04-r-sanitizer|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-ubuntu-18.04-r-sanitizer)|
|test-ubuntu-20.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-ubuntu-20.04-cpp)|
|test-ubuntu-20.04-cpp-14|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-ubuntu-20.04-cpp-14)|
|test-ubuntu-20.04-cpp-17|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-ubuntu-20.04-cpp-17)|
|test-ubuntu-20.04-cpp-bundled|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-ubuntu-20.04-cpp-bundled)|
|test-ubuntu-20.04-cpp-thread-sanitizer|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-ubuntu-20.04-cpp-thread-sanitizer)|
|test-ubuntu-20.04-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-ubuntu-20.04-python-3)|
|test-ubuntu-22.04-cpp|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-ubuntu-22.04-cpp)|
|test-ubuntu-c-glib|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-ubuntu-c-glib)|
|test-ubuntu-default-docs|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-c36c9be408-azure-test-ubuntu-default-docs)|
|test-ubuntu-ruby|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-test-ubuntu-ruby)|
|ubuntu-bionic-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-ubuntu-bionic-amd64)|
|ubuntu-bionic-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|ubuntu-focal-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-ubuntu-focal-amd64)|
|ubuntu-focal-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|ubuntu-jammy-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-ubuntu-jammy-amd64)|
|ubuntu-jammy-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|verify-rc-source-cpp-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-cpp-linux-almalinux-8-amd64)|
|verify-rc-source-cpp-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-cpp-linux-conda-latest-amd64)|
|verify-rc-source-cpp-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-cpp-linux-ubuntu-18.04-amd64)|
|verify-rc-source-cpp-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-cpp-linux-ubuntu-20.04-amd64)|
|verify-rc-source-cpp-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-cpp-linux-ubuntu-22.04-amd64)|
|verify-rc-source-cpp-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-cpp-macos-amd64)|
|verify-rc-source-cpp-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-cpp-macos-arm64)|
|verify-rc-source-cpp-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-cpp-macos-conda-amd64)|
|verify-rc-source-csharp-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-csharp-linux-almalinux-8-amd64)|
|verify-rc-source-csharp-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-csharp-linux-conda-latest-amd64)|
|verify-rc-source-csharp-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-csharp-linux-ubuntu-18.04-amd64)|
|verify-rc-source-csharp-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-csharp-linux-ubuntu-20.04-amd64)|
|verify-rc-source-csharp-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-csharp-linux-ubuntu-22.04-amd64)|
|verify-rc-source-csharp-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-csharp-macos-amd64)|
|verify-rc-source-csharp-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-csharp-macos-arm64)|
|verify-rc-source-go-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-go-linux-almalinux-8-amd64)|
|verify-rc-source-go-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-go-linux-conda-latest-amd64)|
|verify-rc-source-go-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-go-linux-ubuntu-18.04-amd64)|
|verify-rc-source-go-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-go-linux-ubuntu-20.04-amd64)|
|verify-rc-source-go-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-go-linux-ubuntu-22.04-amd64)|
|verify-rc-source-go-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-go-macos-amd64)|
|verify-rc-source-go-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-go-macos-arm64)|
|verify-rc-source-integration-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-integration-linux-almalinux-8-amd64)|
|verify-rc-source-integration-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-integration-linux-conda-latest-amd64)|
|verify-rc-source-integration-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-integration-linux-ubuntu-18.04-amd64)|
|verify-rc-source-integration-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-integration-linux-ubuntu-20.04-amd64)|
|verify-rc-source-integration-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-integration-linux-ubuntu-22.04-amd64)|
|verify-rc-source-integration-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-integration-macos-amd64)|
|verify-rc-source-integration-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-integration-macos-arm64)|
|verify-rc-source-integration-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-integration-macos-conda-amd64)|
|verify-rc-source-java-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-java-linux-almalinux-8-amd64)|
|verify-rc-source-java-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-java-linux-conda-latest-amd64)|
|verify-rc-source-java-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-java-linux-ubuntu-18.04-amd64)|
|verify-rc-source-java-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-java-linux-ubuntu-20.04-amd64)|
|verify-rc-source-java-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-java-linux-ubuntu-22.04-amd64)|
|verify-rc-source-java-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-java-macos-amd64)|
|verify-rc-source-js-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-js-linux-almalinux-8-amd64)|
|verify-rc-source-js-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-js-linux-conda-latest-amd64)|
|verify-rc-source-js-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-js-linux-ubuntu-18.04-amd64)|
|verify-rc-source-js-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-js-linux-ubuntu-20.04-amd64)|
|verify-rc-source-js-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-js-linux-ubuntu-22.04-amd64)|
|verify-rc-source-js-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-js-macos-amd64)|
|verify-rc-source-js-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-js-macos-arm64)|
|verify-rc-source-python-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-python-linux-almalinux-8-amd64)|
|verify-rc-source-python-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-python-linux-conda-latest-amd64)|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
|verify-rc-source-python-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-python-linux-ubuntu-20.04-amd64)|
|verify-rc-source-python-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-python-linux-ubuntu-22.04-amd64)|
|verify-rc-source-python-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-python-macos-amd64)|
|verify-rc-source-python-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-python-macos-arm64)|
|verify-rc-source-python-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-python-macos-conda-amd64)|
|verify-rc-source-ruby-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-ruby-linux-almalinux-8-amd64)|
|verify-rc-source-ruby-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-ruby-linux-conda-latest-amd64)|
|verify-rc-source-ruby-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-ruby-linux-ubuntu-18.04-amd64)|
|verify-rc-source-ruby-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-ruby-linux-ubuntu-20.04-amd64)|
|verify-rc-source-ruby-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-ruby-linux-ubuntu-22.04-amd64)|
|verify-rc-source-ruby-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-ruby-macos-amd64)|
|verify-rc-source-ruby-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-ruby-macos-arm64)|
|verify-rc-source-windows|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-verify-rc-source-windows)|
|wheel-macos-big-sur-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-macos-big-sur-cp310-arm64)|
|wheel-macos-big-sur-cp310-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-macos-big-sur-cp310-universal2)|
|wheel-macos-big-sur-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-macos-big-sur-cp38-arm64)|
|wheel-macos-big-sur-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-macos-big-sur-cp39-arm64)|
|wheel-macos-big-sur-cp39-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-macos-big-sur-cp39-universal2)|
|wheel-macos-high-sierra-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-macos-high-sierra-cp310-amd64)|
|wheel-macos-high-sierra-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-macos-high-sierra-cp37-amd64)|
|wheel-macos-high-sierra-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-macos-high-sierra-cp38-amd64)|
|wheel-macos-high-sierra-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-macos-high-sierra-cp39-amd64)|
|wheel-macos-mavericks-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-macos-mavericks-cp310-amd64)|
|wheel-macos-mavericks-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-macos-mavericks-cp37-amd64)|
|wheel-macos-mavericks-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-macos-mavericks-cp38-amd64)|
|wheel-macos-mavericks-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-macos-mavericks-cp39-amd64)|
|wheel-manylinux2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-manylinux2014-cp310-amd64)|
|wheel-manylinux2014-cp310-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-manylinux2014-cp37-amd64)|
|wheel-manylinux2014-cp37-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-manylinux2014-cp38-amd64)|
|wheel-manylinux2014-cp38-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-manylinux2014-cp39-amd64)|
|wheel-manylinux2014-cp39-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-windows-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-windows-cp310-amd64)|
|wheel-windows-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-windows-cp37-amd64)|
|wheel-windows-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-windows-cp38-amd64)|
|wheel-windows-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c36c9be408-github-wheel-windows-cp39-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #13311: ARROW-16340: [C++][Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1228048577
Could you update the description of this pull request before we merge this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
pitrou commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1226868085
Also I've restarted the ubuntu-jammy build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r913770195
##########
python/pyarrow/src_arrow/flight.cc:
##########
@@ -371,24 +370,21 @@ Status CreateFlightInfo(const std::shared_ptr<arrow::Schema>& schema,
const std::vector<arrow::flight::FlightEndpoint>& endpoints,
int64_t total_records, int64_t total_bytes,
std::unique_ptr<arrow::flight::FlightInfo>* out) {
- arrow::flight::FlightInfo::Data flight_data;
- RETURN_NOT_OK(arrow::flight::internal::SchemaToString(*schema, &flight_data.schema));
- flight_data.descriptor = descriptor;
- flight_data.endpoints = endpoints;
- flight_data.total_records = total_records;
- flight_data.total_bytes = total_bytes;
- arrow::flight::FlightInfo value(flight_data);
- *out = std::unique_ptr<arrow::flight::FlightInfo>(new arrow::flight::FlightInfo(value));
+ auto result =
+ arrow::flight::FlightInfo::Make(*schema,
+ descriptor,
+ endpoints,
+ total_records,
+ total_bytes);
+ *out = std::unique_ptr<arrow::flight::FlightInfo>(&(*result));
Review Comment:
Oh, haven't tried that one. Awesome, thank you very very much =)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1171927470
I would like to put this PR ready for review.
There are still two CI failures I am investigating (see ⬇️ ) plus `test_cython_api()` from `test_cython.py` is broke, so I am researching it at the moment. But I think this shouldn't block this PR to get initial reviews 🙏 @kou @pitrou @kszucs @jorisvandenbossche
- `Could NOT find Python3 (missing: Python3_NumPy_INCLUDE_DIRS NumPy)`
I have seen this error in some other failing builds and am not sure if it is connected to this PR or not.
```
CMake Error at /opt/conda/envs/arrow/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find Python3 (missing: Python3_NumPy_INCLUDE_DIRS NumPy) (found
suitable version "3.9.13", minimum required is "3.7")
Call Stack (most recent call first):
/opt/conda/envs/arrow/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
/opt/conda/envs/arrow/share/cmake-3.23/Modules/FindPython/Support.cmake:3192 (find_package_handle_standard_args)
/opt/conda/envs/arrow/share/cmake-3.23/Modules/FindPython3.cmake:490 (include)
/arrow/python/cmake_modules/FindPython3Alt.cmake:56 (find_package)
CMakeLists.txt:64 (find_package)
```
- `fatal error: arrow/flight/Flight.pb.h: No such file or directory`
This error is for sure connected to this PR. I have bumped into it locally for python flight build but it dissapeared after I continued making changes to the code.
```
In file included from /arrow/cpp/src/arrow/flight/serialization_internal.h:22,
from /arrow/python/pyarrow/src_arrow/flight.cc:21:
/arrow/cpp/src/arrow/flight/protocol_internal.h:23:10: fatal error: arrow/flight/Flight.pb.h: No such file or directory
23 | #include "arrow/flight/Flight.pb.h" // IWYU pragma: export
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/arrow_python_flight_objlib.dir/build.make:76: CMakeFiles/arrow_python_flight_objlib.dir/flight.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:255: CMakeFiles/arrow_python_flight_objlib.dir/all] Error 2
make: *** [Makefile:136: all] Error 2
error: command '/opt/conda/envs/arrow/bin/make' failed with exit code 2
error: subprocess-exited-with-error
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
kou commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1178745550
I think that we don't need to build Flight examples for pyarrow. Can we just remove it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1178771290
On another note: the C++ tests now included in PyArrow are still failing to build when using a conda environment. But I can confirm that using `-DGTest_SOURCE=BUNDLED` works for now (similarly as we already mention to do this for Windows in a conda env, xref https://github.com/apache/arrow/pull/3984).
So it would be good to update the docs for this (and open a JIRA to remove this / to move those C++ tests into cython)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1178913499
I have created two JIRA tickets:
- [ARROW-17016 [C++][Python] Move Arrow Python C++ tests into Cython](https://issues.apache.org/jira/browse/ARROW-17016)
- [ARROW-17017 [C++][Python] Enable automate re-build of Arrow Python](https://issues.apache.org/jira/browse/ARROW-17017)
Please let me know, if I missunderstood anything and I will correct the issues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1184812423
@kou I think this PR is now ready for another round of review.
I will look at the failing AppVeyor build tomorrow.
It is a bit flacky so I will also try to re-run it and see how it goes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1183006403
Revision: d795488929b02a24f836b6e1e187240961c9eefb
Submitted crossbow builds: [ursacomputing/crossbow @ actions-14eec2a333](https://github.com/ursacomputing/crossbow/branches/all?query=actions-14eec2a333)
|Task|Status|
|----|------|
|example-python-minimal-build-fedora-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-example-python-minimal-build-fedora-conda)|
|example-python-minimal-build-ubuntu-venv|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-example-python-minimal-build-ubuntu-venv)|
|python-sdist|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-python-sdist)|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.10)|
|test-conda-python-3.7|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.7)|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.7-hdfs-3.2.1)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-0.24|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.7-pandas-0.24)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-spark-v3.1.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.7-spark-v3.1.2)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.8)|
|test-conda-python-3.8-hypothesis|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.8-hypothesis)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.8-pandas-nightly)|
|test-conda-python-3.8-spark-v3.2.0|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.8-spark-v3.2.0)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.9)|
|test-conda-python-3.9-dask-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.9-dask-latest)|
|test-conda-python-3.9-dask-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.9-dask-master)|
|test-conda-python-3.9-pandas-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.9-pandas-master)|
|test-conda-python-3.9-spark-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-test-conda-python-3.9-spark-master)|
|test-debian-11-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-14eec2a333-azure-test-debian-11-python-3)|
|test-fedora-35-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-14eec2a333-azure-test-fedora-35-python-3)|
|test-ubuntu-20.04-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-14eec2a333-azure-test-ubuntu-20.04-python-3)|
|verify-rc-source-python-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-verify-rc-source-python-linux-almalinux-8-amd64)|
|verify-rc-source-python-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-verify-rc-source-python-linux-conda-latest-amd64)|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
|verify-rc-source-python-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-verify-rc-source-python-linux-ubuntu-20.04-amd64)|
|verify-rc-source-python-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-verify-rc-source-python-linux-ubuntu-22.04-amd64)|
|verify-rc-source-python-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-verify-rc-source-python-macos-amd64)|
|verify-rc-source-python-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-verify-rc-source-python-macos-arm64)|
|verify-rc-source-python-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-14eec2a333-github-verify-rc-source-python-macos-conda-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r930333095
##########
python/setup.py:
##########
@@ -227,6 +228,118 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_pyarrow_cpp(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
+ raise ValueError("--build-type (or PYARROW_BUILD_TYPE) needs to "
+ "be 'release' or 'debug'")
+
+ # The directory containing this setup.py
+ source = os.path.dirname(os.path.abspath(__file__))
+ # The directory containing this PyArrow cpp CMakeLists.txt
+ source_pyarrow_cpp = pjoin(source, "pyarrow/src")
+
+ # The directory for the module being built
+ build_cmd = self.get_finalized_command('build')
+ saved_cwd = os.getcwd()
+ build_dir = pjoin(saved_cwd, 'build', 'dist')
+ build_include = pjoin(saved_cwd, 'build', 'dist', 'include')
+ build_lib = pjoin(os.getcwd(), build_cmd.build_lib)
+
+ # The directory containing Arrow C++ build
+ arrow_build_dir = os.environ.get('ARROW_BUILD_DIR', 'build')
+ if self.inplace:
+ # a bit hacky
+ build_lib = saved_cwd
+ if not os.path.isdir(build_dir):
+ self.mkpath(build_dir)
+ if not os.path.isdir(build_lib):
+ self.mkpath(build_lib)
+ if not os.path.isdir(build_include):
+ self.mkpath(build_include)
+
+ # Change to the build directory
+ with changed_dir(build_dir):
+ # cmake args
+ cmake_options = [
+ '-DCMAKE_INSTALL_PREFIX=' +
+ str(pjoin(saved_cwd, 'build/dist')),
+ '-DCMAKE_BUILD_TYPE={0}'.format(self.build_type.lower()),
+ '-DARROW_BUILD_DIR=' + str(arrow_build_dir),
+ '-DPYTHON_EXECUTABLE=%s' % sys.executable,
+ '-DPython3_EXECUTABLE=%s' % sys.executable,
+ ]
+
+ # Check for specific options
+ def append_cmake_bool(value, varname):
+ cmake_options.append('-D{0}={1}'.format(
+ varname, 'on' if value else 'off'))
+
+ append_cmake_bool(self.with_dataset, 'PYARROW_WITH_DATASET')
+ append_cmake_bool(self.with_parquet_encryption,
+ 'PYARROW_WITH_PARQUET_ENCRYPTION')
+ append_cmake_bool(self.with_hdfs,
+ 'PYARROW_WITH_HDFS')
+
+ # Windows
+ if self.cmake_generator:
+ cmake_options += ['-G', self.cmake_generator]
+
+ # build args
+ build_tool_args = []
+ if os.environ.get('PYARROW_PARALLEL'):
+ build_tool_args.append('--')
+ build_tool_args.append(
+ '-j{0}'.format(os.environ['PYARROW_PARALLEL']))
+
+ # run cmake
+ print("-- Running cmake for pyarrow cpp")
+ self.spawn(['cmake'] + cmake_options + [source_pyarrow_cpp])
+ print("-- Finished cmake for pyarrow cpp")
+ # run make & install
+ print("-- Running cmake build and install for pyarrow cpp")
+ self.spawn(['cmake', '--build', '.', '--target', 'install'] +
+ build_tool_args)
+ print("-- Finished cmake build and install for pyarrow cpp")
+
+ # Move the libraries to the place expected by the Python build
+ try:
+ os.makedirs(pjoin(build_lib, 'pyarrow'))
+ except OSError:
+ pass
+
+ # helper function
+ def copy_libs(folder_name):
+ for libname in os.listdir(pjoin(build_dir, folder_name)):
+ if "python" in libname:
+ libname_path = pjoin(build_lib, "pyarrow", libname)
+ if os.path.exists(libname_path):
+ os.remove(libname_path)
+ print(
+ f"Copying {pjoin(build_dir, folder_name, libname)}"
+ f" to {pjoin(build_lib, 'pyarrow', libname)}")
+ shutil.copy(pjoin(build_dir, folder_name, libname),
+ pjoin(build_lib, "pyarrow"))
+
+ # Move libraries to python/pyarrow
+ # For windows builds, move dll from bin
+ for folder in ['lib', 'lib64', 'bin']:
+ try:
+ copy_libs(folder)
+ except OSError:
Review Comment:
Oh, sorry, I missed this comment. Yes, I will use unpacking 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1196162961
Revision: 7e51766c433c5f3d97b55bfc37af640d61c0bdd9
Submitted crossbow builds: [ursacomputing/crossbow @ actions-38d86c5355](https://github.com/ursacomputing/crossbow/branches/all?query=actions-38d86c5355)
|Task|Status|
|----|------|
|example-python-minimal-build-fedora-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-example-python-minimal-build-fedora-conda)|
|example-python-minimal-build-ubuntu-venv|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-example-python-minimal-build-ubuntu-venv)|
|python-sdist|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-python-sdist)|
|test-conda-python-3.10|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.10)|
|test-conda-python-3.7|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.7)|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.7-hdfs-3.2.1)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-0.24|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.7-pandas-0.24)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-spark-v3.1.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.7-spark-v3.1.2)|
|test-conda-python-3.8|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.8)|
|test-conda-python-3.8-hypothesis|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.8-hypothesis)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-pandas-nightly|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.8-pandas-nightly)|
|test-conda-python-3.8-spark-v3.2.0|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.8-spark-v3.2.0)|
|test-conda-python-3.9|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.9)|
|test-conda-python-3.9-dask-latest|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.9-dask-latest)|
|test-conda-python-3.9-dask-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.9-dask-master)|
|test-conda-python-3.9-pandas-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.9-pandas-master)|
|test-conda-python-3.9-spark-master|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-test-conda-python-3.9-spark-master)|
|test-debian-11-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-38d86c5355-azure-test-debian-11-python-3)|
|test-fedora-35-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-38d86c5355-azure-test-fedora-35-python-3)|
|test-ubuntu-20.04-python-3|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-38d86c5355-azure-test-ubuntu-20.04-python-3)|
|verify-rc-source-python-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-verify-rc-source-python-linux-almalinux-8-amd64)|
|verify-rc-source-python-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-verify-rc-source-python-linux-conda-latest-amd64)|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
|verify-rc-source-python-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-verify-rc-source-python-linux-ubuntu-20.04-amd64)|
|verify-rc-source-python-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-verify-rc-source-python-linux-ubuntu-22.04-amd64)|
|verify-rc-source-python-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-verify-rc-source-python-macos-amd64)|
|verify-rc-source-python-macos-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-verify-rc-source-python-macos-arm64)|
|verify-rc-source-python-macos-conda-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-38d86c5355-github-verify-rc-source-python-macos-conda-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1196162661
Revision: 7e51766c433c5f3d97b55bfc37af640d61c0bdd9
Submitted crossbow builds: [ursacomputing/crossbow @ actions-941f2855ab](https://github.com/ursacomputing/crossbow/branches/all?query=actions-941f2855ab)
|Task|Status|
|----|------|
|verify-rc-binaries-wheels-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-verify-rc-binaries-wheels-linux-almalinux-8-amd64)|
|verify-rc-binaries-wheels-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-verify-rc-binaries-wheels-linux-conda-latest-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64)|
|verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64)|
|verify-rc-binaries-wheels-macos-10.15-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-verify-rc-binaries-wheels-macos-10.15-amd64)|
|verify-rc-binaries-wheels-macos-10.15-amd64-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-verify-rc-binaries-wheels-macos-10.15-amd64-conda)|
|verify-rc-binaries-wheels-macos-11-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-verify-rc-binaries-wheels-macos-11-amd64)|
|verify-rc-binaries-wheels-macos-11-amd64-conda|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-verify-rc-binaries-wheels-macos-11-amd64-conda)|
|verify-rc-binaries-wheels-macos-11-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-verify-rc-binaries-wheels-macos-11-arm64)|
|verify-rc-binaries-wheels-windows|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-verify-rc-binaries-wheels-windows)|
|wheel-macos-big-sur-cp310-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-macos-big-sur-cp310-arm64)|
|wheel-macos-big-sur-cp310-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-macos-big-sur-cp310-universal2)|
|wheel-macos-big-sur-cp38-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-macos-big-sur-cp38-arm64)|
|wheel-macos-big-sur-cp39-arm64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-macos-big-sur-cp39-arm64)|
|wheel-macos-big-sur-cp39-universal2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-macos-big-sur-cp39-universal2)|
|wheel-macos-high-sierra-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-macos-high-sierra-cp310-amd64)|
|wheel-macos-high-sierra-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-macos-high-sierra-cp37-amd64)|
|wheel-macos-high-sierra-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-macos-high-sierra-cp38-amd64)|
|wheel-macos-high-sierra-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-macos-high-sierra-cp39-amd64)|
|wheel-macos-mavericks-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-macos-mavericks-cp310-amd64)|
|wheel-macos-mavericks-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-macos-mavericks-cp37-amd64)|
|wheel-macos-mavericks-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-macos-mavericks-cp38-amd64)|
|wheel-macos-mavericks-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-macos-mavericks-cp39-amd64)|
|wheel-manylinux2014-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-manylinux2014-cp310-amd64)|
|wheel-manylinux2014-cp310-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-manylinux2014-cp37-amd64)|
|wheel-manylinux2014-cp37-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-manylinux2014-cp38-amd64)|
|wheel-manylinux2014-cp38-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-manylinux2014-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-manylinux2014-cp39-amd64)|
|wheel-manylinux2014-cp39-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|wheel-windows-cp310-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-windows-cp310-amd64)|
|wheel-windows-cp37-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-windows-cp37-amd64)|
|wheel-windows-cp38-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-windows-cp38-amd64)|
|wheel-windows-cp39-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-941f2855ab-github-wheel-windows-cp39-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r922973357
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs C PyArrow options
+#
+
+# Check all the options from Arrow and C PyArrow to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+include(SetupCxxFlags)
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+# Changing ARROW_SOURCE_DIR for sdist build
+# In this case cpp/cmake_modules doesn't exist
+if(NOT EXISTS "${ARROW_SOURCE_DIR}/cmake_modules/Find${MODULE}.cmake")
+ set(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR})
+endif()
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS}
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+ # Must link to shared libarrow_flight: we don't want to link more than one
+ # copy of gRPC into the eventual Cython shared object, otherwise gRPC calls
+ # fail with weird errors due to multiple copies of global static state (The
+ # other solution is to link gRPC shared everywhere instead of statically only
+ # in Flight)
+ find_package(ArrowFlight REQUIRED)
+ include_directories("${ARROW_CPP_SOURCE_DIR}/src" "${ARROW_CPP_SOURCE_DIR}/}/${ARROW_BUILD_DIR}/src")
Review Comment:
I think this is a leftover from the more complicated python flight built in https://github.com/apache/arrow/pull/13311/commits/5234ba4d47c2fba64d194c35f527c6648093c4dd (due to `flight_grpc_gen`).
Removed it locally and it works without it 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r922968233
##########
python/pyarrow/src_arrow/CMakeLists.txt:
##########
@@ -0,0 +1,447 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# ARROW_SOURCE_DIR: location of arrow/cpp
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE}/cpp")
+set(ARROW_SOURCE_DIR "${ARROW_CPP_SOURCE_DIR}")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
Review Comment:
See: https://github.com/apache/arrow/pull/13311#discussion_r922967854
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1188807581
Revision: cc7e8c850b6aa600c80e0e3eebffab9f39655452
Submitted crossbow builds: [ursacomputing/crossbow @ actions-2b4d23dc1c](https://github.com/ursacomputing/crossbow/branches/all?query=actions-2b4d23dc1c)
|Task|Status|
|----|------|
|verify-rc-source-python-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-2b4d23dc1c-github-verify-rc-source-python-linux-ubuntu-18.04-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
raulcd commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1189984391
@github-actions crossbow submit *wheel*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r925346281
##########
python/pyarrow/src/common.h:
##########
@@ -21,8 +21,8 @@
#include <utility>
#include "arrow/buffer.h"
-#include "arrow/python/pyarrow.h"
-#include "arrow/python/visibility.h"
+#include "pyarrow.h"
+#include "visibility.h"
Review Comment:
Can we reorder these includes: first Arrow C++ includes, then PyArrow C++ includes?
##########
cpp/cmake_modules/FindArrowPython.cmake:
##########
@@ -46,8 +46,9 @@ endif()
find_package(Arrow ${find_package_arguments})
if(ARROW_FOUND)
+ message(STATUS "PYARROW_CPP_HOME: ${PYARROW_CPP_HOME}")
Review Comment:
Is it customary to add such status messages in `Find*.cmake` files? @kou
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs PyArrow cpp options
+#
+
+# Check all the options from Arrow and PyArrow cpp to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
Review Comment:
```suggestion
# Needed compiler flags
```
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs PyArrow cpp options
+#
+
+# Check all the options from Arrow and PyArrow cpp to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+include(SetupCxxFlags)
+
+#
+# shred/static link libs
Review Comment:
```suggestion
# Shared/static link libs
```
##########
docs/source/developers/python.rst:
##########
@@ -131,6 +131,29 @@ for ``.py`` files or
for ``.pyx`` and ``.pxi`` files. In this case you will also need to
install the `pytest-cython <https://github.com/lgpage/pytest-cython>`_ plugin.
+Testing Arrow Python (C++ code)
Review Comment:
+1
##########
ci/scripts/python_test.sh:
##########
@@ -54,4 +55,14 @@ export PYARROW_TEST_ORC
export PYARROW_TEST_PARQUET
export PYARROW_TEST_S3
+# Testing Arrow Python
Review Comment:
+1
##########
python/pyarrow/src/api.h:
##########
@@ -17,14 +17,14 @@
#pragma once
-#include "arrow/python/arrow_to_pandas.h"
-#include "arrow/python/common.h"
-#include "arrow/python/datetime.h"
-#include "arrow/python/deserialize.h"
-#include "arrow/python/helpers.h"
-#include "arrow/python/inference.h"
-#include "arrow/python/io.h"
-#include "arrow/python/numpy_convert.h"
-#include "arrow/python/numpy_to_arrow.h"
-#include "arrow/python/python_to_arrow.h"
-#include "arrow/python/serialize.h"
+#include "arrow_to_pandas.h"
Review Comment:
Hmm... we can keep it like this for now, but I think we'll have to decide the fate of this file.
If this is an API to be used by third-party projects, then I think there should still be a base directory, for example:
```suggestion
#include "pyarrow/arrow_to_pandas.h"
```
If we decide that PyArrow does not provide any C++ API at all, then this file can simply be deleted.
Perhaps open a separate JIRA for this?
##########
python/pyarrow/src/datetime.cc:
##########
@@ -14,17 +14,17 @@
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
-#include "arrow/python/datetime.h"
+#include "datetime.h"
#include <algorithm>
#include <chrono>
#include <iomanip>
#include "arrow/array.h"
-#include "arrow/python/arrow_to_python_internal.h"
-#include "arrow/python/common.h"
-#include "arrow/python/helpers.h"
-#include "arrow/python/platform.h"
+#include "arrow_to_python_internal.h"
+#include "common.h"
+#include "helpers.h"
+#include "platform.h"
Review Comment:
Reorder includes?
##########
python/pyarrow/tests/test_gdb.py:
##########
@@ -154,7 +154,7 @@ def select_frame(self, func_name):
# but it's not available on old GDB versions (such as 8.1.1),
# so instead parse the stack trace for a matching frame number.
out = self.run_command("info stack")
- pat = r"(?mi)^#(\d+)\s+.* in " + re.escape(func_name) + " "
+ pat = r"(?mi)^#(\d+)\s+.* in " + re.escape(func_name)
Review Comment:
Can we then make the regex a bit less lenient to avoid matching prefixes of `func_name`, e.g.:
```suggestion
pat = r"(?mi)^#(\d+)\s+.* in " + re.escape(func_name) + r"\b"
```
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs PyArrow cpp options
+#
+
+# Check all the options from Arrow and PyArrow cpp to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+include(SetupCxxFlags)
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
Review Comment:
Hmm... wouldn't `parquet_shared` be required even if Parquet encryption isn't enabled?
##########
python/pyarrow/src/csv.h:
##########
@@ -23,7 +23,7 @@
#include <vector>
#include "arrow/csv/options.h"
-#include "arrow/python/common.h"
+#include "common.h"
Review Comment:
Reorder includes?
##########
python/setup.py:
##########
@@ -227,6 +228,118 @@ def initialize_options(self):
'_hdfsio',
'gandiva']
+ def _run_cmake_pyarrow_cpp(self):
+ # check if build_type is correctly passed / set
+ if self.build_type.lower() not in ('release', 'debug'):
+ raise ValueError("--build-type (or PYARROW_BUILD_TYPE) needs to "
+ "be 'release' or 'debug'")
+
+ # The directory containing this setup.py
+ source = os.path.dirname(os.path.abspath(__file__))
+ # The directory containing this PyArrow cpp CMakeLists.txt
+ source_pyarrow_cpp = pjoin(source, "pyarrow/src")
+
+ # The directory for the module being built
+ build_cmd = self.get_finalized_command('build')
+ saved_cwd = os.getcwd()
+ build_dir = pjoin(saved_cwd, 'build', 'dist')
+ build_include = pjoin(saved_cwd, 'build', 'dist', 'include')
+ build_lib = pjoin(os.getcwd(), build_cmd.build_lib)
+
+ # The directory containing Arrow C++ build
+ arrow_build_dir = os.environ.get('ARROW_BUILD_DIR', 'build')
+ if self.inplace:
+ # a bit hacky
+ build_lib = saved_cwd
+ if not os.path.isdir(build_dir):
+ self.mkpath(build_dir)
+ if not os.path.isdir(build_lib):
+ self.mkpath(build_lib)
+ if not os.path.isdir(build_include):
+ self.mkpath(build_include)
+
+ # Change to the build directory
+ with changed_dir(build_dir):
+ # cmake args
+ cmake_options = [
+ '-DCMAKE_INSTALL_PREFIX=' +
+ str(pjoin(saved_cwd, 'build/dist')),
+ '-DCMAKE_BUILD_TYPE={0}'.format(self.build_type.lower()),
+ '-DARROW_BUILD_DIR=' + str(arrow_build_dir),
+ '-DPYTHON_EXECUTABLE=%s' % sys.executable,
+ '-DPython3_EXECUTABLE=%s' % sys.executable,
+ ]
+
+ # Check for specific options
+ def append_cmake_bool(value, varname):
+ cmake_options.append('-D{0}={1}'.format(
+ varname, 'on' if value else 'off'))
+
+ append_cmake_bool(self.with_dataset, 'PYARROW_WITH_DATASET')
+ append_cmake_bool(self.with_parquet_encryption,
+ 'PYARROW_WITH_PARQUET_ENCRYPTION')
+ append_cmake_bool(self.with_hdfs,
+ 'PYARROW_WITH_HDFS')
+
+ # Windows
+ if self.cmake_generator:
+ cmake_options += ['-G', self.cmake_generator]
+
+ # build args
+ build_tool_args = []
+ if os.environ.get('PYARROW_PARALLEL'):
+ build_tool_args.append('--')
+ build_tool_args.append(
+ '-j{0}'.format(os.environ['PYARROW_PARALLEL']))
+
+ # run cmake
+ print("-- Running cmake for pyarrow cpp")
+ self.spawn(['cmake'] + cmake_options + [source_pyarrow_cpp])
+ print("-- Finished cmake for pyarrow cpp")
+ # run make & install
+ print("-- Running cmake build and install for pyarrow cpp")
+ self.spawn(['cmake', '--build', '.', '--target', 'install'] +
+ build_tool_args)
+ print("-- Finished cmake build and install for pyarrow cpp")
+
+ # Move the libraries to the place expected by the Python build
+ try:
+ os.makedirs(pjoin(build_lib, 'pyarrow'))
+ except OSError:
+ pass
+
+ # helper function
+ def copy_libs(folder_name):
+ for libname in os.listdir(pjoin(build_dir, folder_name)):
+ if "python" in libname:
+ libname_path = pjoin(build_lib, "pyarrow", libname)
+ if os.path.exists(libname_path):
+ os.remove(libname_path)
+ print(
+ f"Copying {pjoin(build_dir, folder_name, libname)}"
+ f" to {pjoin(build_lib, 'pyarrow', libname)}")
+ shutil.copy(pjoin(build_dir, folder_name, libname),
+ pjoin(build_lib, "pyarrow"))
+
+ # Move libraries to python/pyarrow
+ # For windows builds, move dll from bin
+ for folder in ['lib', 'lib64', 'bin']:
+ try:
+ copy_libs(folder)
+ except OSError:
Review Comment:
Which do we simply ignore errors here? What kind of error would that be?
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
+
+if(NOT DEFINED CMAKE_BUILD_TYPE)
+ set(CMAKE_BUILD_TYPE Release)
+endif()
+
+#
+# Arrow
+#
+
+find_package(Arrow REQUIRED)
+include(ArrowOptions)
+
+#
+# Python
+#
+# Use the first Python installation on PATH, not the newest one
+set(Python3_FIND_STRATEGY "LOCATION")
+# On Windows, use registry last, not first
+set(Python3_FIND_REGISTRY "LAST")
+# On macOS, use framework last, not first
+set(Python3_FIND_FRAMEWORK "LAST")
+
+find_package(Python3Alt 3.7 REQUIRED)
+include_directories(SYSTEM ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS} src)
+
+add_custom_target(arrow_python-all)
+add_custom_target(arrow_python)
+add_custom_target(arrow_python-tests)
+add_dependencies(arrow_python-all arrow_python arrow_python-tests)
+
+set(ARROW_PYTHON_SRCS
+ arrow_to_pandas.cc
+ benchmark.cc
+ common.cc
+ datetime.cc
+ decimal.cc
+ deserialize.cc
+ extension_type.cc
+ gdb.cc
+ helpers.cc
+ inference.cc
+ init.cc
+ io.cc
+ ipc.cc
+ numpy_convert.cc
+ numpy_to_arrow.cc
+ python_to_arrow.cc
+ pyarrow.cc
+ serialize.cc
+ udf.cc)
+
+set_source_files_properties(init.cc PROPERTIES SKIP_PRECOMPILE_HEADERS ON
+ SKIP_UNITY_BUILD_INCLUSION ON)
+
+#
+# Arrow vs PyArrow cpp options
+#
+
+# Check all the options from Arrow and PyArrow cpp to be in line
+if(PYARROW_WITH_DATASET)
+ find_package(ArrowDataset REQUIRED)
+endif()
+
+if(PYARROW_WITH_PARQUET_ENCRYPTION)
+ if(PARQUET_REQUIRE_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SRCS parquet_encryption.cc)
+ find_package(Parquet REQUIRED)
+ else()
+ message(FATAL_ERROR "You must build Arrow C++ with PARQUET_REQUIRE_ENCRYPTION=ON")
+ endif()
+endif()
+
+if(PYARROW_WITH_HDFS)
+ if(NOT ARROW_HDFS)
+ message(FATAL_ERROR "You must build Arrow C++ with ARROW_HDFS=ON")
+ endif()
+endif()
+
+# Check for only Arrow C++ options
+if(ARROW_CSV)
+ list(APPEND ARROW_PYTHON_SRCS csv.cc)
+endif()
+
+if(ARROW_FILESYSTEM)
+ list(APPEND ARROW_PYTHON_SRCS filesystem.cc)
+endif()
+
+# Link to arrow dependecies
+if(ARROW_BUILD_SHARED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_shared)
+else()
+ set(THREADS_PREFER_PTHREAD_FLAG ON)
+ find_package(Threads REQUIRED)
+ set(ARROW_PYTHON_DEPENDENCIES arrow_static Threads::Threads)
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ set_property(SOURCE pyarrow.cc
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS " -Wno-cast-qual ")
+endif()
+
+#
+# Compiler stuff
+#
+
+include(GNUInstallDirs)
+
+# This ensures that things like gnu++11 get passed correctly
+if(NOT DEFINED CMAKE_CXX_STANDARD)
+ set(CMAKE_CXX_STANDARD 11)
+endif()
+
+# We require a C++11 compliant compiler
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+
+# Needed gdb flags
+include(SetupCxxFlags)
+
+#
+# shred/static link libs
+#
+
+set(ARROW_PYTHON_SHARED_LINK_LIBS arrow_shared)
+set(ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS)
+set(ARROW_PYTHON_STATIC_LINK_LIBS ${PYTHON_OTHER_LIBS})
+
+if(WIN32)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS ${PYTHON_LIBRARIES} ${PYTHON_OTHER_LIBS})
+endif()
+
+if(PARQUET_REQUIRE_ENCRYPTION AND PYARROW_WITH_PARQUET_ENCRYPTION)
+ list(APPEND ARROW_PYTHON_SHARED_LINK_LIBS parquet_shared)
+endif()
+
+set(ARROW_PYTHON_INCLUDES ${NUMPY_INCLUDE_DIRS} ${PYTHON_INCLUDE_DIRS})
+
+# Inlude macros needed to find and use add_arrow_lib function
+include(BuildUtils)
+include(CMakePackageConfigHelpers)
+
+# Set the output directory for cmake module
+# (CMAKE_INSTALL_PREFIX = python/build/dist! should be set in setup.py!)
+set(ARROW_CMAKE_DIR "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}")
+
+# Changing ARROW_SOURCE_DIR for sdist build
+# In this case cpp/cmake_modules doesn't exist
+if(NOT EXISTS "${ARROW_SOURCE_DIR}/cpp/cmake_modules/Find${MODULE}.cmake")
+ set(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR})
+endif()
+
+add_arrow_lib(arrow_python
+ CMAKE_PACKAGE_NAME
+ ArrowPython
+ PKG_CONFIG_NAME
+ arrow-python
+ SOURCES
+ ${ARROW_PYTHON_SRCS}
+ PRECOMPILED_HEADERS
+ "$<$<COMPILE_LANGUAGE:CXX>:pch.h>"
+ OUTPUTS
+ ARROW_PYTHON_LIBRARIES
+ DEPENDENCIES
+ ${ARROW_PYTHON_DEPENDENCIES}
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS}
+ SHARED_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_LINK_LIBS}
+ SHARED_PRIVATE_LINK_LIBS
+ ${ARROW_PYTHON_SHARED_PRIVATE_LINK_LIBS}
+ STATIC_LINK_LIBS
+ ${ARROW_PYTHON_STATIC_LINK_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}")
+
+add_dependencies(arrow_python ${ARROW_PYTHON_LIBRARIES})
+
+foreach(LIB_TARGET ${ARROW_PYTHON_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYTHON_EXPORTING)
+endforeach()
+
+if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_static PUBLIC ARROW_STATIC)
+endif()
+
+if(ARROW_FLIGHT AND ARROW_BUILD_SHARED)
+ # Must link to shared libarrow_flight: we don't want to link more than one
+ # copy of gRPC into the eventual Cython shared object, otherwise gRPC calls
+ # fail with weird errors due to multiple copies of global static state (The
+ # other solution is to link gRPC shared everywhere instead of statically only
+ # in Flight)
+ find_package(ArrowFlight REQUIRED)
+ include_directories("${ARROW_CPP_SOURCE_DIR}/src" "${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/src")
+
+ set(FLIGHT_LINK_LIBS arrow_flight_shared)
+
+ add_arrow_lib(arrow_python_flight
+ CMAKE_PACKAGE_NAME
+ ArrowPythonFlight
+ PKG_CONFIG_NAME
+ arrow-python-flight
+ SOURCES
+ flight.cc
+ OUTPUTS
+ ARROW_PYFLIGHT_LIBRARIES
+ SHARED_LINK_FLAGS
+ ${ARROW_VERSION_SCRIPT_FLAGS}
+ SHARED_LINK_LIBS
+ arrow_python_shared
+ arrow_flight_shared
+ STATIC_LINK_LIBS
+ ${PYTHON_OTHER_LIBS}
+ EXTRA_INCLUDES
+ "${ARROW_PYTHON_INCLUDES}"
+ PRIVATE_INCLUDES
+ "${Protobuf_INCLUDE_DIRS}")
+
+ add_dependencies(arrow_python ${ARROW_PYFLIGHT_LIBRARIES})
+
+ foreach(LIB_TARGET ${ARROW_PYFLIGHT_LIBRARIES})
+ target_compile_definitions(${LIB_TARGET} PRIVATE ARROW_PYFLIGHT_EXPORTING)
+ endforeach()
+
+ if(ARROW_BUILD_STATIC AND MSVC)
+ target_compile_definitions(arrow_python_flight_static PUBLIC ARROW_STATIC)
+ endif()
+endif()
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
+ # Clang, be quiet. Python C API has lots of macros
+ set_property(SOURCE ${ARROW_PYTHON_SRCS}
+ APPEND_STRING
+ PROPERTY COMPILE_FLAGS -Wno-parentheses-equality)
+endif()
+
+arrow_install_all_headers("arrow/python")
+
+# ----------------------------------------------------------------------
+
+#
+# Tests
+# The tests will be moved to Cython and are currently supported for bundled GTest
+# Follow-up: https://issues.apache.org/jira/browse/ARROW-17016
+#
+
+if(ARROW_BUILD_TESTS)
+
+ enable_testing()
+ set(GTEST_ROOT ${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/googletest_ep-prefix)
+
+ # GTest must be built from source
+ if(EXISTS ${ARROW_CPP_SOURCE_DIR}/${ARROW_BUILD_DIR}/googletest_ep-prefix)
+
+ # Set necessary paths for cmake to find GTest
+ set(GTEST_INCLUDE_DIR "${GTEST_ROOT}/include")
+ set(GTEST_LIBRARY ${GTEST_ROOT}/lib)
+ set(GTEST_MAIN_LIBRARY ${GTEST_ROOT}/lib)
+
+ #
+ # Taken from Matlab CMakeLists.txt (enable_gtest and build_gtest)
+ #
+
+ set(ARROW_GTEST_PREFIX "${GTEST_ROOT}")
+ set(ARROW_GTEST_MAIN_PREFIX "${GTEST_ROOT}")
+
+ if(WIN32)
+ set(ARROW_GTEST_SHARED_LIB_DIR "${ARROW_GTEST_PREFIX}/bin")
+ set(ARROW_GTEST_MAIN_SHARED_LIB_DIR "${ARROW_GTEST_MAIN_PREFIX}/bin")
+
+ set(ARROW_GTEST_LINK_LIB_DIR "${ARROW_GTEST_PREFIX}/lib")
+ set(ARROW_GTEST_LINK_LIB
+ "${ARROW_GTEST_LINK_LIB_DIR}/${CMAKE_IMPORT_LIBRARY_PREFIX}gtestd${CMAKE_IMPORT_LIBRARY_SUFFIX}"
+ )
+
+ set(ARROW_GTEST_MAIN_LINK_LIB_DIR "${ARROW_GTEST_MAIN_PREFIX}/lib")
+ set(ARROW_GTEST_MAIN_LINK_LIB
+ "${ARROW_GTEST_MAIN_LINK_LIB_DIR}/${CMAKE_IMPORT_LIBRARY_PREFIX}gtest_maind${CMAKE_IMPORT_LIBRARY_SUFFIX}"
+ )
+ else()
+ set(ARROW_GTEST_SHARED_LIB_DIR "${ARROW_GTEST_PREFIX}/lib")
+ set(ARROW_GTEST_MAIN_SHARED_LIB_DIR "${ARROW_GTEST_MAIN_PREFIX}/lib")
+ endif()
+
+ set(ARROW_GTEST_INCLUDE_DIR "${ARROW_GTEST_PREFIX}/include")
+ set(ARROW_GTEST_SHARED_LIB
+ "${ARROW_GTEST_SHARED_LIB_DIR}/${CMAKE_SHARED_LIBRARY_PREFIX}gtestd${CMAKE_SHARED_LIBRARY_SUFFIX}"
+ )
+
+ set(ARROW_GTEST_MAIN_INCLUDE_DIR "${ARROW_GTEST_MAIN_PREFIX}/include")
+ set(ARROW_GTEST_MAIN_SHARED_LIB
+ "${ARROW_GTEST_MAIN_SHARED_LIB_DIR}/${CMAKE_SHARED_LIBRARY_PREFIX}gtest_maind${CMAKE_SHARED_LIBRARY_SUFFIX}"
+ )
+
+ file(MAKE_DIRECTORY "${ARROW_GTEST_INCLUDE_DIR}")
+
+ # Create target GTest::gtest
+ add_library(GTest::gtest SHARED IMPORTED)
+ set_target_properties(GTest::gtest
+ PROPERTIES IMPORTED_LOCATION ${ARROW_GTEST_SHARED_LIB}
+ INTERFACE_INCLUDE_DIRECTORIES
+ ${ARROW_GTEST_INCLUDE_DIR})
+ if(WIN32)
+ set_target_properties(GTest::gtest PROPERTIES IMPORTED_IMPLIB ${ARROW_GTEST_LINK_LIB})
+ endif()
+
+ # ArrowTesting
+ # needed to be able to use arrow_testing_shared target
+ find_package(ArrowTesting REQUIRED)
+
+ add_custom_target(all-tests)
+
+ add_library(arrow_python_test_main STATIC util/test_main.cc)
+
+ target_link_libraries(arrow_python_test_main GTest::gtest)
+ target_include_directories(arrow_python_test_main SYSTEM
+ PUBLIC ${ARROW_PYTHON_INCLUDES})
+
+ # Link libraries to avoid include error on Linux
+ if(ARROW_TEST_LINKAGE STREQUAL shared)
+ target_link_libraries(arrow_python_test_main arrow_shared)
+ else()
+ target_link_libraries(arrow_python_test_main arrow_static)
+ endif()
+
+ if(APPLE)
+ target_link_libraries(arrow_python_test_main ${CMAKE_DL_LIBS})
+ set_target_properties(arrow_python_test_main PROPERTIES LINK_FLAGS
+ "-undefined dynamic_lookup")
+ elseif(NOT MSVC)
+ target_link_libraries(arrow_python_test_main pthread ${CMAKE_DL_LIBS})
+ endif()
+
+ if(ARROW_TEST_LINKAGE STREQUAL shared)
+ set(ARROW_PYTHON_TEST_LINK_LIBS arrow_python_test_main arrow_python_shared
+ arrow_testing_shared arrow_shared)
+ else()
+ set(ARROW_PYTHON_TEST_LINK_LIBS arrow_python_test_main arrow_python_static
+ arrow_testing_static arrow_static)
+ endif()
+
+ #
+ # Add a test case
+ #
+
+ set(REL_TEST_NAME "python_test")
+ get_filename_component(TEST_NAME ${REL_TEST_NAME} NAME_WE)
+ set(TEST_NAME "arrow-${TEST_NAME}")
+ set(SOURCES "${REL_TEST_NAME}.cc")
+
+ # # Make sure the executable name contains only hyphens, not underscores
+ string(REPLACE "_" "-" TEST_NAME ${TEST_NAME})
+
+ set(TEST_PATH "${CMAKE_BINARY_DIR}/${TEST_NAME}")
+ add_executable(${TEST_NAME} ${SOURCES})
+
+ # We need to set the correct RPATH so that dependencies
+ set_target_properties(${TEST_NAME}
+ PROPERTIES BUILD_WITH_INSTALL_RPATH TRUE
+ INSTALL_RPATH_USE_LINK_PATH TRUE
+ INSTALL_RPATH
+ "${PYTHON_SOURCE_DIR}/pyarrow;$ENV{CONDA_PREFIX}/lib")
+
+ # Customize link libraries
+ target_link_libraries(${TEST_NAME} PRIVATE "${ARROW_PYTHON_TEST_LINK_LIBS}")
+ # Extra link libs
+ target_link_libraries(${TEST_NAME} PRIVATE ${PYTHON_LIBRARIES})
+ # Extra includes
+ target_include_directories(${TEST_NAME} SYSTEM PUBLIC "${ARROW_PYTHON_INCLUDES}")
+
+ # Add the test
+ if(ARROW_TEST_MEMCHECK)
Review Comment:
I don't think we want to keep this specific part.
##########
ci/scripts/python_test.sh:
##########
@@ -20,6 +20,7 @@
set -ex
arrow_dir=${1}
+test_dir=${1}/python/build/dist/temp
Review Comment:
Where is this "temp" directory coming from? Can we name it something else (e.g. "cpp_tests")?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r925384420
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
Review Comment:
I think it is suggested to also change the name of the final library: https://github.com/apache/arrow/pull/13311#discussion_r925231102
which is fine by me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r925388601
##########
python/pyarrow/src/CMakeLists.txt:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#
+# arrow_python
+#
+
+cmake_minimum_required(VERSION 3.5)
+
+# RPATH settings on macOS do not affect install_name.
+# https://cmake.org/cmake/help/latest/policy/CMP0068.html
+if(POLICY CMP0068)
+ cmake_policy(SET CMP0068 NEW)
+endif()
+
+#
+# Define
+# CMAKE_MODULE_PATH: location of cmake_modules in python
+#
+
+get_filename_component(PYARROW_SOURCE_DIR ${CMAKE_SOURCE_DIR} DIRECTORY)
+get_filename_component(PYTHON_SOURCE_DIR ${PYARROW_SOURCE_DIR} DIRECTORY)
+get_filename_component(ARROW_SOURCE_DIR ${PYTHON_SOURCE_DIR} DIRECTORY)
+set(ARROW_CPP_SOURCE_DIR "${ARROW_SOURCE_DIR}/cpp")
+
+# normalize ARROW_HOME path
+file(TO_CMAKE_PATH "$ENV{ARROW_HOME}" ARROW_HOME)
+set(CMAKE_MODULE_PATH "${PYTHON_SOURCE_DIR}/cmake_modules" "${ARROW_HOME}/lib/cmake/arrow")
+
+#
+# Arrow version
+#
+
+set(ARROW_PYTHON_VERSION "9.0.0-SNAPSHOT")
+string(REGEX MATCH "^[0-9]+\\.[0-9]+\\.[0-9]+" ARROW_PYTHON_BASE_VERSION "${ARROW_PYTHON_VERSION}")
+# Need to set to ARRROW_VERSION before finding Arrow package!
+project(arrow_python VERSION "${ARROW_PYTHON_BASE_VERSION}")
Review Comment:
Is that a safe change? (I don't know to what extent external packages depend on the exact library name?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
AlenkaF commented on code in PR #13311:
URL: https://github.com/apache/arrow/pull/13311#discussion_r926226323
##########
cpp/cmake_modules/FindArrowPython.cmake:
##########
@@ -46,8 +46,9 @@ endif()
find_package(Arrow ${find_package_arguments})
if(ARROW_FOUND)
+ message(STATUS "PYARROW_CPP_HOME: ${PYARROW_CPP_HOME}")
Review Comment:
It was mostly added for "debugging" purposes and am happy to remove it =)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13311: ARROW-16340: [Python] Move all Python related code into PyArrow
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13311:
URL: https://github.com/apache/arrow/pull/13311#issuecomment-1225260926
Revision: b2780355cfbe34677815fcd954c0c2b206915319
Submitted crossbow builds: [ursacomputing/crossbow @ actions-e15d161feb](https://github.com/ursacomputing/crossbow/branches/all?query=actions-e15d161feb)
|Task|Status|
|----|------|
|almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-e15d161feb-github-almalinux-8-amd64)|
|almalinux-8-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|almalinux-9-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-e15d161feb-github-almalinux-9-amd64)|
|almalinux-9-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|centos-7-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-e15d161feb-github-centos-7-amd64)|
|centos-8-stream-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-e15d161feb-github-centos-8-stream-amd64)|
|centos-8-stream-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|centos-9-stream-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-e15d161feb-github-centos-9-stream-amd64)|
|centos-9-stream-arm64|[](https://app.travis-ci.com/github/ursacomputing/crossbow/branches)|
|test-r-rstudio-r-base-4.2-centos7-devtoolset-8|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-e15d161feb-azure-test-r-rstudio-r-base-4.2-centos7-devtoolset-8)|
|verify-rc-binaries-jars-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-e15d161feb-github-verify-rc-binaries-jars-linux-almalinux-8-amd64)|
|verify-rc-binaries-wheels-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-e15d161feb-github-verify-rc-binaries-wheels-linux-almalinux-8-amd64)|
|verify-rc-source-cpp-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-e15d161feb-github-verify-rc-source-cpp-linux-almalinux-8-amd64)|
|verify-rc-source-csharp-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-e15d161feb-github-verify-rc-source-csharp-linux-almalinux-8-amd64)|
|verify-rc-source-go-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-e15d161feb-github-verify-rc-source-go-linux-almalinux-8-amd64)|
|verify-rc-source-integration-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-e15d161feb-github-verify-rc-source-integration-linux-almalinux-8-amd64)|
|verify-rc-source-java-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-e15d161feb-github-verify-rc-source-java-linux-almalinux-8-amd64)|
|verify-rc-source-js-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-e15d161feb-github-verify-rc-source-js-linux-almalinux-8-amd64)|
|verify-rc-source-python-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-e15d161feb-github-verify-rc-source-python-linux-almalinux-8-amd64)|
|verify-rc-source-ruby-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-e15d161feb-github-verify-rc-source-ruby-linux-almalinux-8-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org