You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "jorisvandenbossche (via GitHub)" <gi...@apache.org> on 2023/06/30 08:37:52 UTC
[GitHub] [arrow] jorisvandenbossche opened a new issue, #36411: [Python] Improve the pyarrow build system
jorisvandenbossche opened a new issue, #36411:
URL: https://github.com/apache/arrow/issues/36411
The current build system of pyarrow consists of using setuptools with a custom `setup.py` that calls cmake to do the actual build. While this works, there are nowadays newer python "build backends" that we could benefit from to make this easier to maintain and use.
Specifically build backends that are meant to work with CMake projects, such as [`scikit-build`](https://scikit-build-core.readthedocs.io/en/latest/index.html), might be worth exploring.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] h-vetinari commented on issue #36411: [Python] Improve the pyarrow build system
Posted by "h-vetinari (via GitHub)" <gi...@apache.org>.
h-vetinari commented on issue #36411:
URL: https://github.com/apache/arrow/issues/36411#issuecomment-1740754036
I'm aware, I've been building arrow for a while now. ;-)
It was more a rhetorical statement - depends a bit on what one considers "substantial" too. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Improve the pyarrow build system [arrow]
Posted by "vyasr (via GitHub)" <gi...@apache.org>.
vyasr commented on issue #36411:
URL: https://github.com/apache/arrow/issues/36411#issuecomment-1986099419
I'll note that if `scikit-build` is still of interest, the modern solution here would be to use `https://github.com/scikit-build/scikit-build-core`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on issue #36411: [Python] Improve the pyarrow build system
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #36411:
URL: https://github.com/apache/arrow/issues/36411#issuecomment-1614413612
Thanks @jorisvandenbossche !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on issue #36411: [Python] Improve the pyarrow build system
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #36411:
URL: https://github.com/apache/arrow/issues/36411#issuecomment-1740739582
> though if you have substantial CMake infrastructure then that integrates more easily with scikit-build.
We do have that, yes: https://github.com/apache/arrow/blob/main/python/CMakeLists.txt
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] h-vetinari commented on issue #36411: [Python] Improve the pyarrow build system
Posted by "h-vetinari (via GitHub)" <gi...@apache.org>.
h-vetinari commented on issue #36411:
URL: https://github.com/apache/arrow/issues/36411#issuecomment-1740736163
> Specifically build backends that are meant to work with CMake projects, such as [`scikit-build`](https://scikit-build-core.readthedocs.io/en/latest/index.html), might be worth exploring.
Scipy has [pioneered](https://labs.quansight.org/blog/2021/07/moving-scipy-to-meson) meson as the backend (in the wider python ecosystem at least), which is now being used by numpy & pandas as well, and is being picked up by more and more projects. For a regular python project it's probably a better choice than CMake-based builds (the syntax is already nice and pythonic too), though if you have substantial CMake infrastructure then that integrates more easily with scikit-build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Improve the pyarrow build system [arrow]
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #36411:
URL: https://github.com/apache/arrow/issues/36411#issuecomment-1753704373
Major wish-list item: proper incremental / cached cython builds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python] Improve the pyarrow build system [arrow]
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #36411:
URL: https://github.com/apache/arrow/issues/36411#issuecomment-1753695898
Also relevant issue: https://github.com/apache/arrow/issues/36410
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org