You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "fboudra (via GitHub)" <gi...@apache.org> on 2026/09/17 10:24:57 UTC
[PR] GH-50398/GH-50829 6/6: CI: ship cp311-abi3 wheels with symbol audit and perf gates [arrow]
fboudra opened a new pull request, #51377:
URL: https://github.com/apache/arrow/pull/51377
### Rationale for this change
https://github.com/apache/arrow/issues/50398
https://github.com/apache/arrow/issues/50829
### What changes are included in this PR?
CI only, no Python code changes.
The wheel matrix changes from one build per Python version to one cp311-abi3 build per platform (49 wheel tasks down to 7). The build runs on Python 3.14 because limited API builds need >= 3.13 dev headers (the vendored pythoncapi_compat.h shims for older versions reference internals the limited API hides), and the wheel tag is forced with SKBUILD_WHEEL_PY_API. That env var is a rename of SKBUILD_WHEEL__PY_API, which scikit-build-core >= 1.0 silently ignores because it joins the prefix and the dataclass field with single underscores. The per-version cp312 wheel drops out of the matrix, since the cp311-abi3 wheel already covers 3.11 through 3.15.
Gates on the xlinux builds:
- The limited API symbol audit (from PR 5) runs before auditwheel, against libpython3.11 from the wheel image, with a fallback to the python3.11 interpreter binary because the 2026.09 base images ship CPython without a shared libpython. Freethreading builds are exempt.
- A performance gate: python/scripts/perf_gate.py runs 13 benchmarks after auditwheel and fails if any median is more than 2x the committed reference (python/scripts/perf_gate_reference.json). This is a cross-machine tripwire, not a comparison against upstream, so CI has no PyPI dependency. Regenerate the reference after intentional performance changes with --write.
Testing per platform: the Linux abi3 row import-checks the wheel on 3.11, 3.12, 3.13, 3.14 and 3.15-rc, runs the full unittests on the build interpreter, and keeps the OS gates (Ubuntu 22.04 on 3.11, AlmaLinux 8 on 3.12, Ubuntu 24.04 on 3.12). The Windows abi3 row gains a Python 3.11 step that imports the abi3 .pyd against the unversioned python3.dll. The macOS abi3 row tests on 3.14.
The freethreading rows are dropped: CPython does not support the limited API in free-threaded builds (3.14t hard errors on Py_LIMITED_API, gh-111506, and 3.15t limited API headers do not compile). Re-add them once CPython allows it.
Also fixes python_wheel_validate_contents.py to accept the +g<sha> local version segment that setuptools_scm adds on dirty checkouts, which broke local wheel validation runs; CI checkouts are clean and were unaffected.
Verification: local container runs of the manylinux 2.28 and musllinux 1.2 wheel tasks, green end to end (build, strip, symbol audit, auditwheel repair, perf gate), plus the musl wheel import swept on 3.11, 3.12, 3.13, 3.14 and 3.15-rc. First GHA runs of the macOS and Windows rows are pending.
### Are these changes tested?
Yes
### Are there any user-facing changes?
No
### Was AI used for this PR?
**PR code and description written by:**
- [x ] Human
- [ ] AI
**Reviewed before submission by:**
- [ x] Human
- [ ] AI
- [ ] Not reviewed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-50398/GH-50829 6/6: CI: ship cp311-abi3 wheels with symbol audit and perf gates [arrow]
Posted by "xieofxie (via GitHub)" <gi...@apache.org>.
xieofxie commented on code in PR #51377:
URL: https://github.com/apache/arrow/pull/51377#discussion_r4226107932
##########
dev/tasks/tasks.yml:
##########
@@ -138,75 +138,81 @@ tasks:
########################### Python Wheels ############################
-{% for python_version, python_tag, abi_tag in [("3.11", "cp311", "cp311"),
- ("3.12", "cp312", "cp312"),
- ("3.13", "cp313", "cp313"),
- ("3.14", "cp314", "cp314"),
- ("3.14", "cp314", "cp314t"),
- ("3.15", "cp315", "cp315"),
- ("3.15", "cp315", "cp315t")
- ] %}
+{# Single wheel build tagged cp311-abi3 via SKBUILD_WHEEL_PY_API.
+ The abi3 build must run on Python 3.14: the limited-API build (see
+ python/CMakeLists.txt) requires >= 3.13 dev headers.
+ Freethreading (cp314t/cp315t) wheels are not built: CPython does not
+ support the limited API in free-threaded builds (gh-111506; 3.14t's
Review Comment:
Could we restore the separate full-API free-threaded wheel builds? `cp311-abi3` wheels do not support free-threaded Python, so removing the `cp314t`/`cp315t` rows drops coverage rather than consolidating it. Python 3.14t’s Limited API restriction does not prevent building a version-specific full-API wheel.
To retain those builds, we would need to:
- Make `Py_LIMITED_API=0x030B0000`, the `.abi3.so` suffix, and the `cp311-abi3` wheel tagging conditional; free-threaded builds should use the full API and interpreter-specific suffixes/tags.
- Fix the full-API import failure mentioned in the comment, where assignments such as `KeyValueMetadata.get = Mapping.get` depend on mutable heap types. Disabling `Py_LIMITED_API` alone will not fix that.
- Restore the free-threaded CI rows and verify imports and tests on those interpreters.
Most Limited API replacements can remain shared between both build modes. Also, [[python/cpython#111506](https://github.com/python/cpython/issues/111506)](https://github.com/python/cpython/issues/111506) was superseded by Python 3.15’s `abi3t` implementation in [[python/cpython#146636](https://github.com/python/cpython/issues/146636)](https://github.com/python/cpython/issues/146636). Migrating to `abi3t` could be a follow-up for 3.15+, but it would not cover 3.14t or be necessary to restore full-API wheels.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org