You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "gounthar (via GitHub)" <gi...@apache.org> on 2026/03/19 14:24:47 UTC
[PR] ci: add riscv64 manylinux wheel builds to release pipeline [arrow]
gounthar opened a new pull request, #49556:
URL: https://github.com/apache/arrow/pull/49556
### Describe the enhancement requested
Add riscv64 to the PyArrow wheel build pipeline using `manylinux_2_39` and native RISE riscv64 runners.
### Changes
- **`dev/tasks/tasks.yml`**: Add `("manylinux", "riscv64", "2-39", "manylinux_2_39_riscv64")` to the Linux wheel matrix
- **`dev/tasks/python-wheels/github.linux.yml`**: Add `ubuntu-24.04-riscv` runner selection and `riscv64` ARCH mapping
### What's still needed
- [ ] Docker image for riscv64 wheel builds (`apache/arrow-dev:riscv64-python-*-wheel-manylinux-2_39-vcpkg-*`)
- [ ] vcpkg dependency verification for riscv64
- [ ] Testing via Crossbow
This draft PR enables the CI pipeline. The Docker image creation is a separate effort — happy to work on that as well, or coordinate with the team.
### Evidence
- Arrow C++ native build on BananaPi F3 (SpacemiT K1, rv64gc, GCC 14.2.0): **SUCCESS** (1h13m)
- PyArrow install from source (Parquet, CSV, JSON, Compute, Filesystem): **SUCCESS**
- `import pyarrow; print(pyarrow.__version__)` → `24.0.0a1.dev1`
### CI Runners
Native riscv64 runners are available for free via [RISE RISC-V runners](https://github.com/apps/rise-risc-v-runners). numpy, llama.cpp, and pytorch already use them.
Fixes #49555
Note: this work is part of the [RISE Project](https://riseproject.dev/) effort to improve Python ecosystem support on riscv64 platforms.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4093187200
Thanks for the answers @gounthar . Can we perhaps start by having a regular C++ CI job on RISC-V? The Python wheel CI builds do not run the C++ test suite.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "gounthar (via GitHub)" <gi...@apache.org>.
gounthar commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4108621770
Update on the C++ CI work: got the Docker-based build running on a RISE riscv64 runner (ubuntu-24.04-riscv). The build itself completes (2257 C++ files, ~4h), but the test suite crashes systematically - all 59 test binaries fail in ~2 seconds each, which looks like a SIGILL or similar architecture-level issue.
The Docker image build had a couple of hurdles:
- `install_minio.sh` crashes on unknown arches due to `set -u` + unbound array key (one-line fix: `${archs[$arch]:-}`)
- Docker 26.1.5 on the runner has a BuildKit bug; works with `DOCKER_BUILDKIT=0`
I'm going to run the tests directly on native hardware (BananaPi F3) to get the actual error output - the CI redirects test logs into files so the crash reason isn't visible. Will report back with the findings.
Fork CI run: https://github.com/gounthar/arrow/actions/runs/23411999463
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4090904410
Hi @gounthar , the big question here is what happens for ongoing maintenance. I think none of the currently active Arrow maintainers has a RISC-V box at home, and debugging on CI can be painful. But I might be overstating the risks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "gounthar (via GitHub)" <gi...@apache.org>.
gounthar commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4108628733
Correction on the above: the CI run did use the Docker-based approach (`archery docker run ubuntu-cpp`) as you suggested. The build and tests ran inside the Docker container on the RISE runner. The test crashes are from inside the container, not a bare-metal build.
I'm also going to reproduce the Docker-based build on my BananaPi F3 to get the test crash output (the CI redirects logs to files). Will use `archery docker run ubuntu-cpp` there as well to match the CI environment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "luhenry (via GitHub)" <gi...@apache.org>.
luhenry commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4222824331
@pitrou quick update on the native runners. The slowness is a common issue, and I don't expect we will merge it as-is with the current generation of hardware. **_However_**, the next-gen of hardware which is 2-3x faster is coming in ~May-June of this year. We have a smaller pool of slightly faster hardware as well, happy to give it a try on these as well in the meantime. The pool is quite small (we are adding a dozen more machines in coming days) so we won't be able to merge it until then, but worth a try.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "gounthar (via GitHub)" <gi...@apache.org>.
gounthar commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4093372110
@pitrou That makes a lot of sense — start with the foundation. I'll rework this PR to add a C++ CI job on riscv64 instead of jumping straight to wheel builds.
I've already verified that Arrow C++ builds and passes `import pyarrow` on native riscv64. I'll look at the existing C++ CI jobs (`ci/docker/ubuntu-*-cpp.dockerfile` and the corresponding workflows) and add a riscv64 variant.
Would you prefer:
1. A Docker-based build (like the existing CI), or
2. A native build on the RISE runner directly (simpler, but less isolated)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4090534186
:warning: GitHub issue #49555 **has no components**, please add labels for components.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4090523121
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
Thanks for opening a pull request!
If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
Opening GitHub issues ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
Then could you also rename the pull request title in the following format?
GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
See also:
* [Other pull requests](https://github.com/apache/arrow/pulls/)
* [Contribution Guidelines - Contributing Overview](https://arrow.apache.org/docs/developers/overview.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4108643851
> Update on the C++ CI work: got the Docker-based build running on a RISE riscv64 runner (ubuntu-24.04-riscv). The build itself completes (2257 C++ files, ~4h)
4 hours to complete a native build sounds like... a lot.
> `install_minio.sh` crashes on unknown arches due to `set -u` + unbound array key (one-line fix: `${archs[$arch]:-}`)
Please note you can disable individual components using environment variables, you can get an idea by the `cmake` invocation [here](https://github.com/apache/arrow/blob/c24bc29100a141ecc7372e1f786d6505c6337360/ci/scripts/cpp_build.sh#L184).
> I'm going to run the tests directly on native hardware (BananaPi F3) to get the actual error output - the CI redirects test logs into files so the crash reason isn't visible.
Ideally, our test harness is able to display crash tracebacks using gdb, though the logic behind that is fragile and doesn't always work reliably.
See [these instructions](https://github.com/apache/arrow/blob/c24bc29100a141ecc7372e1f786d6505c6337360/compose.yaml#L29-L54) in `compose.yaml`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "gounthar (via GitHub)" <gi...@apache.org>.
gounthar commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4091133921
@pitrou That's a fair concern. A few things that mitigate it:
1. **RISE runners are native hardware** — not QEMU emulation. Debugging and iteration is much faster (we validated the full Arrow C++ + pyarrow build on native hardware in ~1.5h total).
2. **I maintain two BananaPi F3 boards** running 24/7 as riscv64 build/test machines. Happy to help debug any riscv64-specific issues that come up.
3. **The riscv64 job would be additive** — if it breaks, it doesn't affect existing x86_64/aarch64 builds. Same as how aarch64 was initially added.
4. **Arrow C++ is already riscv64-compatible** (PR #13902, 2022). The risk of riscv64-specific breakage in the wheel pipeline is low since the code itself works — it's mainly CI plumbing.
5. **RISE is committed to maintaining riscv64 CI** for key projects. Ludovic Henry (RISE TSC Co-Chair) is actively providing runners and support — this isn't a fire-and-forget contribution.
I'm also happy to be a point of contact for riscv64 issues in Arrow. I've been doing this across 30+ Python projects this week and can respond quickly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4090534108
:warning: GitHub issue #49555 **has been automatically assigned in GitHub** to PR creator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "gounthar (via GitHub)" <gi...@apache.org>.
gounthar commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4108799918
@pitrou Good points.
The 4h includes the Docker image build (~20 min for apt packages) + C++ compilation. The RISE runner is a Scaleway EM-RV1 which isn't the fastest riscv64 hardware. On my BananaPi F3 (8 cores, SpacemiT K1), the C++ build alone takes about 1h13m. Disabling some components would help; I'll trim the feature set for the CI job.
Thanks for the pointer on disabling components via env vars and on the gdb traceback support in compose.yaml. I'm running the Docker-based build on the F3 now with archery; will check if gdb picks up the crash reason.
For the minio fix, I can submit a separate small PR for `install_minio.sh` if that's useful, or fold it into this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "gounthar (via GitHub)" <gi...@apache.org>.
gounthar commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4091169424
@raulcd Great question. The RISE runners app requires these permissions:
- **Actions**: Read and write (to register/deregister runners)
- **Metadata**: Read-only
It's a GitHub App that provides self-hosted runners — similar to how some projects use BuildJet or Actuated for ARM runners.
As for Apache org setup: I'm not aware of other Apache projects using RISE runners yet — you'd likely be the first. I can check with Ludovic Henry (@luhenry, RISE TSC Co-Chair) about the process for Apache org installation. He's been setting up runners for numpy, pytorch, llama.cpp, and pyca this week.
Alternatively, we could initially run the riscv64 job on a separate fork (like we did with [riseproject-dev/numpy](https://github.com/riseproject-dev/numpy/pull/1)) and upstream once the runner setup is confirmed. That way the PR can be reviewed independently of the infra question.
I'll ping Ludovic about Apache org support and report back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4090906592
also cc @raulcd
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "gounthar (via GitHub)" <gi...@apache.org>.
gounthar commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4091238747
@raulcd @pitrou — Ludovic Henry (@luhenry, RISE TSC Co-Chair) will respond directly on this PR about the Apache org runner setup. He hasn't set up RISE runners on Apache projects before but is very interested in making it happen.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "luhenry (via GitHub)" <gi...@apache.org>.
luhenry commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4091406637
> To be set-up as part of the Apache organization. A couple of questions about that. Do we know what are the required permissions? Has this been set-up on other Apache projects or are we the first ones? Asking as might require some ASF INFRA research.
The requested permissions are:
- At repository level:
- [Actions](https://docs.github.com/rest/overview/permissions-required-for-github-apps#repository-permissions-for-actions): read-only
- [Metadata](https://docs.github.com/en/rest/authentication/permissions-required-for-github-apps?apiVersion=2026-03-10#repository-permissions-for-metadata): read-only (mandatory as well)
- At organization level:
- [Self-hosted runners](https://docs.github.com/en/rest/authentication/permissions-required-for-github-apps?apiVersion=2026-03-10#organization-permissions-for-self-hosted-runners): read and write; necessary to add the self-hosted runners and runner group
We are **not** going to require any more credentials as we only want to be able to dynamically register self-hosted runners, and that's it. You can find all the code of the app at https://github.com/riseproject-dev/riscv-runner-app, and a more descriptive website at https://riseproject-dev.github.io/riscv-runner/
> RISE is committed to maintaining riscv64 CI for key projects
[RISE](https://riseproject.dev) is part of Linux Foundation EU, and is committed to this service. We see it as a critical piece to enable RISC-V Software more broadly, which is our entire raison d'être. We are also working with PyTorch, Llama.cpp, and many other projects to enable CI on RISC-V.
Also happy to drastically increase the number of runners available for the Apache organization, given the overall importance of everything that you're doing!
For any direct board access, we are also working on a service of remote, on-demand machines accessible via SSH. Exactly for this kind of purpose where someone needs to debug a sticky issue and for which it's a great productivity loss to go through CI.
Let me know if you have any other questions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4093466969
Definitely a Docker-based job! You can take a look here for inspiration:
https://github.com/apache/arrow/blob/d08d5e64fcfd8759d3a7089eced3e9a2d7a17f20/.github/workflows/cpp.yml#L74
Note that this uses `archery docker`, which relies on [`compose.yaml`](https://github.com/apache/arrow/blob/main/compose.yaml). See https://arrow.apache.org/docs/developers/continuous_integration/docker.html, and feel free to ask any questions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49555: [Python][Packaging] Add riscv64 manylinux wheel builds to release pipeline [arrow]
Posted by "gounthar (via GitHub)" <gi...@apache.org>.
gounthar commented on PR #49556:
URL: https://github.com/apache/arrow/pull/49556#issuecomment-4093477502
@pitrou Thanks for the pointer! Docker-based it is. I'll study the `archery docker` approach and the `compose.yaml` service definitions for the C++ CI job.
I'll rework this PR to add a `ubuntu-cpp-riscv64` service in `compose.yaml` and a corresponding CI workflow, following the pattern from `cpp.yml#L74`. Will likely need to figure out the Docker image base (manylinux_2_39_riscv64 or a Ubuntu-based image for riscv64).
Will report back once I have a working prototype.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org