You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudberry.apache.org by "MisterRaindrop (via GitHub)" <gi...@apache.org> on 2026/10/20 08:41:48 UTC
[PR] Add Arrow and Parquet C++ to the Rocky build images [cloudberry]
MisterRaindrop opened a new pull request, #2155:
URL: https://github.com/apache/cloudberry/pull/2155
### What does this PR do?
Installs Arrow and Parquet C++ in the Rocky 8, 9 and 10 build images, so `contrib/datalake_fdw` (#2044) can build against them without each CI job installing them itself.
- From the Arrow project's own repository, pinned: **22.0.0 on Rocky 8**, the last release whose headers are C++17 (from 23 on they need C++20, which gcc 8 does not have), and **26.0.0 on Rocky 9 and 10**. This is the split agreed in #2044.
- The repository is left disabled afterwards, as EPEL is, so nothing moves the version but a change to the Dockerfile.
- Testinfra checks that `pkg-config --modversion` reports the pinned version for `arrow` and `parquet`.
- The job timeout goes from 60 to 90 minutes (see below).
The Ubuntu images are not changed: Apache's `libarrow-dev` pulls in `libcurl4-openssl-dev`, which removes the `libcurl4-gnutls-dev` that the deb packaging (`devops/build/packaging/deb/ubuntu*/control`) depends on. That needs its own discussion.
### Type of Change
- [ ] Bug fix (non-breaking change)
- [x] New feature (non-breaking change)
- [ ] Breaking change (fix or feature with breaking changes)
- [ ] Documentation update
### Test Plan
- Built all three images locally (native arm64): testinfra passes (10 tests), the Arrow repository is disabled afterwards.
- Ran this workflow on a fork, amd64 plus QEMU arm64, exactly as it runs here: https://github.com/MisterRaindrop/cloudberry/actions/runs/39384206248 — all three Rocky images build and pass testinfra on both architectures.
| Image | Job total | Arrow layer, amd64 | Arrow layer, arm64 (QEMU) |
|---|---|---|---|
| Rocky 8 | 43 min | 48 s | 4.1 min |
| Rocky 9 | 41 min | 47 s | 3.9 min |
| Rocky 10 | 53 min | 33 s | 3.0 min |
**Why the timeout changes.** The first fork run had Rocky 9 cancelled at 60 minutes while exporting its arm64 image. Its Arrow layer took 12 minutes under QEMU, eight of them spent loading repository metadata for a release package that needs none; it now installs with `--disablerepo='*'`. But the part of the build that does not involve Arrow varies a lot on its own: the arm64 main layer took 30 to 41 minutes across these runs, and the Rocky jobs took 35 to 60 minutes before this change. So 60 leaves no margin.
### Impact
**Dependencies:**
Image size grows by about 440–540 MB per Rocky image. Most of it is the static libraries in `arrow-devel` (`libarrow_bundled_dependencies.a` 150 MB, `libarrow.a` 70 MB) and `boost-devel`, which `thrift-devel` requires. `datalake_fdw` links the shared libraries, so the static ones could be deleted in the same layer if the size matters; I left them so the image stays a plain package install.
### Checklist
- [x] Followed [contribution guide](https://cloudberry.apache.org/contribute/code)
- [ ] Added/updated documentation
- [x] Reviewed code for security implications
- [ ] This PR contains AI-assisted code generation
- [ ] Requested review from [cloudberry committers](https://github.com/orgs/apache/teams/cloudberry-committers)
### Additional Context
Once the published images carry Arrow, a follow-up changes the datalake job in `build-cloudberry.yml` to use it and drops the Arrow install there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Add Arrow and Parquet C++ to the Rocky build images [cloudberry]
Posted by "MisterRaindrop (via GitHub)" <gi...@apache.org>.
MisterRaindrop merged PR #2155:
URL: https://github.com/apache/cloudberry/pull/2155
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org