You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by bl...@apache.org on 2026/05/06 17:30:28 UTC
(datafusion) branch speed-up-ci-macos-scope created (now c8761df4d1)
This is an automated email from the ASF dual-hosted git repository.
blaginin pushed a change to branch speed-up-ci-macos-scope
in repository https://gitbox.apache.org/repos/asf/datafusion.git
at c8761df4d1 ci: narrow macOS test scope to datafusion-ffi, run benchmarks on amd64
This branch includes the following new commits:
new c8761df4d1 ci: narrow macOS test scope to datafusion-ffi, run benchmarks on amd64
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org
(datafusion) 01/01: ci: narrow macOS test scope to datafusion-ffi, run benchmarks on amd64
Posted by bl...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
blaginin pushed a commit to branch speed-up-ci-macos-scope
in repository https://gitbox.apache.org/repos/asf/datafusion.git
commit c8761df4d1e191662fe5eee5be5f639bb2596a32
Author: blaginin <gi...@blaginin.me>
AuthorDate: Wed May 6 18:30:05 2026 +0100
ci: narrow macOS test scope to datafusion-ffi, run benchmarks on amd64
The macos-aarch64 job took ~22 min — 68% compile, 25% test. An audit of
macos-only PR failures over Mar–May 2026 (1000 failed PR runs, 30 with
macOS as the sole signal) found that nearly all the unique signal was
either:
- flaky sqllogictest metrics (~67%, e.g. push_down_filter / explain_analyze
.slt files with platform-dependent scan_efficiency_ratio numbers),
- real bugs in datafusion-benchmarks (which amd64 was excluding), or
- one genuinely macOS-specific FFI cdylib loading bug
(datafusion/ffi/src/tests/utils.rs path resolution).
So the only thing macOS uniquely catches is FFI dylib loading. Scope the
macOS job down to `cargo test -p datafusion-ffi --features integration-tests`
and let amd64 pick up the datafusion-benchmarks coverage that was being
dropped by the linux-test --exclude list.
Co-Authored-By: Claude Opus 4.7 (1M context) <no...@anthropic.com>
---
.github/workflows/rust.yml | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/rust.yml b/.github/workflows/rust.yml
index f6140ce5b4..c0c1f1dee6 100644
--- a/.github/workflows/rust.yml
+++ b/.github/workflows/rust.yml
@@ -298,7 +298,6 @@ jobs:
--profile ci \
--exclude datafusion-examples \
--exclude ffi_example_table_provider \
- --exclude datafusion-benchmarks \
--exclude datafusion-cli \
--workspace \
--lib \
@@ -557,6 +556,11 @@ jobs:
# export PATH=$PATH:$HOME/d/protoc/bin
# cargo test --lib --tests --bins --features avro,json,backtrace
+ # macOS scope is narrowed to `datafusion-ffi`: the only bug class amd64
+ # cannot reproduce is FFI cdylib loading (`.dylib` vs `.so` resolution in
+ # datafusion/ffi/src/tests/utils.rs). All other macos-only failures
+ # historically came from datafusion-benchmarks (now covered on amd64) or
+ # flaky sqllogictest metrics.
macos-aarch64:
name: cargo test (macos-aarch64)
runs-on: macos-15
@@ -567,9 +571,9 @@ jobs:
fetch-depth: 1
- name: Setup Rust toolchain
uses: ./.github/actions/setup-macos-aarch64-builder
- - name: Run tests (excluding doctests)
+ - name: Run datafusion-ffi tests
shell: bash
- run: cargo test --profile ci --exclude datafusion-cli --workspace --lib --tests --bins --features avro,json,backtrace,integration-tests,substrait
+ run: cargo test --profile ci -p datafusion-ffi --lib --tests --features integration-tests
vendor:
name: Verify Vendored Code
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org