(datafusion) branch main updated: ci: narrow macOS test scope to datafusion-ffi, run benchmarks on amd64 (#22048)

[gi...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

github-merge-queue[bot] pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion.git


The following commit(s) were added to refs/heads/main by this push:
     new 8d9ec685d6 ci: narrow macOS test scope to datafusion-ffi, run benchmarks on amd64 (#22048)
8d9ec685d6 is described below

commit 8d9ec685d669b2f5f98c1d0d3a358fd2d22d3186
Author: Dmitrii Blaginin <dm...@blaginin.me>
AuthorDate: Wed May 6 20:49:47 2026 +0100

    ci: narrow macOS test scope to datafusion-ffi, run benchmarks on amd64 (#22048)
    
    Follow up https://github.com/apache/datafusion/pull/21941
    
    The macos-aarch64 job took ~22 min - 68% compile, 25% test. An audit of
    macos-only PR failures over Mar-May 2026 (1000 failed PR runs, 30 with
    macOS as the sole signal) found that nearly all the unique signal was
    either:
    - flaky sqllogictest metrics (~67%, e.g. push_down_filter /
    explain_analyze .slt files with platform-dependent scan_efficiency_ratio
    numbers),
      - real bugs in datafusion-benchmarks (which amd64 was excluding), or
    - one genuinely macOS-specific FFI cdylib loading bug
    (datafusion/ffi/src/tests/utils.rs path resolution).
    
    So the only thing macOS uniquely catches is FFI dylib loading. Scope the
    macOS job down to `cargo test -p datafusion-ffi --features
    integration-tests` and let amd64 pick up the datafusion-benchmarks
    coverage that was being dropped by the linux-test --exclude list.
    
    Co-authored-by: Claude Opus 4.7 (1M context) <no...@anthropic.com>
---
 .github/workflows/rust.yml | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/.github/workflows/rust.yml b/.github/workflows/rust.yml
index f6140ce5b4..c0c1f1dee6 100644
--- a/.github/workflows/rust.yml
+++ b/.github/workflows/rust.yml
@@ -298,7 +298,6 @@ jobs:
             --profile ci \
             --exclude datafusion-examples \
             --exclude ffi_example_table_provider \
-            --exclude datafusion-benchmarks \
             --exclude datafusion-cli \
             --workspace \
             --lib \
@@ -557,6 +556,11 @@ jobs:
   #          export PATH=$PATH:$HOME/d/protoc/bin
   #          cargo test --lib --tests --bins --features avro,json,backtrace
 
+  # macOS scope is narrowed to `datafusion-ffi`: the only bug class amd64
+  # cannot reproduce is FFI cdylib loading (`.dylib` vs `.so` resolution in
+  # datafusion/ffi/src/tests/utils.rs). All other macos-only failures
+  # historically came from datafusion-benchmarks (now covered on amd64) or
+  # flaky sqllogictest metrics.
   macos-aarch64:
     name: cargo test (macos-aarch64)
     runs-on: macos-15
@@ -567,9 +571,9 @@ jobs:
           fetch-depth: 1
       - name: Setup Rust toolchain
         uses: ./.github/actions/setup-macos-aarch64-builder
-      - name: Run tests (excluding doctests)
+      - name: Run datafusion-ffi tests
         shell: bash
-        run: cargo test --profile ci --exclude datafusion-cli --workspace --lib --tests --bins --features avro,json,backtrace,integration-tests,substrait
+        run: cargo test --profile ci -p datafusion-ffi --lib --tests --features integration-tests
 
   vendor:
     name: Verify Vendored Code


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org