(datafusion) 01/01: dev: include dependency checks in the local lint suite (#25399)

[gi...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-25399-c4f72bad34249798182ac7de605eb63ab134f26e
in repository https://gitbox.apache.org/repos/asf/datafusion.git

commit b752f4722bffd008b5eda8de35623de0f4d25641
Author: Kumar Ujjawal <uj...@gmail.com>
AuthorDate: Thu Sep 17 15:23:59 2026 +0000

    dev: include dependency checks in the local lint suite (#25399)
    
    ## Which issue does this PR close?
    
    - Part of #21048.
    
    ## Rationale for this change
    
    `dev/rust_lint.sh` does not run the two dependency checks from the
    "Dependencies" workflow. A contributor finds a dependency cycle or an
    unused dependency only in CI.
    
    ## What changes are included in this PR?
    
    - `ci/scripts/check_circular_dependencies.sh` runs `cargo run --locked`
    in `dev/depcheck`.
    - `ci/scripts/check_unused_dependencies.sh` runs `cargo machete
    --with-metadata` from the repository root.
    - The workflow calls both scripts. `CARGO_MACHETE_VERSION="0.9"` moves
    to `ci/scripts/utils/tool_versions.sh`, and the workflow loads it from
    there.
    - `dev/rust_lint.sh` installs a missing `cargo-machete` and runs both
    scripts as read-only steps.
    - `docs/source/contributor-guide/testing.md` documents the standalone
    commands.
    
    ## What is the testing strategy for this PR?
    
    - In a clean clone, each original command and its wrapper gave identical
    output with exit 0, and no lockfile changed.
    - A fixture with stubbed steps and tools covered the working
    directories, exit codes, install path, and fail-fast behavior.
    - The parsed workflow YAML matches `main` except the changed steps. The
    full `./dev/rust_lint.sh` passes.
    
    ## Are there any user-facing changes?
    
    No.
---
 .github/workflows/dependencies.yml                 | 12 ++++++-----
 ..._versions.sh => check_circular_dependencies.sh} | 18 +++++++++++++----
 ...ol_versions.sh => check_unused_dependencies.sh} | 23 ++++++++++++++++++----
 ci/scripts/utils/tool_versions.sh                  |  1 +
 dev/rust_lint.sh                                   |  3 +++
 docs/source/contributor-guide/testing.md           | 20 +++++++++++++++++++
 6 files changed, 64 insertions(+), 13 deletions(-)

diff --git a/.github/workflows/dependencies.yml b/.github/workflows/dependencies.yml
index 6cab01c2a5..4d6549d941 100644
--- a/.github/workflows/dependencies.yml
+++ b/.github/workflows/dependencies.yml
@@ -51,9 +51,7 @@ jobs:
         with:
           rust-version: stable
       - name: Check dependencies
-        working-directory: dev/depcheck
-        run: |
-          cargo run --locked
+        run: bash ci/scripts/check_circular_dependencies.sh
 
   detect-unused-dependencies:
     name: Detect Unused Dependencies
@@ -62,9 +60,13 @@ jobs:
       image: amd64/rust
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1  # v7.0.1
+      - name: Load tool versions
+        run: |
+          . ci/scripts/utils/tool_versions.sh
+          echo "CARGO_MACHETE_VERSION=${CARGO_MACHETE_VERSION}" >> "$GITHUB_ENV"
       - name: Install cargo-machete
         uses: taiki-e/install-action@3f74d7c16a4242f1c95561e98edc25d36adb4375  # v2.87.12
         with:
-          tool: cargo-machete@0.9
+          tool: cargo-machete@${{ env.CARGO_MACHETE_VERSION }}
       - name: Detect unused dependencies
-        run: cargo machete --with-metadata
+        run: bash ci/scripts/check_unused_dependencies.sh
diff --git a/ci/scripts/utils/tool_versions.sh b/ci/scripts/check_circular_dependencies.sh
old mode 100644
new mode 100755
similarity index 59%
copy from ci/scripts/utils/tool_versions.sh
copy to ci/scripts/check_circular_dependencies.sh
index 237b18b62e..7cdc97b0d6
--- a/ci/scripts/utils/tool_versions.sh
+++ b/ci/scripts/check_circular_dependencies.sh
@@ -17,8 +17,18 @@
 # specific language governing permissions and limitations
 # under the License.
 
-# This file defines centralized tool versions used by CI and development scripts.
-# It is intended to be sourced by other scripts and should not be executed directly.
+# Checks for circular dependencies between DataFusion crates with
+# `dev/depcheck`, the same way the "Circular Dependency Check" job does.
 
-PRETTIER_VERSION="2.7.1"
-LYCHEE_VERSION="0.23.0"
+set -euo pipefail
+
+SCRIPT_NAME="$(basename "${BASH_SOURCE[0]}")"
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ROOT_DIR="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+
+# depcheck is outside the root workspace and finds the root manifest from its
+# own directory, so run Cargo from there.
+cd "${ROOT_DIR}/dev/depcheck"
+
+echo "[${SCRIPT_NAME}] \`cargo run --locked\` in dev/depcheck"
+cargo run --locked
diff --git a/ci/scripts/utils/tool_versions.sh b/ci/scripts/check_unused_dependencies.sh
old mode 100644
new mode 100755
similarity index 55%
copy from ci/scripts/utils/tool_versions.sh
copy to ci/scripts/check_unused_dependencies.sh
index 237b18b62e..0221d50dfd
--- a/ci/scripts/utils/tool_versions.sh
+++ b/ci/scripts/check_unused_dependencies.sh
@@ -17,8 +17,23 @@
 # specific language governing permissions and limitations
 # under the License.
 
-# This file defines centralized tool versions used by CI and development scripts.
-# It is intended to be sourced by other scripts and should not be executed directly.
+# Detects unused dependencies with `cargo machete`, the same way the
+# "Detect Unused Dependencies" job does.
 
-PRETTIER_VERSION="2.7.1"
-LYCHEE_VERSION="0.23.0"
+set -euo pipefail
+
+SCRIPT_NAME="$(basename "${BASH_SOURCE[0]}")"
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ROOT_DIR="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+
+source "${SCRIPT_DIR}/utils/tool_versions.sh"
+
+if ! command -v cargo-machete &> /dev/null; then
+  echo "[${SCRIPT_NAME}] cargo-machete is required. Install it with: cargo install cargo-machete --locked --version ^${CARGO_MACHETE_VERSION}" >&2
+  exit 1
+fi
+
+cd "${ROOT_DIR}"
+
+echo "[${SCRIPT_NAME}] \`cargo machete --with-metadata\`"
+cargo machete --with-metadata
diff --git a/ci/scripts/utils/tool_versions.sh b/ci/scripts/utils/tool_versions.sh
index 237b18b62e..b0a514edd0 100644
--- a/ci/scripts/utils/tool_versions.sh
+++ b/ci/scripts/utils/tool_versions.sh
@@ -22,3 +22,4 @@
 
 PRETTIER_VERSION="2.7.1"
 LYCHEE_VERSION="0.23.0"
+CARGO_MACHETE_VERSION="0.9"
diff --git a/dev/rust_lint.sh b/dev/rust_lint.sh
index 5862302d25..7e17dea9db 100755
--- a/dev/rust_lint.sh
+++ b/dev/rust_lint.sh
@@ -113,6 +113,7 @@ ensure_tool "hawkeye" "cargo install hawkeye --locked"
 ensure_tool "typos" "cargo install typos-cli --locked"
 ensure_tool "lychee" "cargo install lychee --locked --version ${LYCHEE_VERSION}"
 ensure_tool "cargo-audit" "cargo install cargo-audit --locked"
+ensure_tool "cargo-machete" "cargo install cargo-machete --locked --version ^${CARGO_MACHETE_VERSION}"
 
 run_step() {
   local name="$1"
@@ -135,6 +136,8 @@ declare -a READONLY_STEPS=(
   "ci/scripts/check_asf_yaml_status_checks.py|false"
   "ci/scripts/markdown_link_check.sh|false"
   "ci/scripts/security_audit.sh|false"
+  "ci/scripts/check_circular_dependencies.sh|false"
+  "ci/scripts/check_unused_dependencies.sh|false"
   "ci/scripts/rust_docs.sh|false"
 )
 
diff --git a/docs/source/contributor-guide/testing.md b/docs/source/contributor-guide/testing.md
index e50e0942d0..a4332352fa 100644
--- a/docs/source/contributor-guide/testing.md
+++ b/docs/source/contributor-guide/testing.md
@@ -254,6 +254,26 @@ The audit fetches the RustSec advisory database. A new advisory or a different
 
 [cargo-audit]: https://github.com/rustsec/rustsec/blob/main/cargo-audit/README.md
 
+## Dependency Checks
+
+CI runs two dependency checks, and `./dev/rust_lint.sh` runs both:
+
+- `ci/scripts/check_circular_dependencies.sh` builds and runs [`dev/depcheck`],
+  which fails on dependency cycles between DataFusion crates.
+- `ci/scripts/check_unused_dependencies.sh` runs `cargo machete --with-metadata`
+  from the repository root. The lint suite installs [cargo-machete] with the
+  version in `ci/scripts/utils/tool_versions.sh` if it is missing.
+
+To run either check on its own:
+
+```shell
+./ci/scripts/check_circular_dependencies.sh
+./ci/scripts/check_unused_dependencies.sh
+```
+
+[`dev/depcheck`]: https://github.com/apache/datafusion/tree/main/dev/depcheck
+[cargo-machete]: https://github.com/bnjbvr/cargo-machete
+
 ## Benchmarks
 
 ### Criterion Benchmarks


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org