You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/09/17 15:59:57 UTC

(datafusion) branch gh-readonly-queue/main/pr-25399-22f9a921781ccfdf29be71bceac1f266e87cfa0e created (now 62f039f0d9)

This is an automated email from the ASF dual-hosted git repository.

github-merge-queue[bot] pushed a change to branch gh-readonly-queue/main/pr-25399-22f9a921781ccfdf29be71bceac1f266e87cfa0e
in repository https://gitbox.apache.org/repos/asf/datafusion.git


      at 62f039f0d9 dev: include dependency checks in the local lint suite (#25399)

This branch includes the following new commits:

     new 62f039f0d9 dev: include dependency checks in the local lint suite (#25399)

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org

(datafusion) 01/01: dev: include dependency checks in the local lint suite (#25399)

Posted by gi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-25399-22f9a921781ccfdf29be71bceac1f266e87cfa0e
in repository https://gitbox.apache.org/repos/asf/datafusion.git

commit 62f039f0d9583054b81d47b2f08b6728a14ac452
Author: Kumar Ujjawal <uj...@gmail.com>
AuthorDate: Thu Sep 17 15:59:34 2026 +0000

    dev: include dependency checks in the local lint suite (#25399)
    
    ## Which issue does this PR close?
    
    - Part of #21048.
    
    ## Rationale for this change
    
    `dev/rust_lint.sh` does not run the two dependency checks from the
    "Dependencies" workflow. A contributor finds a dependency cycle or an
    unused dependency only in CI.
    
    ## What changes are included in this PR?
    
    - `ci/scripts/check_circular_dependencies.sh` runs `cargo run --locked`
    in `dev/depcheck`.
    - `ci/scripts/check_unused_dependencies.sh` runs `cargo machete
    --with-metadata` from the repository root.
    - The workflow calls both scripts. `CARGO_MACHETE_VERSION="0.9"` moves
    to `ci/scripts/utils/tool_versions.sh`, and the workflow loads it from
    there.
    - `dev/rust_lint.sh` installs a missing `cargo-machete` and runs both
    scripts as read-only steps.
    - `docs/source/contributor-guide/testing.md` documents the standalone
    commands.
    
    ## What is the testing strategy for this PR?
    
    - In a clean clone, each original command and its wrapper gave identical
    output with exit 0, and no lockfile changed.
    - A fixture with stubbed steps and tools covered the working
    directories, exit codes, install path, and fail-fast behavior.
    - The parsed workflow YAML matches `main` except the changed steps. The
    full `./dev/rust_lint.sh` passes.
    
    ## Are there any user-facing changes?
    
    No.
---
 .github/workflows/dependencies.yml                 | 12 ++++++-----
 ..._versions.sh => check_circular_dependencies.sh} | 18 +++++++++++++----
 ...ol_versions.sh => check_unused_dependencies.sh} | 23 ++++++++++++++++++----
 ci/scripts/utils/tool_versions.sh                  |  1 +
 dev/rust_lint.sh                                   |  3 +++
 docs/source/contributor-guide/testing.md           | 20 +++++++++++++++++++
 6 files changed, 64 insertions(+), 13 deletions(-)

diff --git a/.github/workflows/dependencies.yml b/.github/workflows/dependencies.yml
index 6cab01c2a5..4d6549d941 100644
--- a/.github/workflows/dependencies.yml
+++ b/.github/workflows/dependencies.yml
@@ -51,9 +51,7 @@ jobs:
         with:
           rust-version: stable
       - name: Check dependencies
-        working-directory: dev/depcheck
-        run: |
-          cargo run --locked
+        run: bash ci/scripts/check_circular_dependencies.sh
 
   detect-unused-dependencies:
     name: Detect Unused Dependencies
@@ -62,9 +60,13 @@ jobs:
       image: amd64/rust
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1  # v7.0.1
+      - name: Load tool versions
+        run: |
+          . ci/scripts/utils/tool_versions.sh
+          echo "CARGO_MACHETE_VERSION=${CARGO_MACHETE_VERSION}" >> "$GITHUB_ENV"
       - name: Install cargo-machete
         uses: taiki-e/install-action@3f74d7c16a4242f1c95561e98edc25d36adb4375  # v2.87.12
         with:
-          tool: cargo-machete@0.9
+          tool: cargo-machete@${{ env.CARGO_MACHETE_VERSION }}
       - name: Detect unused dependencies
-        run: cargo machete --with-metadata
+        run: bash ci/scripts/check_unused_dependencies.sh
diff --git a/ci/scripts/utils/tool_versions.sh b/ci/scripts/check_circular_dependencies.sh
old mode 100644
new mode 100755
similarity index 59%
copy from ci/scripts/utils/tool_versions.sh
copy to ci/scripts/check_circular_dependencies.sh
index 237b18b62e..7cdc97b0d6
--- a/ci/scripts/utils/tool_versions.sh
+++ b/ci/scripts/check_circular_dependencies.sh
@@ -17,8 +17,18 @@
 # specific language governing permissions and limitations
 # under the License.
 
-# This file defines centralized tool versions used by CI and development scripts.
-# It is intended to be sourced by other scripts and should not be executed directly.
+# Checks for circular dependencies between DataFusion crates with
+# `dev/depcheck`, the same way the "Circular Dependency Check" job does.
 
-PRETTIER_VERSION="2.7.1"
-LYCHEE_VERSION="0.23.0"
+set -euo pipefail
+
+SCRIPT_NAME="$(basename "${BASH_SOURCE[0]}")"
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ROOT_DIR="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+
+# depcheck is outside the root workspace and finds the root manifest from its
+# own directory, so run Cargo from there.
+cd "${ROOT_DIR}/dev/depcheck"
+
+echo "[${SCRIPT_NAME}] \`cargo run --locked\` in dev/depcheck"
+cargo run --locked
diff --git a/ci/scripts/utils/tool_versions.sh b/ci/scripts/check_unused_dependencies.sh
old mode 100644
new mode 100755
similarity index 55%
copy from ci/scripts/utils/tool_versions.sh
copy to ci/scripts/check_unused_dependencies.sh
index 237b18b62e..0221d50dfd
--- a/ci/scripts/utils/tool_versions.sh
+++ b/ci/scripts/check_unused_dependencies.sh
@@ -17,8 +17,23 @@
 # specific language governing permissions and limitations
 # under the License.
 
-# This file defines centralized tool versions used by CI and development scripts.
-# It is intended to be sourced by other scripts and should not be executed directly.
+# Detects unused dependencies with `cargo machete`, the same way the
+# "Detect Unused Dependencies" job does.
 
-PRETTIER_VERSION="2.7.1"
-LYCHEE_VERSION="0.23.0"
+set -euo pipefail
+
+SCRIPT_NAME="$(basename "${BASH_SOURCE[0]}")"
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ROOT_DIR="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+
+source "${SCRIPT_DIR}/utils/tool_versions.sh"
+
+if ! command -v cargo-machete &> /dev/null; then
+  echo "[${SCRIPT_NAME}] cargo-machete is required. Install it with: cargo install cargo-machete --locked --version ^${CARGO_MACHETE_VERSION}" >&2
+  exit 1
+fi
+
+cd "${ROOT_DIR}"
+
+echo "[${SCRIPT_NAME}] \`cargo machete --with-metadata\`"
+cargo machete --with-metadata
diff --git a/ci/scripts/utils/tool_versions.sh b/ci/scripts/utils/tool_versions.sh
index 237b18b62e..b0a514edd0 100644
--- a/ci/scripts/utils/tool_versions.sh
+++ b/ci/scripts/utils/tool_versions.sh
@@ -22,3 +22,4 @@
 
 PRETTIER_VERSION="2.7.1"
 LYCHEE_VERSION="0.23.0"
+CARGO_MACHETE_VERSION="0.9"
diff --git a/dev/rust_lint.sh b/dev/rust_lint.sh
index 5862302d25..7e17dea9db 100755
--- a/dev/rust_lint.sh
+++ b/dev/rust_lint.sh
@@ -113,6 +113,7 @@ ensure_tool "hawkeye" "cargo install hawkeye --locked"
 ensure_tool "typos" "cargo install typos-cli --locked"
 ensure_tool "lychee" "cargo install lychee --locked --version ${LYCHEE_VERSION}"
 ensure_tool "cargo-audit" "cargo install cargo-audit --locked"
+ensure_tool "cargo-machete" "cargo install cargo-machete --locked --version ^${CARGO_MACHETE_VERSION}"
 
 run_step() {
   local name="$1"
@@ -135,6 +136,8 @@ declare -a READONLY_STEPS=(
   "ci/scripts/check_asf_yaml_status_checks.py|false"
   "ci/scripts/markdown_link_check.sh|false"
   "ci/scripts/security_audit.sh|false"
+  "ci/scripts/check_circular_dependencies.sh|false"
+  "ci/scripts/check_unused_dependencies.sh|false"
   "ci/scripts/rust_docs.sh|false"
 )
 
diff --git a/docs/source/contributor-guide/testing.md b/docs/source/contributor-guide/testing.md
index e50e0942d0..a4332352fa 100644
--- a/docs/source/contributor-guide/testing.md
+++ b/docs/source/contributor-guide/testing.md
@@ -254,6 +254,26 @@ The audit fetches the RustSec advisory database. A new advisory or a different
 
 [cargo-audit]: https://github.com/rustsec/rustsec/blob/main/cargo-audit/README.md
 
+## Dependency Checks
+
+CI runs two dependency checks, and `./dev/rust_lint.sh` runs both:
+
+- `ci/scripts/check_circular_dependencies.sh` builds and runs [`dev/depcheck`],
+  which fails on dependency cycles between DataFusion crates.
+- `ci/scripts/check_unused_dependencies.sh` runs `cargo machete --with-metadata`
+  from the repository root. The lint suite installs [cargo-machete] with the
+  version in `ci/scripts/utils/tool_versions.sh` if it is missing.
+
+To run either check on its own:
+
+```shell
+./ci/scripts/check_circular_dependencies.sh
+./ci/scripts/check_unused_dependencies.sh
+```
+
+[`dev/depcheck`]: https://github.com/apache/datafusion/tree/main/dev/depcheck
+[cargo-machete]: https://github.com/bnjbvr/cargo-machete
+
 ## Benchmarks
 
 ### Criterion Benchmarks


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org