(datafusion) 01/01: dev: include Markdown link checks in the local lint suite (#25287)

[gi...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-25287-22cf99d82bd18d19b63dbbe0246cfd2e6d264f96
in repository https://gitbox.apache.org/repos/asf/datafusion.git

commit afaee3d4cece5f2d4784df2bd02c9b1a5402ab03
Author: Kumar Ujjawal <uj...@gmail.com>
AuthorDate: Mon Sep 14 15:16:09 2026 +0000

    dev: include Markdown link checks in the local lint suite (#25287)
    
    ## Which issue does this PR close?
    
    - Part of #21048. The issue stays open for the other checklist items.
    
    ## Rationale for this change
    
    `dev/rust_lint.sh` is the local mirror of the CI lint jobs, but it does
    not run the Markdown link check. CI runs
    `ci/scripts/markdown_link_check.sh` in `dev.yml`. A developer can push a
    broken internal link after a clean local lint run and then see the
    failure only in CI. This PR adds the existing checker to the local
    suite.
    
    ## What changes are included in this PR?
    
    - `dev/rust_lint.sh` runs `ci/scripts/markdown_link_check.sh` as a
    read-only step, after the workflow install check and before the Rust
    documentation build. The `--write` and `--allow-dirty` flags never reach
    it.
    - The runner loads `LYCHEE_VERSION` from
    `ci/scripts/utils/tool_versions.sh` and installs that version when
    `lychee` is missing. An installed `lychee` is used as is, which is the
    runner's existing policy for other tools.
    - `ci/scripts/markdown_link_check.sh` becomes executable, because the
    runner invokes each registered script directly. The script body,
    `lychee.toml`, the file selection, and the GitHub workflow do not
    change.
    - `docs/source/contributor-guide/testing.md` documents the new behavior
    and keeps the standalone instructions.
    
    ## What is the testing strategy for this PR?
    
    - A disposable fixture with stubbed steps and tools: the checker runs
    once with no arguments in check mode and in both write modes, a missing
    `lychee` triggers exactly the pinned install command, a present `lychee`
    triggers no install, and a checker failure stops the suite before the
    later steps.
    - A disposable clone with an injected broken internal link: the
    standalone checker and `./dev/rust_lint.sh` both fail with the same exit
    code and name the file. After the fix, the full suite passes.
    - The real checker and the full `./dev/rust_lint.sh` pass on this branch
    with `lychee` 0.23.0.
    
    ## Are there any user-facing changes?
    
    No. Developers get the link check in the local lint suite. CI is
    unchanged.
    
    🤖 Generated with [Claude Code](https://claude.com/claude-code)
---
 ci/scripts/markdown_link_check.sh        |  0
 dev/rust_lint.sh                         | 10 +++++++++-
 docs/source/contributor-guide/testing.md |  8 +++++++-
 3 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/ci/scripts/markdown_link_check.sh b/ci/scripts/markdown_link_check.sh
old mode 100644
new mode 100755
diff --git a/dev/rust_lint.sh b/dev/rust_lint.sh
index 73cab9c7f7..542ed18c6b 100755
--- a/dev/rust_lint.sh
+++ b/dev/rust_lint.sh
@@ -22,7 +22,9 @@
 #
 # Note: The installed checking tools (e.g., taplo) are not guaranteed to match
 # the CI versions for simplicity, there might be some minor differences. Check
-# `.github/workflows` for the CI versions.
+# `.github/workflows` for the CI versions. When this script installs a missing
+# tool that has a pinned version in `ci/scripts/utils/tool_versions.sh`, it
+# installs that pinned version. An already installed tool is used as is.
 #
 #
 #
@@ -84,10 +86,15 @@ while [[ $# -gt 0 ]]; do
 done
 
 SCRIPT_NAME="$(basename "${BASH_SOURCE[0]}")"
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+# Load the tool versions shared with CI (for example, LYCHEE_VERSION).
+source "${SCRIPT_DIR}/../ci/scripts/utils/tool_versions.sh"
 
 ensure_tool "taplo" "cargo install taplo-cli --locked"
 ensure_tool "hawkeye" "cargo install hawkeye --locked"
 ensure_tool "typos" "cargo install typos-cli --locked"
+ensure_tool "lychee" "cargo install lychee --locked --version ${LYCHEE_VERSION}"
 
 run_step() {
   local name="$1"
@@ -107,6 +114,7 @@ declare -a WRITE_STEPS=(
 
 declare -a READONLY_STEPS=(
   "ci/scripts/check_no_cargo_install_in_workflows.sh|false"
+  "ci/scripts/markdown_link_check.sh|false"
   "ci/scripts/rust_docs.sh|false"
 )
 
diff --git a/docs/source/contributor-guide/testing.md b/docs/source/contributor-guide/testing.md
index 44bd327013..2d0cad99cf 100644
--- a/docs/source/contributor-guide/testing.md
+++ b/docs/source/contributor-guide/testing.md
@@ -188,7 +188,12 @@ tested in the same way using the [doc_comment] crate. See the end of
 
 ## Documentation Link Checks
 
-Run the internal markdown link check locally:
+`./dev/rust_lint.sh` runs the internal markdown link check. If `lychee` is
+missing, the script installs the version pinned in
+`ci/scripts/utils/tool_versions.sh`. It uses an existing installation as is,
+even if the version differs from the pin.
+
+To run the check on its own:
 
 ```shell
 source ci/scripts/utils/tool_versions.sh
@@ -200,6 +205,7 @@ Notes:
 
 - The script is run with `bash` and is compatible with the default Bash on macOS (no `mapfile` dependency).
 - The CI configuration currently checks internal markdown links only. External `http(s)` and `mailto` links are excluded to avoid flaky failures.
+- The check only reports broken links. `./dev/rust_lint.sh --write` does not change them.
 
 When a link is broken, lychee prints the file and URL/path that failed. For example:
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org