You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/09/24 06:30:16 UTC
(datafusion) branch gh-readonly-queue/main/pr-25586-2306a4b7599dc88490c0b39f082b4cdf554a5fb9 created (now 63ab2a855e)
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a change to branch gh-readonly-queue/main/pr-25586-2306a4b7599dc88490c0b39f082b4cdf554a5fb9
in repository https://gitbox.apache.org/repos/asf/datafusion.git
at 63ab2a855e dev: include the HTML documentation build in the local lint suite (#25586)
This branch includes the following new commits:
new 63ab2a855e dev: include the HTML documentation build in the local lint suite (#25586)
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org
(datafusion) 01/01: dev: include the HTML documentation build in the local lint suite (#25586)
Posted by gi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-25586-2306a4b7599dc88490c0b39f082b4cdf554a5fb9
in repository https://gitbox.apache.org/repos/asf/datafusion.git
commit 63ab2a855eea79f33b11ebeb6408537b7017c54f
Author: Kumar Ujjawal <uj...@gmail.com>
AuthorDate: Thu Sep 24 06:13:18 2026 +0000
dev: include the HTML documentation build in the local lint suite (#25586)
## Which issue does this PR close?
- Part of #21048.
## Rationale for this change
`./dev/rust_lint.sh` runs the non-functional CI checks locally, but not
the
documentation website build. Sphinx builds with `-W`, so one warning
fails CI.
A contributor sees that failure only after a push.
## What changes are included in this PR?
- `ci/scripts/check_docs_html.sh` runs the existing `docs/build.sh`
through uv.
It works from any directory. It reports a missing `uv`, `cargo`,
`cargo-depgraph`, `dot`, or `make` before the build starts, and installs
nothing. It has no write mode.
- `dev/rust_lint.sh` runs the new script as a read-only step.
- The "Test doc build" job runs the same script. The path filters of the
job
also watch the script.
- `docs/source/contributor-guide/testing.md` documents the command.
`docs/build.sh` still owns the Sphinx and dependency-graph commands. The
Python
dependencies and the Sphinx settings do not change.
## What is the testing strategy for this PR?
Local checks with command stubs and a real build:
- The script starts the same build from the repository root and from
other
directories.
- `-h`, `--help`, `--write`, `--allow-dirty`, and an unknown flag all
fail
before any tool runs.
- Each missing tool gives one message and a non-zero exit.
- Failures in `uv`, `cargo depgraph`, and `dot` keep their exit code.
- A real build writes `docs/build/html/index.html` and changes no
tracked file.
- One Sphinx warning, a page outside every toctree, fails the build.
`uv run ./dev/rust_lint.sh` passes with the new step.
## Are there any user-facing changes?
No. This changes the contributor tooling only.
---
.github/workflows/docs_pr.yaml | 7 ++-
ci/scripts/check_docs_html.sh | 87 ++++++++++++++++++++++++++++++++
dev/rust_lint.sh | 1 +
docs/source/contributor-guide/testing.md | 13 +++++
4 files changed, 104 insertions(+), 4 deletions(-)
diff --git a/.github/workflows/docs_pr.yaml b/.github/workflows/docs_pr.yaml
index 64be66f350..771e950723 100644
--- a/.github/workflows/docs_pr.yaml
+++ b/.github/workflows/docs_pr.yaml
@@ -26,12 +26,14 @@ on:
push:
paths:
- "docs/**"
+ - "ci/scripts/check_docs_html.sh"
branches:
- main
- branch-*
pull_request:
paths:
- "docs/**"
+ - "ci/scripts/check_docs_html.sh"
# manual trigger
# https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
workflow_dispatch:
@@ -63,7 +65,4 @@ jobs:
with:
tool: cargo-depgraph@1.6
- name: Build docs html and check for warnings
- run: |
- set -x
- cd docs
- uv run --package datafusion-docs ./build.sh # fails on errors
+ run: ./ci/scripts/check_docs_html.sh # fails on errors
diff --git a/ci/scripts/check_docs_html.sh b/ci/scripts/check_docs_html.sh
new file mode 100755
index 0000000000..0d04afff13
--- /dev/null
+++ b/ci/scripts/check_docs_html.sh
@@ -0,0 +1,87 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Builds the documentation website with `docs/build.sh`, the same way the
+# "Test doc build" job does. Sphinx runs with `-W`, so a warning fails the
+# build.
+
+set -euo pipefail
+
+SCRIPT_NAME="$(basename "${BASH_SOURCE[0]}")"
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ROOT_DIR="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+
+usage() {
+ cat >&2 <<USAGE
+Usage: $0
+
+Builds the HTML documentation with docs/build.sh and fails on any Sphinx
+warning. There is no write mode.
+
+Needs uv, cargo, cargo-depgraph, Graphviz dot, and make. See docs/README.md
+for the Python and Graphviz setup.
+
+Each build rewrites docs/build and docs/source/_static/data/deps.svg, which
+Git ignores. The HTML entry point is docs/build/html/index.html.
+USAGE
+ exit 1
+}
+
+while [[ $# -gt 0 ]]; do
+ case "$1" in
+ -h|--help)
+ usage
+ ;;
+ *)
+ usage
+ ;;
+ esac
+ shift
+done
+
+# Report every missing prerequisite in one run. This check installs nothing.
+missing=0
+require_tool() {
+ local cmd="$1"
+ local hint="$2"
+ if ! command -v "$cmd" > /dev/null 2>&1; then
+ echo "[${SCRIPT_NAME}] ${cmd} was not found on PATH. ${hint}" >&2
+ missing=1
+ fi
+}
+
+require_tool "uv" "Install uv to get Python and the documentation dependencies: https://docs.astral.sh/uv/getting-started/installation/"
+require_tool "cargo" "Install the Rust toolchain with rustup: https://rustup.rs/"
+require_tool "cargo-depgraph" "Install it with: cargo install cargo-depgraph --version '^1.6' --locked"
+require_tool "dot" "Install Graphviz (e.g., brew install graphviz, or apt-get install graphviz)."
+require_tool "make" "Install make (e.g., xcode-select --install, or apt-get install make)."
+
+if [[ "${missing}" -ne 0 ]]; then
+ exit 1
+fi
+
+cd "${ROOT_DIR}"
+
+echo "[${SCRIPT_NAME}] Building the documentation website with docs/build.sh"
+
+# `docs/build.sh` owns the dependency graph and Sphinx commands and moves to
+# the docs directory itself.
+uv run --package datafusion-docs ./docs/build.sh
+
+echo "[${SCRIPT_NAME}] Wrote the HTML documentation to docs/build/html/index.html"
diff --git a/dev/rust_lint.sh b/dev/rust_lint.sh
index 74d5fc16d7..dbb6454302 100755
--- a/dev/rust_lint.sh
+++ b/dev/rust_lint.sh
@@ -142,6 +142,7 @@ declare -a READONLY_STEPS=(
"ci/scripts/check_circular_dependencies.sh|false"
"ci/scripts/check_unused_dependencies.sh|false"
"ci/scripts/rust_docs.sh|false"
+ "ci/scripts/check_docs_html.sh|false"
)
for entry in "${WRITE_STEPS[@]}" "${READONLY_STEPS[@]}"; do
diff --git a/docs/source/contributor-guide/testing.md b/docs/source/contributor-guide/testing.md
index 89dc7969b3..4ba7d9d011 100644
--- a/docs/source/contributor-guide/testing.md
+++ b/docs/source/contributor-guide/testing.md
@@ -320,6 +320,19 @@ the README:
[`dev/update_function_docs.sh`]: https://github.com/apache/datafusion/blob/main/dev/update_function_docs.sh
[`ci/scripts/check_generated_docs.sh`]: https://github.com/apache/datafusion/blob/main/ci/scripts/check_generated_docs.sh
+## Documentation HTML Build
+
+[`ci/scripts/check_docs_html.sh`] builds the documentation website, the same
+build the "Test doc build" job runs. Sphinx builds with `-W`, so one warning
+fails the check. `./dev/rust_lint.sh` runs it. Run the script with `--help`
+for the tools it needs and the files it writes.
+
+```shell
+./ci/scripts/check_docs_html.sh
+```
+
+[`ci/scripts/check_docs_html.sh`]: https://github.com/apache/datafusion/blob/main/ci/scripts/check_docs_html.sh
+
## Benchmarks
### Criterion Benchmarks
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org