You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "kumarUjjawal (via GitHub)" <gi...@apache.org> on 2026/09/20 05:25:37 UTC
[PR] dev: include generated config and function docs in the local lint suite [datafusion]
kumarUjjawal opened a new pull request, #25525:
URL: https://github.com/apache/datafusion/pull/25525
## Which issue does this PR close?
- Part of #21048.
## Rationale for this change
`dev/rust_lint.sh` does not check the generated `configs.md` and function pages. A contributor finds a stale page only in the `config-docs-check` job. That job also regenerates the tracked pages in place and then runs `git diff`, so the same check cannot run in a working tree with other changes.
## What changes are included in this PR?
- `ci/scripts/check_config_function_docs.sh` regenerates the four pages into per-run scratch directories, compares each with its working file, and prints a diff for each stale page. Check mode never rewrites a page. `--write` replaces the four pages, with the same clean-tree rule as `rust_fmt.sh`.
- `dev/update_config_docs.sh` and `dev/update_function_docs.sh` gain `--output-dir DIR`, so the checker reuses the existing generators and headers. Without the option, both scripts behave as before.
- `dev/rust_lint.sh` runs the checker as a write step.
- The `config-docs-check` job runs the checker and keeps its `git diff --exit-code` guard. The job identity and setup do not change.
- `docs/source/contributor-guide/testing.md` documents the check and the update command.
## What is the testing strategy for this PR?
- In a clean clone with the real generators: the no-argument scripts and `--output-dir` produce byte-identical pages, the four committed pages pass, a committed stale page fails with its diff, `--write` restores it byte for byte, the dirty-tree gate refuses, and staged edits survive check mode. No lockfile changed.
- A fixture with stub `cargo` and `npx` covered each page stale on its own, generator and formatter failures in both modes, cleanup after failure and interruption, concurrent runs, missing `npx`, comparison errors, and argument errors. No page changed in any failure case.
- The parsed workflow matches `main` except the replaced steps. The full `./dev/rust_lint.sh` passes.
## Are there any user-facing changes?
No.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] dev: include generated config and function docs in the local lint suite [datafusion]
Posted by "kumarUjjawal (via GitHub)" <gi...@apache.org>.
kumarUjjawal merged PR #25525:
URL: https://github.com/apache/datafusion/pull/25525
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] dev: include generated config and function docs in the local lint suite [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #25525:
URL: https://github.com/apache/datafusion/pull/25525#discussion_r4056709529
##########
ci/scripts/check_config_function_docs.sh:
##########
Review Comment:
Minor: I suggest calling this `check_generated_docs.sh` as it is both shorter and more general
##########
docs/source/contributor-guide/testing.md:
##########
@@ -304,6 +304,23 @@ the README:
./ci/scripts/check_examples_docs.sh --write
```
+## Config and Function Docs Check
+
+`docs/source/user-guide/configs.md` and the three function pages under
Review Comment:
Can we also please make file references github url links so it is easier to follow what is going on?
ALso I think a bunch of details of the implementation are irrelevant for people running this check (e.g. that needs cargo and npx ,etc) is irrelevant here -- we could just say somehting like
> To check they are up to date, run the `ci/scripts/check_config_function_docs.sh` script, which is also run as part of `./dev/rust_lint.sh`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] dev: include generated config and function docs in the local lint suite [datafusion]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #25525:
URL: https://github.com/apache/datafusion/pull/25525#issuecomment-5747947812
## [Codecov](https://app.codecov.io/gh/apache/datafusion/pull/25525?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 82.38%. Comparing base ([`b4a8c82`](https://app.codecov.io/gh/apache/datafusion/commit/b4a8c824b44bffacf681f74d064107a0c1f71fff?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`ce84b5c`](https://app.codecov.io/gh/apache/datafusion/commit/ce84b5cbb843492772a91af64d7f9a745be6ef43?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #25525 +/- ##
=======================================
Coverage 82.38% 82.38%
=======================================
Files 1138 1138
Lines 434328 434328
Branches 434328 434328
=======================================
+ Hits 357824 357826 +2
+ Misses 54872 54870 -2
Partials 21632 21632
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/datafusion/pull/25525?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] dev: include generated config and function docs in the local lint suite [datafusion]
Posted by "kumarUjjawal (via GitHub)" <gi...@apache.org>.
kumarUjjawal commented on PR #25525:
URL: https://github.com/apache/datafusion/pull/25525#issuecomment-5750317342
Thank you @alamb for suggestions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org