You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@juneau.apache.org by ja...@apache.org on 2026/10/23 18:39:03 UTC
(juneau) branch juneau-release-manager updated: Add a .md doc next to every script with a checker test (test_script_docs.py), plus argparse-first --help handling in coverage, push and release; adopt the juneau-run session header and lines= title in push.py; strip live-run JUNEAU_RUN_* variables from the script tests' environment (test_env_isolation.py); and add @SuppressWarnings("resource") with ownership rationales to the Eclipse resource-leak warnings in the release service, REST and tests.
This is an automated email from the ASF dual-hosted git repository.
jamesbognar pushed a commit to branch juneau-release-manager
in repository https://gitbox.apache.org/repos/asf/juneau.git
The following commit(s) were added to refs/heads/juneau-release-manager by this push:
new e252001893 Add a .md doc next to every script with a checker test (test_script_docs.py), plus argparse-first --help handling in coverage, push and release; adopt the juneau-run session header and lines= title in push.py; strip live-run JUNEAU_RUN_* variables from the script tests' environment (test_env_isolation.py); and add @SuppressWarnings("resource") with ownership rationales to the Eclipse resource-leak warnings in the release service, REST and tests.
e252001893 is described below
commit e252001893006f79982e55cba85eeba58ebaa621
Author: James Bognar <ja...@apache.org>
AuthorDate: Fri Oct 23 14:38:56 2026 -0400
Add a .md doc next to every script with a checker test (test_script_docs.py), plus argparse-first --help handling in coverage, push and release; adopt the juneau-run session header and lines= title in push.py; strip live-run JUNEAU_RUN_* variables from the script tests' environment (test_env_isolation.py); and add @SuppressWarnings("resource") with ownership rationales to the Eclipse resource-leak warnings in the release service, REST and tests.
---
scripts/coverage.md | 62 +++
scripts/coverage.py | 7 +-
scripts/jrm_release/README.md | 32 ++
scripts/push.md | 71 ++++
scripts/push.py | 145 +++----
scripts/release.md | 75 ++++
scripts/release.py | 18 +-
scripts/script-docs-exclude.txt | 18 +
scripts/tests/README.md | 16 +-
scripts/tests/conftest.py | 19 +
scripts/tests/test_env_isolation.py | 43 +++
scripts/tests/test_push.py | 50 ++-
scripts/tests/test_script_docs.py | 428 +++++++++++++++++++++
.../org/apache/juneau/releng/AppConfiguration.java | 1 +
.../apache/juneau/releng/rest/ReleaseRunRest.java | 1 +
.../juneau/releng/run/ReleaseProcessService.java | 5 +
.../juneau/releng/log/EventBroadcasterTest.java | 2 +
.../juneau/releng/log/RunEventsSseServletTest.java | 1 +
.../juneau/releng/rest/ReleaseRunRestTest.java | 2 +
.../apache/juneau/releng/run/PromptBrokerTest.java | 3 +
.../releng/run/ReleaseProcessServiceTest.java | 5 +
.../juneau/releng/run/RunFileWatcherTest.java | 1 +
22 files changed, 859 insertions(+), 146 deletions(-)
diff --git a/scripts/coverage.md b/scripts/coverage.md
new file mode 100644
index 0000000000..7a891ada37
--- /dev/null
+++ b/scripts/coverage.md
@@ -0,0 +1,62 @@
+# coverage.py
+
+Per-package Java coverage report for the Release Manager, built on the JaCoCo data a `mvn test` run leaves in `target/`.
+
+## When to use
+
+- You want per-package branch and instruction coverage, or the uncovered lines of one package or file.
+- To run the Java tests themselves, use `mvn verify` or `push.py --test-only`; this script only reports (or, with `--run`, refreshes the data first).
+
+## Usage
+
+```bash
+python3 scripts/coverage.py --by-module
+python3 scripts/coverage.py engine
+python3 scripts/coverage.py --branches engine
+python3 scripts/coverage.py --run --by-module
+python3 scripts/coverage.py --help
+```
+
+This file only supplies this repository's settings (`REPO_ROOT`, base package `org/apache/juneau/releng`, report name, Maven arguments). The implementation is `juneau_scripts.coverage` from the Juneau `juneau-run` artifact, loaded through `scripts/bootstrap.py`.
+
+## Options
+
+The toolkit parses these by hand, not with `argparse`. Running with no arguments, `--help` or `-h` prints the help text and exits 0.
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `<package>` | top-level package name or a source path or file | none | Per-file detail and uncovered lines for that package, path or file. Example: `engine`. |
+| `--by-module` | none | off | Summary table with one row per top-level package under `org.apache.juneau.releng`, sorted by branch percent ascending. Classes directly in that package are the `(root)` row. |
+| `--run`, `-r` | none | off | Run `mvn test -Drat.skip=true -q` first, to refresh `target/jacoco.exec`. |
+| `--branches`, `-b` | none | off | In the detail view, show only lines with missed branches. |
+| `--help`, `-h` | none | | Print the help text. |
+
+An unknown option that starts with `-` prints `ERROR: Unknown option: ...` and exits 1.
+
+## What it does
+
+1. Loads the toolkit through `bootstrap.load(pom.xml, run=False, cache_dir=target/juneau-run)`. If the installed toolkit has no `juneau_scripts.coverage`, it exits with an error asking you to refresh `juneau-run`.
+2. Parses the arguments. With `--run`, runs `mvn test -Drat.skip=true -q`.
+3. Requires `target/jacoco.exec` (otherwise exits 1, telling you to run `mvn clean test -Drat.skip=true` or pass `--run`), then runs the JaCoCo CLI over `target/classes` and `src/main/java` to write `target/coverage/jacoco.xml`.
+4. Prints the `--by-module` table or the detail report for the named package. Lines marked with a standalone `HTT` token in a comment (hard to test) are dropped from the uncovered list and subtracted from the totals; their reasons are printed.
+5. With neither `--by-module` nor a package, exits 1 with `No package or path specified`.
+
+## Output and exit codes
+
+Reports go to stdout. The generated XML is `target/coverage/jacoco.xml`.
+
+| Code | Meaning |
+|---|---|
+| 0 | Report printed, or help shown. |
+| 1 | Missing `target/jacoco.exec`, unknown option, no target given, or a stale toolkit. |
+
+## Gotchas
+
+- Needs the Juneau `juneau-run` toolkit (build or install Juneau first, or set `JUNEAU_SCRIPTS_ZIP`); see `bootstrap.py`.
+- Coverage data comes from the JaCoCo agent wired into `pom.xml`, so a plain compile does not produce it.
+- `HTT` matches only as a whole word inside a comment, not `HTTP`, and not inside a string literal.
+
+## See also
+
+- `push.md` (the Java Tests gate)
+- `tests/README.md`
diff --git a/scripts/coverage.py b/scripts/coverage.py
index 9fc6e8570c..07886a1c49 100755
--- a/scripts/coverage.py
+++ b/scripts/coverage.py
@@ -11,12 +11,7 @@
# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
# * specific language governing permissions and limitations under the License.
# ***************************************************************************************************************************
-"""
-Per-package Java coverage report for the Juneau Release Manager. The implementation is Juneau's
-juneau_scripts.coverage, loaded by the vendored scripts/bootstrap.py; this file only supplies this repository's settings.
-
- ./scripts/coverage.py --help
-"""
+"""Per-package Java coverage report for the Release Manager. Full docs: coverage.md"""
import sys
from pathlib import Path
diff --git a/scripts/jrm_release/README.md b/scripts/jrm_release/README.md
new file mode 100644
index 0000000000..62bd8b113c
--- /dev/null
+++ b/scripts/jrm_release/README.md
@@ -0,0 +1,32 @@
+# jrm_release
+
+The step implementations behind `scripts/release.py`: one module per concern, imported by `release.py`, not run directly.
+
+`release.py` owns the command line, the run lock and the session; this package owns what each step does. Steps are functions of one argument, `fn(ctx)`, returning an optional summary string, and are registered by name in a `FUNCS` map per phase. For usage see `../release.md`.
+
+## Modules
+
+| Module | Role |
+|---|---|
+| `inventory.py` | Constants (repository, Nexus and dist URLs, mailing lists), the table of 25 steps with their phase and whether they are skippable, and version helpers. |
+| `ctx.py` | The `Ctx` object every step receives (arguments, home, runner, Nexus, SVN, run file, credentials) and `StepError`. |
+| `runfile.py` | The `run.json` checkpoint (atomic writes, in-memory under `--dry-run`, refusal to read a corrupt file) and attempt directories. |
+| `runner.py` | The one place a command runs; under `--dry-run` a mutating command is printed, never executed. Masks secrets. |
+| `prereqs.py` | The checks that run before any prompt: tools, versions and reachability. |
+| `details.py` | The release details form. |
+| `rc.py`, `vote.py`, `promote.py` | The release-candidate, vote and promote phase steps. |
+| `revert.py` | The `--revert` rows. |
+| `diffcheck.py` | A pure checker for the release plugin's two commits; callers pass diff text in. |
+| `emails.py` | Vote, result and announce email text and the send flow. |
+| `nexus.py`, `svn.py`, `gpgkeys.py` | Nexus staging REST, `svn` and `svnmucc`, and parsing of `gpg --list-secret-keys`. |
+| `askpass.py` | The `GIT_ASKPASS` helper: answers git's prompts from `JRM_ASKPASS_USER` and `JRM_ASKPASS_PASS` so no secret is on a command line. |
+
+## Conventions
+
+- Every external command goes through `Runner`, so `--dry-run` and secret masking apply uniformly.
+- A step that fails raises `StepError`; its message is the row's failure text.
+- Tests use fakes only (see `../tests/README.md`).
+
+## See also
+
+- `../release.md`
diff --git a/scripts/push.md b/scripts/push.md
new file mode 100644
index 0000000000..bcc2ed8fb3
--- /dev/null
+++ b/scripts/push.md
@@ -0,0 +1,71 @@
+# push.py
+
+Runs the build and test gates for the Release Manager, then commits the already-staged index and pushes it to the branch's upstream.
+
+## When to use
+
+- You have staged your reviewed changes and want them verified, committed and pushed in one command.
+- You want only the gates, with no commit or push (works on a dirty tree): use `--test-only`.
+- You want to see what would happen without changing anything: use `--dry-run`.
+- To run only the Python suite, use pytest directly (see `tests/README.md`).
+
+## Usage
+
+```bash
+python3 scripts/push.py "Fix credential validator NPE"
+python3 scripts/push.py
+python3 scripts/push.py --dry-run "Would-be commit message"
+python3 scripts/push.py --test-only
+python3 scripts/push.py --skip-e2e "Doc-only change"
+python3 scripts/push.py --console full "Fix credential validator NPE"
+```
+
+Run it from the repository root. The commit message is prompted for when omitted (not used with `--test-only`). There is no root-level `push.py`.
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `message` | text (optional positional) | prompted | Commit message. An empty or aborted prompt ends the run with nothing committed. |
+| `--dry-run` | none | off | Show what would happen without changing anything. Compile, Java Tests and Browser Tests are shown as "would run", not executed; Prechecks and Python Tests do run (they only read) and a failure is reported as "would refuse". |
+| `--test-only` | none | off | Run the gates only: no Prechecks, no commit, no push. |
+| `--skip-tests` | none | off | Skip Compile and Java Tests (the Maven build). |
+| `--skip-e2e` | none | off | Skip Browser Tests (the Playwright suite). |
+| `--skip-pytest` | none | off | Skip Python Tests (the `scripts/tests` suite). |
+| `--console` | `full`, `condensed`, `none` | `condensed` | Console view of the steps; also `JUNEAU_RUN_CONSOLE`. The flag wins over the environment. |
+
+## What it does
+
+1. Parses the arguments and loads the `juneau_scripts` toolkit through `bootstrap.py`, applying `--console` before `juneau_run` loads.
+2. Prechecks (skipped by `--test-only`): one row of four checks that all run, none stopping at the first failure: git identity, unreviewed changes (any unstaged modification to a tracked file, or any untracked file that is not gitignored), staged (nothing staged ends the run with "nothing to do", before any gate), and upstream (fetches; refuses when the branch is behind its upstream; no upstream proceeds; a fetch error fails closed).
+3. Compile: `mvn clean test-compile`. Java Tests: `mvn verify` (the Java tests plus Apache RAT). Never `install`, since this repo only consumes the Juneau snapshot already in `~/.m2`. Both are removed by `--skip-tests`.
+4. Python Tests: the `scripts/tests` pytest suite, so a push cannot ship a broken release CLI. The launcher is `python3 -m pytest` when pytest is importable, else `uv run --native-tls --with pytest pytest` when `uv` is on the `PATH`, else the step is skipped with a notice. Removed by `--skip-pytest`.
+5. Browser Tests: `npx playwright test` in `e2e/`, whose web server boots the app itself. Skipped with a warning when `npx` is not on the `PATH`. Removed by `--skip-e2e`.
+6. A failing gate refuses the push; nothing is committed.
+7. Commits the staged index exactly as found (message on stdin) and pushes to the tracked upstream (setting it with `-u` when the branch has none). It never stages anything.
+
+## Output and exit codes
+
+Progress is a condensed run view (rows: Prechecks, Compile, Java Tests, Python Tests, Browser Tests, Commit, Push). The full log path is printed in the session header. On success it prints the branch, short commit and remote URL. When a run ends on a terminal it rings the bell.
+
+| Code | Meaning |
+|---|---|
+| 0 | Pushed, nothing to commit, `--dry-run` report, or `--test-only` passed. |
+| 1 | A precheck or gate failed, the commit or push failed, or the toolkit could not be loaded. |
+| 2 | Argparse rejected the arguments, or loading the toolkit raised `ValueError`. |
+
+## Gotchas
+
+- It never runs `git add`, `git config`, `--force` or `--no-verify`. A staged path is treated as reviewed, so any unstaged tracked change or untracked non-ignored file makes the review gate refuse, with no override. Use `git stash push -- <paths>` for genuinely partial work.
+- A fully clean tree is a benign no-op and runs no gate.
+- Needs `juneau-run` 10.0.0-SNAPSHOT in `~/.m2` (build Juneau first).
+- `JRM_PUSH_NO_BELL=1` silences the bell (it is also skipped when `SANDBOX_RUNTIME=1`).
+- `JAVA_HOME` defaults to `~/jdk/default` when unset and that directory exists.
+- `JRM_E2E_REUSE`, `JRM_E2E_BASE_URL` and `JRM_E2E_HOME` are removed from the Playwright environment; `PLAYWRIGHT_HTML_OPEN` defaults to `never`.
+- `JRM_SCRIPT_TESTS_ACTIVE` is set in the pytest child and the step is skipped when it is already set, so tests that call `main()` cannot recurse. `JUNEAU_RUN_PY` is exported to the child when unset and the canonical Juneau checkout has `juneau_run.py`.
+- The toolkit is cached under `~/.cache/juneau-run/juneau-release-manager` because Compile's `mvn clean` would delete a copy under `target/`.
+
+## See also
+
+- `release.md`, `coverage.md`
+- `tests/README.md`
diff --git a/scripts/push.py b/scripts/push.py
index c5b5f3ba01..f1483d3ef4 100755
--- a/scripts/push.py
+++ b/scripts/push.py
@@ -14,68 +14,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-"""
-Commit and Push Script for juneau-release-manager
-
-Runs the build and test gates, then commits the reviewed (staged) index and pushes it. Lives in scripts/, like
-apache/juneau's and sandbox-support-console's push.py; there is no root push.py.
-
-Usage:
- python3 scripts/push.py "commit message"
- python3 scripts/push.py # prompts for a commit message
- python3 scripts/push.py --dry-run "message" # show what would happen, change nothing
- python3 scripts/push.py --test-only # the gates only; no commit, no push
- python3 scripts/push.py --skip-e2e "message" # skip the E2E row (--skip-tests, --skip-pytest skip theirs)
- python3 scripts/push.py --console full "message" # plain output; the default is the condensed console rows
-
- --console full|condensed|none sets JUNEAU_RUN_CONSOLE before juneau_run is loaded (the flag wins over a value
- already in the environment; with neither, the default is condensed), exactly as scripts/test.py does in
- apache/juneau. Condensed (the default, or RUN_MARKERS=1) makes this script load juneau_run, with the git output
- going to the full log; full and none leave it unloaded, so both give the plain output. The console view shows
- the Prechecks, Compile, Java tests, Python tests, E2E, Commit and Push rows.
-
- When a run ends (success or failure) on a terminal, push.py rings the terminal bell.
- Set JRM_PUSH_NO_BELL=1 to silence it (it is also skipped when SANDBOX_RUNTIME=1).
-
-Steps: Prechecks, Compile, Java tests, Python tests, E2E, Commit, Push. Prechecks is one row of four checks that all run,
- none stopping at the first failure: identity, unreviewed (the review gate), staged (a warning, "nothing staged",
- ends the run with nothing to do, before any gate) and upstream. The upstream check fetches and refuses (before the
- commit) when the branch is behind its upstream, since the push would be rejected anyway; a branch with no upstream
- proceeds (the push then sets it), and a fetch error fails closed. Under --dry-run each failing check shows as a
- "would refuse" warning.
- Python tests runs the scripts/tests pytest suite, so a push cannot ship a broken release CLI. The launcher is
- `python3 -m pytest` when pytest is importable, else `uv run --native-tls --with pytest pytest` when uv is on the
- PATH, else the step is skipped with a visible notice (as in Juneau's scripts/test.py). JUNEAU_RUN_PY is exported
- to the child when it is unset and the canonical Juneau checkout has juneau_run.py, so the release CLI tests run
- instead of skipping. JRM_SCRIPT_TESTS_ACTIVE is set in the child and the step is skipped when it is already set,
- so tests that call main() cannot recurse. Under --dry-run the suite still runs (it only reads), and a failure is
- a "would refuse" warning.
- Compile runs `mvn clean test-compile` and Java tests runs `mvn verify` (the Java tests and Apache RAT); never
- `install`, since this repo only consumes the Juneau snapshot already in ~/.m2. JAVA_HOME defaults to
- ~/jdk/default when unset. E2E runs the full Playwright suite (`npx playwright test` in e2e/), whose webServer
- boots the app on the test port itself; JRM_E2E_REUSE, JRM_E2E_BASE_URL and JRM_E2E_HOME are removed from its
- environment and PLAYWRIGHT_HTML_OPEN defaults to never. Without npx on the PATH, E2E is skipped with a warning. A failing gate refuses the push
- (nothing committed). Under --dry-run, Compile, Java tests and E2E are shown as "would run" and not executed.
- --test-only runs every gate that is not skipped, with no Prechecks, commit or push, so it works on a dirty tree.
- --skip-pytest, --skip-tests (Compile and Java tests) and --skip-e2e each remove their steps.
-
-Safety:
- - Never uses --force or --no-verify.
- - Never runs `git config` (it only reads config; you fix a missing
- identity yourself).
- - Never runs `git add`: a staged path is the operator's record that
- they've reviewed it and it's good, so this script commits the index
- exactly as it finds it and refuses outright -- commits and pushes
- nothing -- if the working tree has any unstaged modification or any
- untracked, non-gitignored file.
- - Exits non-zero with a clear message on any precondition failure
- (no git identity configured, unreviewed changes present, push
- rejected, etc).
-
-Built on Apache Juneau's juneau_scripts toolkit, loaded by the vendored scripts/bootstrap.py from the juneau-run
-artifact at the pom's juneau.version. The policy above is this file's code; the toolkit supplies the git, run and
-bell helpers.
-"""
+"""Build, test, commit and push juneau-release-manager. Full docs: push.md"""
import argparse
import os
@@ -86,7 +25,7 @@ from pathlib import Path
REPO_DIR = Path(__file__).resolve().parent.parent
-# Set in the pytest child so a test that reaches the Python tests step again cannot launch the suite recursively.
+# Set in the pytest child so a test that reaches the Python Tests step again cannot launch the suite recursively.
SCRIPT_TESTS_ACTIVE = "JRM_SCRIPT_TESTS_ACTIVE"
# Where the canonical Juneau checkout keeps juneau_run.py; test_release_cli.py skips without JUNEAU_RUN_PY.
JUNEAU_RUN_PY_DEFAULT = Path.home() / "git" / "apache" / "juneau" / "juneau-run" / "src" / "main" / "python" / "juneau_run.py"
@@ -222,16 +161,16 @@ def script_tests_env():
def _skip_script_tests(why, short):
- """The Python tests row, opened and skipped, so it does not stay pending."""
- with run.step("script-tests", "Python tests") as st: # NOSONAR python:S1192 - a step title reads more clearly inline
- say(f"โ Skipping Python tests: {why}.")
- run_markers.note("warn", f"Python tests skipped: {short}.")
+ """The Python Tests row, opened and skipped, so it does not stay pending."""
+ with run.step("script-tests", "Python Tests") as st: # NOSONAR python:S1192 - a step title reads more clearly inline
+ say(f"โ Skipping Python Tests: {why}.")
+ run_markers.note("warn", f"Python Tests skipped: {short}.")
st.skip(short)
def _script_tests(s, repo_dir, dry_run, launcher=None):
"""
- The Python tests row: the scripts/tests pytest suite, run before the commit. A failure refuses the push -- nothing
+ The Python Tests row: the scripts/tests pytest suite, run before the commit. A failure refuses the push -- nothing
is committed -- except under --dry-run, where it is a "would refuse" warning. Skipped, with a notice, when already
inside the suite (the recursion guard) or when neither pytest nor uv is available. Returns True when the push may
go on.
@@ -245,20 +184,20 @@ def _script_tests(s, repo_dir, dry_run, launcher=None):
"neither pytest nor uv found")
return True
cmd = script_tests_command(launcher)
- say("\n๐ Running Python tests (scripts/tests)...")
+ say("\n๐ Running Python Tests (scripts/tests)...")
say(f"Running: {' '.join(cmd)}")
- r = run.cmd(cmd, title="Python tests", id="script-tests", tool="pytest", cwd=str(repo_dir), env=script_tests_env(),
+ r = run.cmd(cmd, title="Python Tests", id="script-tests", tool="pytest", cwd=str(repo_dir), env=script_tests_env(),
capture=True)
if r.ok:
- say("โ
Python tests passed!")
+ say("โ
Python Tests passed!")
return True
if r.out:
say(r.out[-4000:])
if dry_run:
- say(f"โ Would refuse to push: Python tests failed (exit code {r.code}).")
- run_markers.note("warn", f"Dry run โ would refuse to push: Python tests failed (exit code {r.code}).")
+ say(f"โ Would refuse to push: Python Tests failed (exit code {r.code}).")
+ run_markers.note("warn", f"Dry run โ would refuse to push: Python Tests failed (exit code {r.code}).")
return False
- s.fail(f"\nโ ERROR: Python tests failed (exit code {r.code}); nothing committed.")
+ s.fail(f"\nโ ERROR: Python Tests failed (exit code {r.code}); nothing committed.")
return False
@@ -325,13 +264,13 @@ def _compile(s, repo_dir, dry_run):
def _maven_tests(s, repo_dir, dry_run):
- """The Java tests row: `mvn verify`, the Java tests plus Apache RAT. No clean: Compile just ran it."""
- _maven_gate(s, repo_dir, dry_run, ["verify"], "tests", "Java tests")
+ """The Java Tests row: `mvn verify`, the Java Tests plus Apache RAT. No clean: Compile just ran it."""
+ _maven_gate(s, repo_dir, dry_run, ["verify"], "tests", "Java Tests")
def e2e_env():
"""
- The E2E child's environment: build_env() without the variables in E2E_REDIRECT_ENV (they point the suite at a
+ The Browser Tests child's environment: build_env() without the variables in E2E_REDIRECT_ENV (they point the suite at a
server or release home other than the tree under test), and PLAYWRIGHT_HTML_OPEN=never unless already set.
On a terminal the playwright parser leaves PLAYWRIGHT_HTML_OPEN alone, and the html reporter's default (open on
failure) would serve the report and hold the push open.
@@ -345,29 +284,29 @@ def e2e_env():
def _e2e(s, repo_dir, dry_run):
"""
- The E2E row: the full Playwright suite (`npx playwright test` in e2e/). Its webServer boots the app on the test
+ The Browser Tests row: the full Playwright suite (`npx playwright test` in e2e/). Its webServer boots the app on the test
port itself, so no server is started here. The playwright parser adds its reporter through PW_TEST_REPORTER and no
CLI arguments, so the config's own html reporter still runs. Skipped with a warning when npx is not on the PATH;
npm dependencies and Chromium are installed only when missing, and a failed install fails the gate. A failing
suite refuses the push: nothing is committed.
"""
if dry_run:
- _would_run("e2e", "E2E", "npx playwright test (in e2e/)")
+ _would_run("e2e", "Browser Tests", "npx playwright test (in e2e/)")
return
if shutil.which("npx") is None:
- with run.step("e2e", "E2E") as st:
- say("โ Skipping E2E: npx not found on the PATH (install Node.js to run the Playwright suite).")
- run_markers.note("warn", "E2E skipped: npx not found.")
+ with run.step("e2e", "Browser Tests") as st:
+ say("โ Skipping Browser Tests: npx not found on the PATH (install Node.js to run the Playwright suite).")
+ run_markers.note("warn", "Browser Tests skipped: npx not found.")
st.skip("npx not found")
return
e2e_dir = Path(repo_dir) / "e2e"
- say("\n๐ญ Running E2E (npx playwright test in e2e/)...")
+ say("\n๐ญ Running Browser Tests (npx playwright test in e2e/)...")
if not tools.playwright_ensure(e2e_dir).ok:
- s.fail("\nโ ERROR: E2E setup failed (npm dependencies or Chromium could not be installed); nothing committed.")
- r = tools.playwright([], cwd=str(e2e_dir), env=e2e_env(), title="E2E", id="e2e")
+ s.fail("\nโ ERROR: Browser Tests setup failed (npm dependencies or Chromium could not be installed); nothing committed.")
+ r = tools.playwright([], cwd=str(e2e_dir), env=e2e_env(), title="Browser Tests", id="e2e")
if not r.ok:
- s.fail(f"\nโ ERROR: E2E failed (exit code {r.code}); nothing committed. Report: e2e/playwright-report/")
- say("โ
E2E passed!")
+ s.fail(f"\nโ ERROR: Browser Tests failed (exit code {r.code}); nothing committed. Report: e2e/playwright-report/")
+ say("โ
Browser Tests passed!")
def _skip_commit_and_push():
@@ -439,21 +378,21 @@ def _skipped(flag, what):
def _gates(s, repo_dir, args):
"""
- Compile, Java tests, Python tests and E2E, in that order (every test step after Compile), each unless its skip flag is
- set. A failing gate ends the session with nothing committed. Returns False only for a Python tests "would refuse" under --dry-run;
+ Compile, Java Tests, Python Tests and Browser Tests, in that order (every test step after Compile), each unless its skip flag is
+ set. A failing gate ends the session with nothing committed. Returns False only for a Python Tests "would refuse" under --dry-run;
otherwise True.
"""
if args.skip_tests:
- _skipped("--skip-tests", "Compile and Java tests")
+ _skipped("--skip-tests", "Compile and Java Tests")
else:
_compile(s, repo_dir, args.dry_run)
_maven_tests(s, repo_dir, args.dry_run)
if args.skip_pytest:
- _skipped("--skip-pytest", "Python tests")
+ _skipped("--skip-pytest", "Python Tests")
elif not _script_tests(s, repo_dir, args.dry_run):
return False
if args.skip_e2e:
- _skipped("--skip-e2e", "E2E")
+ _skipped("--skip-e2e", "Browser Tests")
else:
_e2e(s, repo_dir, args.dry_run)
return True
@@ -461,7 +400,7 @@ def _gates(s, repo_dir, args):
def commit_and_push(s, repo_dir, args):
"""
- Run the Prechecks and the gates (Compile, Java tests, Python tests, E2E), then commit whatever is already staged in
+ Run the Prechecks and the gates (Compile, Java Tests, Python Tests, Browser Tests), then commit whatever is already staged in
repo_dir and push, refusing to commit or push ANYTHING if a precheck fails (identity, unreviewed content, behind
upstream; see _prechecks()) or a gate fails.
@@ -523,9 +462,9 @@ def planned_steps(args):
figure. A skip flag removes its steps; --test-only has no Prechecks, Commit or Push."""
steps = [] if args.test_only else [("prechecks", "Prechecks")]
steps += [(step_id, label) for step_id, label, skipped in (("compile", "Compile", args.skip_tests),
- ("tests", "Java tests", args.skip_tests),
- ("script-tests", "Python tests", args.skip_pytest),
- ("e2e", "E2E", args.skip_e2e)) if not skipped]
+ ("tests", "Java Tests", args.skip_tests),
+ ("script-tests", "Python Tests", args.skip_pytest),
+ ("e2e", "Browser Tests", args.skip_e2e)) if not skipped]
if not args.test_only:
steps += [("commit", "Commit"), ("push", "Push")]
return steps
@@ -545,7 +484,7 @@ Examples:
python3 scripts/push.py --console full "Fix credential validator NPE"
Steps: Prechecks (identity, review gate, staged, upstream: refuses when behind upstream, before committing), Python
-tests (scripts/tests pytest), Compile (mvn clean test-compile), Java tests (mvn verify: Java tests and RAT), E2E (npx
+tests (scripts/tests pytest), Compile (mvn clean test-compile), Java Tests (mvn verify: Java Tests and RAT), Browser Tests (npx
playwright test in e2e/), Commit, Push. A failing gate refuses the push; nothing is committed.
""",
)
@@ -558,16 +497,16 @@ playwright test in e2e/), Commit, Push. A failing gate refuses the push; nothin
parser.add_argument(
"--dry-run",
action="store_true",
- help="Show what would be done without making any changes (Compile, Java tests and E2E are shown, not run).",
+ help="Show what would be done without making any changes (Compile, Java Tests and Browser Tests are shown, not run).",
)
parser.add_argument(
"--test-only",
action="store_true",
help="Run the gates only: no Prechecks, no commit, no push (works on a dirty tree).",
)
- parser.add_argument("--skip-tests", action="store_true", help="Skip Compile and Java tests (the Maven build).")
- parser.add_argument("--skip-e2e", action="store_true", help="Skip E2E (the Playwright suite).")
- parser.add_argument("--skip-pytest", action="store_true", help="Skip Python tests (the scripts/tests suite).")
+ parser.add_argument("--skip-tests", action="store_true", help="Skip Compile and Java Tests (the Maven build).")
+ parser.add_argument("--skip-e2e", action="store_true", help="Skip Browser Tests (the Playwright suite).")
+ parser.add_argument("--skip-pytest", action="store_true", help="Skip Python Tests (the scripts/tests suite).")
parser.add_argument(
"--console",
choices=("full", "condensed", "none"),
@@ -593,7 +532,9 @@ def main(argv=None, repo_dir=REPO_DIR):
return 2
run_markers = jr if jr is not None else _NoMarkers()
- with run.session("๐ juneau-release-manager push", header=(git.current_branch(repo_dir) or "HEAD",), bell=True,
+ with run.session("Test" if args.test_only else "Push",
+ header=("JRM", git.current_branch(repo_dir) or "HEAD"),
+ lines=(f"Commit Message: {args.message}",) if args.message else (), bell=True,
silence_env="JRM_PUSH_NO_BELL", mode="test" if args.test_only else "push",
project="juneau-release-manager", repo=repo_dir, exit=False, plan=planned_steps(args)) as s:
if not run_markers.console_active():
diff --git a/scripts/release.md b/scripts/release.md
new file mode 100644
index 0000000000..967e7fc160
--- /dev/null
+++ b/scripts/release.md
@@ -0,0 +1,75 @@
+# release.py
+
+Drives an Apache Juneau release from a terminal: build and stage a release candidate, open the vote, then (once the vote passes) promote it.
+
+## When to use
+
+- You are the release manager cutting a release candidate, resuming after the vote, or dropping a candidate.
+- To see where a release stands without changing anything: `--list-steps`.
+- To rehearse: `--dry-run` runs read-only commands and prints every mutation.
+- To run the Python tests for this script, use pytest (see `tests/README.md`); to push repository changes, use `push.md`.
+
+## Usage
+
+```bash
+python3 scripts/release.py # start a new release (asks for the details)
+python3 scripts/release.py --version 10.0.0 # start a new release, prefilling the details
+python3 scripts/release.py --list-steps # steps by phase, finished ones marked
+python3 scripts/release.py --resume # continue after the last finished step
+python3 scripts/release.py --start-step vote-tally
+python3 scripts/release.py --skip-step javadoc-verify
+python3 scripts/release.py --revert # undo the current release candidate
+python3 scripts/release.py --dry-run
+```
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `--resume` | none | off | Continue the active run after its last finished step. Mutually exclusive with `--start-step`. |
+| `--start-step` | `NAME` (one of the step names) | none | Start, or restart, at this step. Refused when the release is already public and the step is before `nexus-release`, when a promote step is chosen before the vote has passed, and when a vote step is chosen before `vote-gate` has passed. |
+| `--skip-step` | `NAME` (repeatable) | none | Skip a step. Only skippable steps are accepted (currently `javadoc-verify`). |
+| `--list-steps` | none | off | Print the steps by phase (Release candidate, Vote, Promote), with finished ones marked, and exit. |
+| `--version` | `x.y.z` | none | With `--list-steps`, which run's marks to show; on a new release, prefills the details. Must look like `x.y.z`. |
+| `--revert` | none | off | Undo the current release candidate. Cannot be combined with `--resume`, `--start-step` or `--skip-step`. |
+| `--dry-run` | none | off | Run read-only commands; print every mutation instead of running it. State is kept in memory, and `run.json` is never written. |
+| `--console` | `full`, `condensed`, `none` | toolkit default | Console view; the flag beats `JUNEAU_RUN_CONSOLE`. |
+| `--home` | `DIR` | `$JUNEAU_RELEASE_HOME`, else `~/.juneau-release-manager` | State directory. |
+
+`--attempt-dir` also exists but is hidden from the help (used by tests).
+
+## What it does
+
+1. Parses and validates the arguments (a bad combination exits 2). `--list-steps` prints the table and exits 0.
+2. Looks for the active run under `<home>/runs/<version>/run.json` (the newest one with details that is not yet `released`). `--resume`, `--revert` and `--start-step` with no active run exit 2; starting a new release while one is in progress also exits 2 and tells you to use `--resume` or `--revert`.
+3. Loads the `juneau_scripts` toolkit through `bootstrap.py` (needs `juneau-run` 10.0.0-SNAPSHOT in `~/.m2`; build Juneau first).
+4. Creates an attempt directory `<home>/attempts/<YYYYMMDD-HHMMSS>-<NNN>/` with `full.log` (a dry run uses `attempts/dry-run`).
+5. Runs the prerequisite checks before asking anything (tools and minimum versions: `git`, `mvn` 3.6, `java` 17, `svn` and `svnmucc` 1.10, `gpg` 2.1, `gh` 2.28, `unzip`). Every failure is listed together; nothing was asked or changed. Skipped for `--revert`.
+6. For a new release, asks for the release details (version, development version, RC number, summary, highlights, known issues, acknowledgements) and starts a fresh `run.json`.
+7. Takes a non-blocking lock (`.lock` in the run directory) and runs the steps in order, checkpointing `run.json` after each. A second `release.py` on the same release exits 1 with the holder's pid.
+8. The 25 steps fall in three phases: release candidate (prerequisites through `vote-gate`), vote (`vote-tally`, result email) and promote (`nexus-release` through `finalize`). The run pauses at `vote-gate`; after the vote closes, run `release.py --resume`.
+9. `--revert` runs six rows instead: confirm, drop the Nexus staging repository, remove the RC from dist/dev, delete the RC tag, reset the pom versions, record the revert. Each row checks remote state first, so a rerun is harmless.
+
+## Output and exit codes
+
+A condensed run view with one row per step. State lives under the home directory: `runs/<version>/run.json` (the checkpoint) and `attempts/<stamp>-<NNN>/full.log` (the full log, whose `tail -f` line the banner prints). Secrets are masked in output and never put on a command line or in `run.json`.
+
+| Code | Meaning |
+|---|---|
+| 0 | Finished, or paused (for example at the vote gate). |
+| 1 | A step or prerequisite failed, another run holds the lock, `run.json` is corrupt, or the toolkit could not be loaded. |
+| 2 | Usage error, or the requested action does not fit the active run. |
+| 130 | Cancelled with Ctrl-C. |
+
+## Gotchas
+
+- Everything from `nexus-release` on is irreversible (the release is public). After that, `--start-step` before it is refused; continue from `dist-promote`.
+- A corrupt `run.json` is never guessed at: the script stops and asks you to inspect or delete it.
+- `JUNEAU_RELEASE_HOME` moves the state directory; `--home` overrides it. `JRM_RELEASE_NO_BELL` silences the completion bell.
+- The Juneau checkout is expected at `~/git/apache/juneau`.
+- The Apache id, password, GPG key id and passphrase, and GitHub token are asked for as needed. The password reaches `svn` over stdin, and git gets credentials through the `askpass.py` helper reading `JRM_ASKPASS_USER` and `JRM_ASKPASS_PASS`.
+
+## See also
+
+- `jrm_release/README.md` (the step implementations)
+- `push.md`, `tests/README.md`
diff --git a/scripts/release.py b/scripts/release.py
index dcb5cb13dd..9d558a10b6 100644
--- a/scripts/release.py
+++ b/scripts/release.py
@@ -11,15 +11,7 @@
# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
# * specific language governing permissions and limitations under the License.
# ***************************************************************************************************************************
-"""
-Apache Juneau release, from a terminal: release candidate, vote, promote.
-
- release.py [--resume | --start-step NAME] [--skip-step NAME ...] [--list-steps] [--revert] [--dry-run]
- [--console full|condensed|none] [--home DIR]
-
-Exit codes: 0 ok or paused, 1 failed, 2 usage, 130 cancelled. State: $JUNEAU_RELEASE_HOME or
-~/.juneau-release-manager (runs/<version>/run.json, attempts/<stamp>-<NNN>/full.log).
-"""
+"""Apache Juneau release, from a terminal: release candidate, vote, promote. Full docs: release.md"""
import argparse
import os
import sys
@@ -36,7 +28,7 @@ from jrm_release import inventory # noqa: E402
from jrm_release.ctx import Ctx # noqa: E402
from jrm_release.runfile import RunFile, RunFileError, new_attempt_dir # noqa: E402
-TITLE = "๐ juneau-release-manager release"
+TITLE = "Release"
FINISHED = ("released",)
@@ -45,7 +37,7 @@ def default_home():
def build_parser():
- p = argparse.ArgumentParser(prog="release.py", description=__doc__.strip().splitlines()[0])
+ p = argparse.ArgumentParser(prog="release.py", description="Apache Juneau release, from a terminal: release candidate, vote, promote.")
g = p.add_mutually_exclusive_group()
g.add_argument("--resume", action="store_true", help="continue the active run after its last finished step")
g.add_argument("--start-step", choices=inventory.NAMES, metavar="NAME",
@@ -351,8 +343,8 @@ def run_release(args, deps, path, active):
attempt.mkdir(parents=True, exist_ok=True)
(attempt / "full.log").touch() # the banner's `tail -f` target; a dry run writes logs but never runs/
version = (active.get("details") or {}).get("version")
- header = (f"drop RC{(active.get('details') or {}).get('rc')} of {version}" if args.revert
- else version or "new release",)
+ header = ("JRM", f"drop RC{(active.get('details') or {}).get('rc')} of {version}" if args.revert
+ else version or "new release")
plan = ([(n, t) for n, t, _ in __import__("jrm_release.revert", fromlist=["ROWS"]).ROWS] if args.revert
else [(s.name, s.title) for s in inventory.STEPS])
saved = os.environ.get("JUNEAU_RUN_FULL_LOG")
diff --git a/scripts/script-docs-exclude.txt b/scripts/script-docs-exclude.txt
new file mode 100644
index 0000000000..b8ce210f6f
--- /dev/null
+++ b/scripts/script-docs-exclude.txt
@@ -0,0 +1,18 @@
+# ***************************************************************************************************************************
+# * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an
+# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# * specific language governing permissions and limitations under the License.
+# ***************************************************************************************************************************
+# Entry points and packages that scripts/tests/test_script_docs.py treats specially.
+# One entry per line. Path patterns are relative to scripts/ (fnmatch); each needs a '# reason'.
+# <pattern> # reason skip the doc and flag checks for matching entry points
+# flags: <pattern> # reason require the doc, but skip the flag check
+# packages: a, b library directories that must have a README.md
+packages: jrm_release
diff --git a/scripts/tests/README.md b/scripts/tests/README.md
index 3c21e3b58d..39bf9d39a1 100644
--- a/scripts/tests/README.md
+++ b/scripts/tests/README.md
@@ -2,13 +2,13 @@
This repo's `scripts/todo-next-id.py` and `scripts/todo-status-audit.py` (and their tests,
`conftest.py` / `test_todo_next_id.py` / `test_todo_status_audit.py`) were consolidated
-2026-08-30 into a single parameterized copy at `~/Project Work/scripts/` (`--project
-release-manager`), with tests alongside at `~/Project Work/scripts/tests/`. See
-`@todo-and-waves` and `~/agents/AGENTS.md`.
+2026-08-30 into a single parameterized copy (`--project
+release-manager`) in the shared tracker scripts, which live outside this repo, with tests alongside them. See
+`@todo-and-waves` and the operator's agent instructions.
This repo's `scripts/push.py` (moved here from the repo root, matching `apache/juneau` and
-`sandbox-support-console`) runs the build and test gates before it commits: Prechecks, Compile (`mvn clean test-compile`), Java tests (`mvn verify`), Python tests
-(this suite), E2E (`npx playwright test` in `e2e/`), Commit, Push. The suite never writes the real run history:
+`sandbox-support-console`) runs the build and test gates before it commits: Prechecks, Compile (`mvn clean test-compile`), Java Tests (`mvn verify`), Python Tests
+(this suite), Browser Tests (`npx playwright test` in `e2e/`), Commit, Push. The suite never writes the real run history:
`conftest.py` points `JUNEAU_RUN_HISTORY_DIR` at a temp dir and sets `JUNEAU_RUN_HISTORY=0`. Its review gate refuses to commit or push **anything** when the working tree has
any unstaged modification to a tracked file, or any untracked file that isn't gitignored (`_prechecks()` /
`commit_and_push()`, built on the `juneau_scripts` toolkit's `git`, `run` and `tools` modules, loaded through
@@ -31,6 +31,12 @@ this repo's does not need; this repo also has no `reset-side-clones.py` or `repi
(juneau's `test_reset_side_clones.py` / `test_repin_consumers.py` cover those, and stay in
juneau since the scripts they test do too).
+## Script docs checker
+
+`test_script_docs.py` keeps `scripts/` documented. It is read-only: it parses files and never runs a script. It fails when an entry point (a `*.sh`, an extensionless file with a shell or Python shebang, or a `*.py` with a top-level `if __name__ == "__main__":`) has no `<name>.md` beside it; when a script doc (H1 `# <name>.py`) has no script beside it; when a script doc lacks the seven headings in order (When to use, Usage, Options, What it does, Output and exit codes, Gotchas, See also); [...]
+
+`scripts/script-docs-exclude.txt` holds the exceptions, one per line: `<pattern> # reason` skips an entry point (fnmatch, relative to `scripts/`), `flags: <pattern> # reason` skips only the flag check (needed when argparse flags are added dynamically, reported as `FLAGS-UNVERIFIABLE`), and `packages: a, b` lists library directories that must have a `README.md` (here `jrm_release`). Every path and `flags:` line needs a reason, and a pattern that matches nothing fails the test.
+
## Running
`pytest` is not vendored anywhere in this repo. Two easy ways to run this suite without
diff --git a/scripts/tests/conftest.py b/scripts/tests/conftest.py
index 8216839aa1..72102451b1 100644
--- a/scripts/tests/conftest.py
+++ b/scripts/tests/conftest.py
@@ -230,3 +230,22 @@ def make_ctx(home):
ctx.fake = fake
return ctx
return make
+
+
+# Every JUNEAU_RUN_* variable that points a session at a live parent run (its logs, cancel command, markers, result,
+# header and pending/overall rows, or the fact that one is active). JUNEAU_RUN_CONSOLE, _DETAIL and _HISTORY* are
+# settings, not pointers, and are handled elsewhere.
+LIVE_RUN_ENV = (
+ "JUNEAU_RUN_ACTIVE", "JUNEAU_RUN_SESSION", "JUNEAU_RUN_LIVE", "JUNEAU_RUN_N_BASE", "JUNEAU_RUN_LABEL_WIDTH",
+ "JUNEAU_RUN_FULL_LOG", "JUNEAU_RUN_CONDENSED_LOG", "JUNEAU_RUN_CANCEL", "JUNEAU_RUN_HEADER",
+ "JUNEAU_RUN_PENDING", "JUNEAU_RUN_OVERALL", "JUNEAU_RUN_HANDBACK",
+ "JUNEAU_RUN_MARKERS_FILE", "JUNEAU_RUN_RESULT", "JUNEAU_RUN_RESULT_PARTS",
+)
+
+
+@pytest.fixture(autouse=True)
+def _no_outer_run_files(monkeypatch):
+ """A run under watch-run.sh or push.py exports its live-run variables (see LIVE_RUN_ENV) to this suite; drop them so
+ the tests see the same run state as under a plain run (and don't write the outer run's logs or files)."""
+ for name in LIVE_RUN_ENV:
+ monkeypatch.delenv(name, raising=False)
diff --git a/scripts/tests/test_env_isolation.py b/scripts/tests/test_env_isolation.py
new file mode 100644
index 0000000000..627cf481dd
--- /dev/null
+++ b/scripts/tests/test_env_isolation.py
@@ -0,0 +1,43 @@
+# ***************************************************************************************************************************
+# * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an
+# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# * specific language governing permissions and limitations under the License.
+# ***************************************************************************************************************************
+"""A live run's JUNEAU_RUN_* variables must not reach the tests, or their sessions write into the real run's logs."""
+
+import os
+import subprocess
+import sys
+from pathlib import Path
+
+from conftest import LIVE_RUN_ENV
+
+_CHECKED = "JUNEAU_TEST_ENV_ISOLATION_INNER"
+
+
+def test_inner_sees_no_live_run_variables():
+ """Runs only in the subprocess below: the suite's fixtures must have dropped every live-run variable."""
+ if not os.environ.get(_CHECKED):
+ return
+ assert [n for n in LIVE_RUN_ENV if n in os.environ] == []
+
+
+def test_live_run_variables_do_not_reach_tests(tmp_path):
+ env = dict(os.environ)
+ env[_CHECKED] = "1"
+ live = tmp_path / "live"
+ for name in LIVE_RUN_ENV:
+ env[name] = str(live / (name + ".txt"))
+ result = subprocess.run(
+ [sys.executable, "-m", "pytest", "-q", "-p", "no:cacheprovider",
+ f"{Path(__file__)}::test_inner_sees_no_live_run_variables"],
+ env=env, capture_output=True, text=True, cwd=Path(__file__).parent)
+ assert result.returncode == 0, result.stdout + result.stderr
+ assert not live.exists()
diff --git a/scripts/tests/test_push.py b/scripts/tests/test_push.py
index aab1d72083..df3c40fe10 100644
--- a/scripts/tests/test_push.py
+++ b/scripts/tests/test_push.py
@@ -16,7 +16,7 @@
Tests for scripts/push.py: the end-of-run terminal bell (TestEndOfRunBell), the
refuse-on-unreviewed-changes gate, ported from the already-reviewed gate in apache/juneau's
scripts/push.py and sandbox-support-console's scripts/push.py, and the build gates (Compile,
-Java tests, Python tests, E2E) with their skip flags, --test-only and --dry-run. The review-gate tests cover:
+Java Tests, Python Tests, Browser Tests) with their skip flags, --test-only and --dry-run. The review-gate tests cover:
the toolkit's path readers as push.py uses them, and the refuse / no-op / proceed decision (the
three refusal shapes plus the allowed staged-only case). push.py is built on the juneau_scripts
toolkit; the tests drive it through main(argv, repo_dir) and read the exit code.
@@ -90,12 +90,12 @@ class _Push:
def push_module():
"""A fresh scripts/push.py module object (toolkit bound), loaded once per test."""
module = _load_push_module()
- # The Python tests step would launch this very suite from every main()-calling test (Juneau J0867: a runaway nested
+ # The Python Tests step would launch this very suite from every main()-calling test (Juneau J0867: a runaway nested
# pytest), so stub it; TestScriptTests drives the real one through _real_script_tests.
module.gate_calls = []
module._real_script_tests = module._script_tests
module._script_tests = lambda *_a, **_k: module.gate_calls.append("script-tests") or True
- # The Maven and E2E gates would build and boot the real app from every main()-calling test: stub them, recording the
+ # The Maven and Browser Tests gates would build and boot the real app from every main()-calling test: stub them, recording the
# order they ran in. The real ones stay reachable as _real_compile, _real_maven_tests and _real_e2e.
module._real_compile, module._compile = module._compile, lambda *_a, **_k: module.gate_calls.append("compile")
module._real_maven_tests, module._maven_tests = (
@@ -634,6 +634,16 @@ class TestCondensedConsole:
assert push_module.main(["--dry-run", "msg"], repo_dir=repo) == 0
assert "juneau_run" not in sys.modules
+ def test_the_header_is_project_and_branch_and_the_message_is_its_own_line(self, push_module, repo, remote_repo,
+ capfd):
+ (repo / "README.md").write_text("new content\n", encoding="utf-8")
+ _run(repo, "git", "add", "README.md")
+ assert push_module.main(["--console", "condensed", "msg"], repo_dir=repo) == 0
+ out = capfd.readouterr().out
+ assert "Push: JRM ยท " in out and "Commit Message: msg" in out
+ assert " ยท msg" not in out
+ assert "Overall โ
Done in" in out
+
# ---------------------------------------------------------------------------------------
# The upstream gate: a branch behind its upstream is refused before the commit.
@@ -805,7 +815,7 @@ class TestPushTracked:
# ---------------------------------------------------------------------------------------
-# The Python tests step: the scripts/tests pytest suite, run before the commit.
+# The Python Tests step: the scripts/tests pytest suite, run before the commit.
# ---------------------------------------------------------------------------------------
class _Result:
def __init__(self, code=0, out=""):
@@ -893,7 +903,7 @@ class TestScriptTests:
monkeypatch.setattr(push_module.module, "script_tests_launcher", lambda: None)
assert push_module._real_script_tests(_FakeSession(), tmp_path, False) is True
assert not cmds
- assert "Skipping Python tests" in capsys.readouterr().out
+ assert "Skipping Python Tests" in capsys.readouterr().out
assert rows["script-tests"] # the row is opened and skipped, not left pending
def test_passing_suite_runs_in_the_repo_and_lets_the_push_continue(self, push_module, cmds, tmp_path):
@@ -976,7 +986,7 @@ class TestLocation:
# ---------------------------------------------------------------------------------------
-# The Compile and Java tests rows: mvn clean test-compile, then mvn verify; never install.
+# The Compile and Java Tests rows: mvn clean test-compile, then mvn verify; never install.
# ---------------------------------------------------------------------------------------
class TestBuildGates:
def test_compile_runs_clean_test_compile_in_the_repo(self, push_module, cmds, tmp_path):
@@ -990,7 +1000,7 @@ class TestBuildGates:
push_module._real_maven_tests(_FakeSession(), tmp_path, False)
cmd, kwargs = cmds[0]
assert cmd == ["mvn", "verify"]
- assert (kwargs["cwd"], kwargs["id"], kwargs["title"]) == (str(tmp_path), "tests", "Java tests")
+ assert (kwargs["cwd"], kwargs["id"], kwargs["title"]) == (str(tmp_path), "tests", "Java Tests")
def test_neither_row_ever_installs(self, push_module, cmds, tmp_path):
push_module._real_compile(_FakeSession(), tmp_path, False)
@@ -998,7 +1008,7 @@ class TestBuildGates:
assert len(cmds) == 2
assert all("install" not in cmd for cmd, _ in cmds)
- @pytest.mark.parametrize("gate, title", [("_real_compile", "Compile"), ("_real_maven_tests", "Java tests")])
+ @pytest.mark.parametrize("gate, title", [("_real_compile", "Compile"), ("_real_maven_tests", "Java Tests")])
def test_a_failure_refuses_the_push(self, push_module, cmds, tmp_path, gate, title):
cmds.code = 1
s = _FakeSession()
@@ -1008,7 +1018,7 @@ class TestBuildGates:
assert f"{title} failed" in s.failed
assert "nothing committed" in s.failed
- @pytest.mark.parametrize("gate, title", [("_real_compile", "Compile"), ("_real_maven_tests", "Java tests")])
+ @pytest.mark.parametrize("gate, title", [("_real_compile", "Compile"), ("_real_maven_tests", "Java Tests")])
def test_dry_run_reports_would_run_and_runs_nothing(self, push_module, cmds, tmp_path, capsys, gate, title):
getattr(push_module, gate)(_FakeSession(), tmp_path, True)
assert not cmds
@@ -1058,7 +1068,7 @@ class TestBuildGates:
# ---------------------------------------------------------------------------------------
-# The E2E row: the full Playwright suite in e2e/, through the playwright parser.
+# The Browser Tests row: the full Playwright suite in e2e/, through the playwright parser.
# ---------------------------------------------------------------------------------------
class TestE2EGate:
@pytest.fixture
@@ -1073,7 +1083,7 @@ class TestE2EGate:
push_module._real_e2e(_FakeSession(), tmp_path, False)
cmd, kwargs = cmds[0]
assert cmd == ["npx", "playwright", "test"]
- assert (kwargs["tool"], kwargs["id"], kwargs["title"]) == ("playwright", "e2e", "E2E")
+ assert (kwargs["tool"], kwargs["id"], kwargs["title"]) == ("playwright", "e2e", "Browser Tests")
assert kwargs["cwd"] == str(tmp_path / "e2e")
assert not any(a.startswith("--reporter") for a in cmd)
assert npx == [tmp_path / "e2e"]
@@ -1104,7 +1114,7 @@ class TestE2EGate:
s = _FakeSession()
with pytest.raises(SystemExit):
push_module._real_e2e(s, tmp_path, False)
- assert "E2E failed" in s.failed
+ assert "Browser Tests failed" in s.failed
assert "nothing committed" in s.failed
def test_a_failed_install_fails_the_gate_before_the_suite(self, push_module, cmds, monkeypatch, tmp_path):
@@ -1113,7 +1123,7 @@ class TestE2EGate:
s = _FakeSession()
with pytest.raises(SystemExit):
push_module._real_e2e(s, tmp_path, False)
- assert "E2E setup failed" in s.failed
+ assert "Browser Tests setup failed" in s.failed
assert not cmds
def test_missing_npx_skips_with_a_warning_and_lets_the_push_continue(self, push_module, cmds, monkeypatch,
@@ -1123,13 +1133,13 @@ class TestE2EGate:
push_module._real_e2e(s, tmp_path, False)
assert s.failed is None
assert not cmds
- assert "Skipping E2E: npx not found" in capsys.readouterr().out
+ assert "Skipping Browser Tests: npx not found" in capsys.readouterr().out
def test_dry_run_reports_would_run_and_runs_nothing(self, push_module, cmds, npx, tmp_path, capsys):
push_module._real_e2e(_FakeSession(), tmp_path, True)
assert not cmds
assert not npx
- assert "Would run E2E: npx playwright test" in capsys.readouterr().out
+ assert "Would run Browser Tests: npx playwright test" in capsys.readouterr().out
# ---------------------------------------------------------------------------------------
@@ -1160,8 +1170,8 @@ class TestPlannedSteps:
def test_the_labels(self, push_module):
assert dict(push_module.planned_steps(push_module.build_parser().parse_args(["msg"]))) == {
- "prechecks": "Prechecks", "compile": "Compile", "tests": "Java tests", "script-tests": "Python tests",
- "e2e": "E2E", "commit": "Commit", "push": "Push"}
+ "prechecks": "Prechecks", "compile": "Compile", "tests": "Java Tests", "script-tests": "Python Tests",
+ "e2e": "Browser Tests", "commit": "Commit", "push": "Push"}
@pytest.mark.parametrize("flag, removed", [("--skip-pytest", ["script-tests"]),
("--skip-tests", ["compile", "tests"]),
@@ -1192,9 +1202,9 @@ class TestGatesThroughMain:
assert push_module.gate_calls == []
@pytest.mark.parametrize("flag, expected, line", [
- ("--skip-pytest", ["compile", "tests", "e2e"], "Skipping Python tests (--skip-pytest)"),
- ("--skip-tests", ["script-tests", "e2e"], "Skipping Compile and Java tests (--skip-tests)"),
- ("--skip-e2e", ["compile", "tests", "script-tests"], "Skipping E2E (--skip-e2e)"),
+ ("--skip-pytest", ["compile", "tests", "e2e"], "Skipping Python Tests (--skip-pytest)"),
+ ("--skip-tests", ["script-tests", "e2e"], "Skipping Compile and Java Tests (--skip-tests)"),
+ ("--skip-e2e", ["compile", "tests", "script-tests"], "Skipping Browser Tests (--skip-e2e)"),
])
def test_each_skip_flag_skips_exactly_its_gates_and_says_so(self, push_module, repo, capsys, flag, expected,
line):
diff --git a/scripts/tests/test_script_docs.py b/scripts/tests/test_script_docs.py
new file mode 100644
index 0000000000..c5e22af300
--- /dev/null
+++ b/scripts/tests/test_script_docs.py
@@ -0,0 +1,428 @@
+# ***************************************************************************************************************************
+# * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an
+# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# * specific language governing permissions and limitations under the License.
+# ***************************************************************************************************************************
+"""
+Keeps scripts/ documented: every entry point has a <name>.md beside it, every script doc has its script, every script
+doc has the seven fixed headings, and every flag a script declares appears in its doc's Options section.
+
+Read-only and hermetic: it reads files, never runs a script and writes nothing. Flags are found by parsing the source
+(ast for Python, the option `case` for shell), never by running --help. Exclusions and the library packages that need a
+README.md are listed in scripts/script-docs-exclude.txt.
+
+One copy per repository, the same shape in all of them; only SKIP_DIRS may differ.
+"""
+import ast
+import fnmatch
+import re
+import warnings
+from pathlib import Path
+
+import pytest
+
+SCRIPTS = Path(__file__).resolve().parents[1]
+SKIP_DIRS = ("tests", "vendor", "node_modules", "__pycache__", "fixtures")
+EXCLUDE_FILE = "script-docs-exclude.txt"
+HEADINGS = ["When to use", "Usage", "Options", "What it does", "Output and exit codes", "Gotchas", "See also"]
+SHEBANG = re.compile(r"^#!.*\b(sh|bash|zsh|python3?)\b")
+H1_SCRIPT = re.compile(r"^# (\S+\.(?:py|sh))\s*$")
+
+
+# ---------------------------------------------------------------------------------------------------------------------
+# Finding files
+
+def candidate_files(root):
+ """Every file under root outside the skipped and dot-directories, as paths."""
+ out = []
+ for p in sorted(root.rglob("*")):
+ rel = p.relative_to(root)
+ if p.is_file() and not any(part in SKIP_DIRS or part.startswith(".") for part in rel.parts[:-1]):
+ out.append(p)
+ return out
+
+
+def _first_line(path):
+ try:
+ with open(path, "rb") as f:
+ return f.readline(200).decode("utf-8", "replace")
+ except OSError:
+ return ""
+
+
+def has_main_guard(source):
+ """True when the source has a top-level `if __name__ == "__main__":` (either operand order)."""
+ try:
+ tree = ast.parse(source)
+ except SyntaxError:
+ return False
+ for node in tree.body:
+ if isinstance(node, ast.If) and isinstance(node.test, ast.Compare) and len(node.test.ops) == 1:
+ sides = [node.test.left, *node.test.comparators]
+ names = [s.id for s in sides if isinstance(s, ast.Name)]
+ consts = [s.value for s in sides if isinstance(s, ast.Constant)]
+ if names == ["__name__"] and consts == ["__main__"]:
+ return True
+ return False
+
+
+def script_kind(path):
+ """'py', 'sh' or None: whether path is an entry point (a runnable script)."""
+ if path.suffix == ".sh":
+ return "sh"
+ if path.suffix == ".py":
+ return "py" if has_main_guard(path.read_text(encoding="utf-8", errors="replace")) else None
+ if path.suffix == "":
+ m = SHEBANG.match(_first_line(path))
+ if m:
+ return "py" if m.group(1).startswith("python") else "sh"
+ return None
+
+
+def doc_for(path):
+ """The doc beside a script: push.py -> push.md, pd-close -> pd-close.md."""
+ return path.with_suffix(".md") if path.suffix else path.with_name(path.name + ".md")
+
+
+# ---------------------------------------------------------------------------------------------------------------------
+# The exclude file
+
+class Excludes:
+ def __init__(self):
+ self.paths = [] # (pattern, reason)
+ self.flags = [] # (pattern, reason)
+ self.packages = [] # directory names relative to scripts/
+ self.errors = []
+
+
+def parse_excludes(text):
+ ex = Excludes()
+ for n, raw in enumerate(text.splitlines(), 1):
+ line = raw.strip()
+ if not line or line.startswith("#"):
+ continue
+ m = re.match(r"^(.*?)\s+#\s*(.*)$", line)
+ body, reason = m.groups() if m else (line, "")
+ body, reason = body.strip(), reason.strip()
+ if body.startswith("packages:"):
+ ex.packages += [p.strip() for p in body[len("packages:"):].split(",") if p.strip()]
+ elif not reason:
+ ex.errors.append(f"{EXCLUDE_FILE}:{n}: '{body}' has no '# reason'")
+ elif body.startswith("flags:"):
+ ex.flags.append((body[len("flags:"):].strip(), reason))
+ else:
+ ex.paths.append((body, reason))
+ return ex
+
+
+def load_excludes(root):
+ path = root / EXCLUDE_FILE
+ return parse_excludes(path.read_text(encoding="utf-8")) if path.exists() else Excludes()
+
+
+def matches(rel, pattern):
+ return fnmatch.fnmatchcase(rel, pattern)
+
+
+# ---------------------------------------------------------------------------------------------------------------------
+# Finding flags without running anything
+
+def py_flags(source):
+ """(flags, subcommands, uses_argparse) from the source: string constants starting with '-' in add_argument calls, the
+ names given to add_parser, and whether argparse is imported at all."""
+ tree = ast.parse(source)
+ uses = False
+ flags, subs = set(), set()
+ for node in ast.walk(tree):
+ if isinstance(node, ast.Import) and any(a.name.split(".")[0] == "argparse" for a in node.names):
+ uses = True
+ elif isinstance(node, ast.ImportFrom) and (node.module or "").split(".")[0] == "argparse":
+ uses = True
+ elif isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute):
+ strings = [a.value for a in node.args if isinstance(a, ast.Constant) and isinstance(a.value, str)]
+ if node.func.attr == "add_argument":
+ flags.update(s for s in strings if s.startswith("-") and s not in ("-h", "--help"))
+ elif node.func.attr == "add_parser" and strings:
+ subs.add(strings[0])
+ return flags, subs, uses
+
+
+def sh_flags(text):
+ """The flags a shell script accepts: --long and -s alternatives of `case` patterns whose subject is an option
+ variable, plus tokens starting with '-' on Usage: lines of the leading comment block."""
+ flags = set()
+ in_case = False
+ for line in text.splitlines():
+ if re.match(r"\s*case\s+\"?\$\{?(1|opt|arg|a)\}?\"?\s+in\b", line):
+ in_case = True
+ continue
+ if in_case and re.match(r"\s*esac\b", line):
+ in_case = False
+ continue
+ if in_case:
+ m = re.match(r"\s*([^)#]+)\)", line)
+ if m:
+ flags.update(t.strip() for t in m.group(1).split("|") if re.fullmatch(r"-{1,2}[A-Za-z][\w-]*", t.strip()))
+ for line in text.splitlines():
+ if line.strip() and not line.startswith("#"):
+ if not line.startswith("#!"):
+ break
+ if re.match(r"#\s*Usage:", line):
+ flags.update(re.findall(r"(?<![\w-])(--?[A-Za-z][\w-]*)", line))
+ return flags - {"-h", "--help"}
+
+
+# ---------------------------------------------------------------------------------------------------------------------
+# Reading docs
+
+def fenced_free_lines(text):
+ """The lines of a markdown text outside code fences."""
+ out, fenced = [], False
+ for line in text.splitlines():
+ if line.lstrip().startswith("```"):
+ fenced = not fenced
+ continue
+ if not fenced:
+ out.append(line)
+ return out
+
+
+def h2_headings(text):
+ return [l[3:].strip() for l in fenced_free_lines(text) if l.startswith("## ")]
+
+
+def section(text, name):
+ lines, grab, out = fenced_free_lines(text), False, []
+ for l in lines:
+ if l.startswith("## "):
+ grab = l[3:].strip() == name
+ elif grab:
+ out.append(l)
+ return "\n".join(out)
+
+
+def backticked(text):
+ """Every token inside backticks, with multi-word spans also split on whitespace and commas."""
+ toks = set()
+ for span in re.findall(r"`([^`]+)`", text):
+ toks.add(span)
+ toks.update(t for t in re.split(r"[\s,]+", span) if t)
+ return toks
+
+
+def is_subsequence(want, have):
+ it = iter(have)
+ return all(any(w == h for h in it) for w in want)
+
+
+def has_paragraph(text):
+ lines = fenced_free_lines(text)
+ return any(l.strip() and not l.startswith("#") for l in lines)
+
+
+# ---------------------------------------------------------------------------------------------------------------------
+# The checks
+
+def collect(root):
+ """Run every check over root (a scripts/ directory). Returns {check name: [failure text]}."""
+ ex = load_excludes(root)
+ files = candidate_files(root)
+ rels = {p: p.relative_to(root).as_posix() for p in files}
+ f = {"exclude-file": list(ex.errors), "no-doc": [], "no-script": [], "headings": [], "flags": [], "packages": []}
+ for kind, pats in (("path", ex.paths), ("flags", ex.flags)):
+ for pat, _ in pats:
+ if not any(matches(r, pat) for r in rels.values()):
+ f["exclude-file"].append(f"stale {kind} exclude matches no file: {pat}")
+ entries = {}
+ for p in files:
+ k = script_kind(p)
+ if k and not any(matches(rels[p], pat) for pat, _ in ex.paths):
+ entries[p] = k
+ for p in entries:
+ if not doc_for(p).exists():
+ f["no-doc"].append(f"{rels[p]}: no {doc_for(p).name} beside it")
+ docs = [p for p in files if p.suffix == ".md"]
+ for d in docs:
+ text = d.read_text(encoding="utf-8")
+ first = text.splitlines()[0] if text.strip() else ""
+ m = H1_SCRIPT.match(first)
+ if m:
+ if not (d.parent / m.group(1)).exists():
+ f["no-script"].append(f"{rels[d]}: H1 names {m.group(1)}, which is not beside it")
+ have = h2_headings(text)
+ if not is_subsequence(HEADINGS, have):
+ f["headings"].append(f"{rels[d]}: needs, in order, the headings {HEADINGS}; has {have}")
+ elif d.name == "README.md" and d.parent != root and not any(c.suffix == ".py" for c in d.parent.iterdir()):
+ f["no-script"].append(f"{rels[d]}: a package README needs at least one .py module in its directory")
+ for p, k in entries.items():
+ doc = doc_for(p)
+ if not doc.exists():
+ continue
+ text = doc.read_text(encoding="utf-8")
+ if not H1_SCRIPT.match(text.splitlines()[0] if text.strip() else "") and not is_subsequence(HEADINGS, h2_headings(text)):
+ f["headings"].append(f"{rels[doc]}: needs, in order, the headings {HEADINGS}")
+ if any(matches(rels[p], pat) for pat, _ in ex.flags):
+ continue
+ src = p.read_text(encoding="utf-8", errors="replace")
+ if k == "py":
+ flags, subs, uses = py_flags(src)
+ if uses and not flags and not subs:
+ f["flags"].append(f"FLAGS-UNVERIFIABLE {rels[p]}: argparse is used but no add_argument was found; "
+ f"add a 'flags: {rels[p]} # reason' line to {EXCLUDE_FILE}")
+ continue
+ else:
+ flags, subs = sh_flags(src), set()
+ options = backticked(section(text, "Options"))
+ missing = sorted(x for x in (flags | subs) if x not in options)
+ if missing:
+ f["flags"].append(f"{rels[doc]}: Options does not list {missing}")
+ elif k == "py":
+ shown = {t for t in re.findall(r"^\|\s*`(-[^`]+)`", section(text, "Options"), flags=re.M)}
+ stale = sorted(t for t in shown if t not in flags and t not in ("-h", "--help") and (flags or subs))
+ if stale:
+ warnings.warn(f"{rels[doc]}: Options lists {stale}, which the code does not declare")
+ for pkg in ex.packages:
+ if not (root / pkg / "README.md").exists():
+ f["packages"].append(f"{pkg}/README.md: listed under 'packages:' but missing")
+ elif not has_paragraph((root / pkg / "README.md").read_text(encoding="utf-8")):
+ f["packages"].append(f"{pkg}/README.md: needs an H1 and at least one paragraph")
+ return f
+
+
+# ---------------------------------------------------------------------------------------------------------------------
+# The real repository
+
+@pytest.fixture(scope="module")
+def failures():
+ return collect(SCRIPTS)
+
+
+@pytest.mark.parametrize("check", ["exclude-file", "no-doc", "no-script", "headings", "flags", "packages"])
+def test_scripts_are_documented(failures, check):
+ assert not failures[check], f"[{check}]\n " + "\n ".join(failures[check])
+
+
+# ---------------------------------------------------------------------------------------------------------------------
+# The checker itself, against small inline fixtures
+
+GOOD_DOC = "# {name}\n\nWhat it is.\n\n" + "".join(f"## {h}\n\n{{body}}\n\n" for h in HEADINGS)
+
+
+def good_doc(name, options="None."):
+ return GOOD_DOC.replace("{name}", name).replace("{body}", "x").replace("## Options\n\nx", "## Options\n\n" + options)
+
+
+PY_ARGS = ('import argparse\np = argparse.ArgumentParser()\ng = p.add_mutually_exclusive_group()\n'
+ 'g.add_argument("--sonar", "--sonarqube")\nsub = p.add_subparsers()\nsub.add_parser("run").add_argument("-v")\n'
+ 'p.add_argument("message")\nif __name__ == "__main__":\n pass\n')
+
+
+def tree(tmp_path, files):
+ for name, text in files.items():
+ p = tmp_path / name
+ p.parent.mkdir(parents=True, exist_ok=True)
+ p.write_text(text)
+ return tmp_path
+
+
+def test_py_flags_collects_aliases_groups_and_subparsers():
+ flags, subs, uses = py_flags(PY_ARGS)
+ assert flags == {"--sonar", "--sonarqube", "-v"}
+ assert subs == {"run"}
+ assert uses
+
+
+def test_py_flags_without_argparse():
+ assert py_flags("import sys\nprint(sys.argv)\n") == (set(), set(), False)
+
+
+def test_main_guard_either_order_and_quote():
+ assert has_main_guard('if __name__ == "__main__":\n pass\n')
+ assert has_main_guard("if '__main__' == __name__:\n pass\n")
+ assert not has_main_guard("def f():\n if __name__ == '__main__':\n pass\n")
+
+
+def test_sh_flags_reads_option_case_and_usage_but_not_os_case():
+ src = ('#!/bin/bash\n# Usage: tool.sh [--fast] [-n]\nwhile [ $# -gt 0 ]; do\n case "$1" in\n --dry-run|-d) x=1;;\n'
+ ' *) ;;\n esac\ndone\ncase "$(uname)" in\n Darwin) y=1;;\nesac\n')
+ assert sh_flags(src) == {"--fast", "-n", "--dry-run", "-d"}
+
+
+def test_sh_flags_none():
+ assert sh_flags("#!/bin/sh\necho hi\n") == set()
+
+
+def test_parse_excludes_requires_reason_and_reads_prefixes():
+ ex = parse_excludes("# c\n\na/*.sh # fixture\nflags: b.py # dynamic\npackages: p, q\nbad.py\n")
+ assert ex.paths == [("a/*.sh", "fixture")]
+ assert ex.flags == [("b.py", "dynamic")]
+ assert ex.packages == ["p", "q"]
+ assert len(ex.errors) == 1 and "bad.py" in ex.errors[0]
+
+
+def test_clean_tree_passes(tmp_path):
+ t = tree(tmp_path, {"a.py": PY_ARGS, "a.md": good_doc("a.py", "`--sonar`, `--sonarqube`, `-v`, `run`"),
+ "pkg/__init__.py": "", "pkg/README.md": "# pkg\n\nA library.\n", EXCLUDE_FILE: "packages: pkg\n"})
+ assert not any(collect(t).values())
+
+
+def test_missing_doc_is_reported(tmp_path):
+ t = tree(tmp_path, {"a.py": PY_ARGS})
+ assert collect(t)["no-doc"]
+
+
+def test_stray_doc_is_reported(tmp_path):
+ t = tree(tmp_path, {"gone.md": good_doc("gone.py")})
+ assert collect(t)["no-script"]
+
+
+def test_missing_heading_is_reported(tmp_path):
+ t = tree(tmp_path, {"a.py": "if __name__ == '__main__':\n pass\n",
+ "a.md": good_doc("a.py").replace("## Gotchas\n\nx\n\n", "")})
+ assert collect(t)["headings"]
+
+
+def test_undocumented_flag_is_reported(tmp_path):
+ t = tree(tmp_path, {"a.py": PY_ARGS, "a.md": good_doc("a.py", "`--sonar`, `-v`, `run`")})
+ assert any("--sonarqube" in m for m in collect(t)["flags"])
+
+
+def test_unverifiable_flags_need_an_exclude(tmp_path):
+ src = "import argparse\nfor n in ('--a',):\n pass\nif __name__ == '__main__':\n pass\n"
+ t = tree(tmp_path, {"a.py": src, "a.md": good_doc("a.py")})
+ assert any(m.startswith("FLAGS-UNVERIFIABLE") for m in collect(t)["flags"])
+ (t / EXCLUDE_FILE).write_text("flags: a.py # flags added by a loop\n")
+ assert not collect(t)["flags"]
+
+
+def test_stale_exclude_is_reported(tmp_path):
+ t = tree(tmp_path, {EXCLUDE_FILE: "nothing/*.py # matches nothing\n"})
+ assert collect(t)["exclude-file"]
+
+
+def test_excluded_script_needs_no_doc(tmp_path):
+ t = tree(tmp_path, {"a.py": PY_ARGS, EXCLUDE_FILE: "a.py # not documented here\n"})
+ assert not collect(t)["no-doc"]
+
+
+def test_missing_package_readme_is_reported(tmp_path):
+ t = tree(tmp_path, {"pkg/__init__.py": "", EXCLUDE_FILE: "packages: pkg\n"})
+ assert collect(t)["packages"]
+
+
+def test_extensionless_shebang_script_is_an_entry_point(tmp_path):
+ t = tree(tmp_path, {"close": "#!/usr/bin/env bash\necho hi\n"})
+ assert collect(t)["no-doc"]
+
+
+def test_reverse_check_only_warns(tmp_path):
+ t = tree(tmp_path, {"a.py": PY_ARGS, "a.md": good_doc("a.py", "`--sonar`, `--sonarqube`, `-v`, `run`\n\n| `--old` | x |")})
+ with pytest.warns(UserWarning, match="--old"):
+ assert not collect(t)["flags"]
diff --git a/src/main/java/org/apache/juneau/releng/AppConfiguration.java b/src/main/java/org/apache/juneau/releng/AppConfiguration.java
index d09582385e..6f2e26defc 100644
--- a/src/main/java/org/apache/juneau/releng/AppConfiguration.java
+++ b/src/main/java/org/apache/juneau/releng/AppConfiguration.java
@@ -321,6 +321,7 @@ public class AppConfiguration {
/**
* Publishes a {@code run} event when any {@code run.json} changes, CLI runs included.
*/
+ @SuppressWarnings("resource") // Spring owns the bean and closes it via destroyMethod.
@Bean(initMethod = "start", destroyMethod = "close")
public RunFileWatcher runFileWatcher(@Value("${rm.home}") String home, EventBroadcaster runEvents) {
return new RunFileWatcher(Path.of(home), runEvents);
diff --git a/src/main/java/org/apache/juneau/releng/rest/ReleaseRunRest.java b/src/main/java/org/apache/juneau/releng/rest/ReleaseRunRest.java
index b734447557..aadc0151d5 100644
--- a/src/main/java/org/apache/juneau/releng/rest/ReleaseRunRest.java
+++ b/src/main/java/org/apache/juneau/releng/rest/ReleaseRunRest.java
@@ -236,6 +236,7 @@ public class ReleaseRunRest extends BasicRestResource implements TerminalMixin {
* An attempt's terminal. The id is {@code attempt-<attempt dir name>}: the live process's own source while it
* runs, then the dir's {@code terminal.log}, then (for a CLI attempt run without one) its {@code full.log}.
*/
+ @SuppressWarnings("resource") // The terminal source is handed to the framework, which closes it when the stream ends.
@Override /* TerminalMixin */
public Optional<TerminalSource> terminalSource(String id, RestRequest req) {
var live = processes.live();
diff --git a/src/main/java/org/apache/juneau/releng/run/ReleaseProcessService.java b/src/main/java/org/apache/juneau/releng/run/ReleaseProcessService.java
index 784519321a..8994957484 100644
--- a/src/main/java/org/apache/juneau/releng/run/ReleaseProcessService.java
+++ b/src/main/java/org/apache/juneau/releng/run/ReleaseProcessService.java
@@ -130,6 +130,7 @@ public class ReleaseProcessService {
}
/** Cancels the live process, if any: stdin closes, the runner SIGTERMs, release.py records {@code cancelled}. */
+ @SuppressWarnings("resource") // The live process is owned by the Live record and closed by cancel()/shutdown()/awaitExit, not by this method.
public void cancel() {
Live l;
synchronized (this) {
@@ -159,6 +160,7 @@ public class ReleaseProcessService {
}
/** Spring destroy method (run from the JVM shutdown hook): marks the live attempt {@code jrm-stopped}, then closes. */
+ @SuppressWarnings("resource") // The live process is owned by the Live record and closed by cancel()/shutdown()/awaitExit, not by this method.
public void shutdown() {
Live l;
synchronized (this) {
@@ -174,6 +176,7 @@ public class ReleaseProcessService {
l.process().close();
}
+ @SuppressWarnings("resource") // The live process is owned by the Live record and closed by cancel()/shutdown()/awaitExit, not by this method.
private Live launch(String command, List<String> extra, boolean dryRun, Map<String, String> autoAnswers) {
if (live != null)
throw new BusyException(live.attemptId());
@@ -207,6 +210,7 @@ public class ReleaseProcessService {
}
}
+ @SuppressWarnings("resource") // Ownership of the started process transfers to the caller.
private TerminalProcess startProcess(List<String> cmd, Path dir, PromptBroker broker, String attemptId) throws IOException {
try {
return TerminalProcess.start("generic", cmd, 120, 40, dir, broker);
@@ -232,6 +236,7 @@ public class ReleaseProcessService {
}
}
+ @SuppressWarnings("resource") // The live process is owned by the Live record and closed by cancel()/shutdown()/awaitExit, not by this method.
private void awaitExit(Live l, Map<String, Object> sidecar) {
Integer exit = null;
try {
diff --git a/src/test/java/org/apache/juneau/releng/log/EventBroadcasterTest.java b/src/test/java/org/apache/juneau/releng/log/EventBroadcasterTest.java
index 492915cd78..2ba2c949ac 100644
--- a/src/test/java/org/apache/juneau/releng/log/EventBroadcasterTest.java
+++ b/src/test/java/org/apache/juneau/releng/log/EventBroadcasterTest.java
@@ -36,6 +36,7 @@ class EventBroadcasterTest {
assertEquals("{\"type\":\"run\",\"version\":\"9.2.1\"}", got.get(0).replace(" ", ""));
}
+ @SuppressWarnings("resource") // Subscriptions to the in-memory broadcaster hold no external resource; they die with the test.
@Test
void a03_throwingSinkDoesNotStopOthers() {
var bc = new EventBroadcaster();
@@ -46,6 +47,7 @@ class EventBroadcasterTest {
assertEquals(1, got.size());
}
+ @SuppressWarnings("resource") // Subscriptions to the in-memory broadcaster hold no external resource; they die with the test.
@Test
void a02_payloadTypeFieldNeverOverridesTheEventType() {
var bc = new EventBroadcaster();
diff --git a/src/test/java/org/apache/juneau/releng/log/RunEventsSseServletTest.java b/src/test/java/org/apache/juneau/releng/log/RunEventsSseServletTest.java
index 4e16a8b889..4f6416c18c 100644
--- a/src/test/java/org/apache/juneau/releng/log/RunEventsSseServletTest.java
+++ b/src/test/java/org/apache/juneau/releng/log/RunEventsSseServletTest.java
@@ -31,6 +31,7 @@ import jakarta.servlet.http.HttpServletResponse;
class RunEventsSseServletTest {
+ @SuppressWarnings("resource") // Subscriptions to the in-memory broadcaster hold no external resource; they die with the test.
@Test
void a01_streamsPublishedEventsAsSseData() throws Exception {
var subscribed = new CountDownLatch(1);
diff --git a/src/test/java/org/apache/juneau/releng/rest/ReleaseRunRestTest.java b/src/test/java/org/apache/juneau/releng/rest/ReleaseRunRestTest.java
index d89c1e295d..db3d8ba07c 100644
--- a/src/test/java/org/apache/juneau/releng/rest/ReleaseRunRestTest.java
+++ b/src/test/java/org/apache/juneau/releng/rest/ReleaseRunRestTest.java
@@ -59,6 +59,7 @@ class ReleaseRunRestTest {
static final String HELLO = "{\"v\":1,\"type\":\"hello\",\"pid\":4242}";
static final String PROMPT = "{\"v\":1,\"type\":\"prompt\",\"id\":\"p1\",\"kind\":\"text\",\"key\":\"release.branch\",\"message\":\"m\"}";
+ @SuppressWarnings("resource") // Mocked process and source hold no real resource; ownership passes to the returned Live.
static ReleaseProcessService.Live live(String attemptId, boolean hello, boolean prompt) {
var broker = new PromptBroker(k -> Optional.empty(), new EventBroadcaster(), java.util.Map.of());
broker.attach(line -> true);
@@ -72,6 +73,7 @@ class ReleaseRunRestTest {
return new ReleaseProcessService.Live(attemptId, java.nio.file.Path.of("x"), "start", true, proc, broker);
}
+ @SuppressWarnings("resource") // Mocked process and source hold no real resource.
@Test
void a01_terminalSourceLiveThenFile(@TempDir Path home) throws Exception {
var dir = Files.createDirectories(home.resolve("attempts/20261020-100000-001"));
diff --git a/src/test/java/org/apache/juneau/releng/run/PromptBrokerTest.java b/src/test/java/org/apache/juneau/releng/run/PromptBrokerTest.java
index ff860af821..814b156332 100644
--- a/src/test/java/org/apache/juneau/releng/run/PromptBrokerTest.java
+++ b/src/test/java/org/apache/juneau/releng/run/PromptBrokerTest.java
@@ -50,6 +50,7 @@ class PromptBrokerTest {
final List<String> written = new ArrayList<>();
final List<String> events = new ArrayList<>();
+ @SuppressWarnings("resource") // Subscriptions to the in-memory broadcaster hold no external resource; they die with the test.
PromptBroker broker(Map<String, String> stored, Map<String, String> auto) {
var bus = new EventBroadcaster();
bus.subscribe(events::add);
@@ -97,6 +98,7 @@ class PromptBrokerTest {
assertTrue(b.pending().isPresent(), "the real prompt stays pending");
}
+ @SuppressWarnings("resource") // Subscriptions to the in-memory broadcaster hold no external resource; they die with the test.
@Test
void a04_secretNeverLogged() {
var records = new ArrayList<LogRecord>();
@@ -151,6 +153,7 @@ class PromptBrokerTest {
assertTrue(b.pending().isEmpty());
}
+ @SuppressWarnings("resource") // Subscriptions to the in-memory broadcaster hold no external resource; they die with the test.
@Test
void a08_throwingStoredLookupFallsThroughToDialogWithoutLeaking() {
var bus = new EventBroadcaster();
diff --git a/src/test/java/org/apache/juneau/releng/run/ReleaseProcessServiceTest.java b/src/test/java/org/apache/juneau/releng/run/ReleaseProcessServiceTest.java
index f86fade2a9..dbb57d486a 100644
--- a/src/test/java/org/apache/juneau/releng/run/ReleaseProcessServiceTest.java
+++ b/src/test/java/org/apache/juneau/releng/run/ReleaseProcessServiceTest.java
@@ -45,6 +45,7 @@ class ReleaseProcessServiceTest {
return svc;
}
+ @SuppressWarnings("resource") // Processes started by the service are closed by the test's cleanup (svc.cancel()).
@Test
void a01_startSpawnsWithAttemptDir(@TempDir Path home) throws Exception {
var live = service(home, "{\"sleep\":0,\"exit\":0}").start(true);
@@ -58,6 +59,7 @@ class ReleaseProcessServiceTest {
assertEquals("attempt-" + live.dir().getFileName(), live.attemptId());
}
+ @SuppressWarnings("resource") // Processes started by the service are closed by the test's cleanup (svc.cancel()).
@Test
void a02_secondStart409(@TempDir Path home) throws Exception {
var s = service(home, "{\"sleep\":30}");
@@ -66,6 +68,7 @@ class ReleaseProcessServiceTest {
assertTrue(e.getMessage().contains("already running"), e.getMessage());
}
+ @SuppressWarnings("resource") // Processes started by the service are closed by the test's cleanup (svc.cancel()).
@Test
void a03_cancelRecordsCancelled(@TempDir Path home) throws Exception {
var s = service(home, "{\"sleep\":30}");
@@ -79,6 +82,7 @@ class ReleaseProcessServiceTest {
assertTrue(readRun(runJson).contains("\"cancelled\""), readRun(runJson));
}
+ @SuppressWarnings("resource") // Processes started by the service are closed by the test's cleanup (svc.cancel()).
@Test
void a04_shutdownHookMarksJrmStopped(@TempDir Path home) throws Exception {
var s = service(home, "{\"sleep\":30}");
@@ -91,6 +95,7 @@ class ReleaseProcessServiceTest {
assertEquals("interrupted (JRM stopped)", outcome);
}
+ @SuppressWarnings("resource") // Processes started by the service are closed by the test's cleanup (svc.cancel()).
@Test
void a05_dropRcAnswersTheTypedConfirm(@TempDir Path home) throws Exception {
var s = service(home, "{\"prompts\":[{\"kind\":\"confirm\",\"key\":\"confirm.revert\",\"message\":\"Type RC2\",\"expect\":\"RC2\"}]}");
diff --git a/src/test/java/org/apache/juneau/releng/run/RunFileWatcherTest.java b/src/test/java/org/apache/juneau/releng/run/RunFileWatcherTest.java
index ba91c7eb7d..32ae2cf3de 100644
--- a/src/test/java/org/apache/juneau/releng/run/RunFileWatcherTest.java
+++ b/src/test/java/org/apache/juneau/releng/run/RunFileWatcherTest.java
@@ -27,6 +27,7 @@ import org.junit.jupiter.api.io.TempDir;
class RunFileWatcherTest {
+ @SuppressWarnings("resource") // The watcher is never start()ed, so it holds no scheduler; subscriptions are in-memory.
@Test
void a01_publishesRunOnMtimeChangeOnly(@TempDir Path home) throws Exception {
var bus = new EventBroadcaster();