You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@juneau.apache.org by ja...@apache.org on 2026/09/13 13:16:54 UTC
(juneau) 16/29: Add atomic TODO id claiming, HOLD/NEED_INPUT lifecycle and declared-counts checks, and a hermetic pytest harness for the tracker scripts.
This is an automated email from the ASF dual-hosted git repository.
jamesbognar pushed a commit to branch juneau-release-manager
in repository https://gitbox.apache.org/repos/asf/juneau.git
commit f9765045b406c9b99080d3f8814cc8996027fdc3
Author: James Bognar <ja...@apache.org>
AuthorDate: Wed Aug 26 10:05:34 2026 -0400
Add atomic TODO id claiming, HOLD/NEED_INPUT lifecycle and declared-counts checks, and a hermetic pytest harness for the tracker scripts.
todo-next-id.py had no atomic claim: it read the tracker, computed the next free id, and
printed it, so concurrent callers could race and be handed the same id -- the exact defect
that duplicated TODO-C0021 and TODO-C0022 on 2026-08-25. It now claims the id by creating a
placeholder ("TODO-<id>-CLAIMED.md") with O_CREAT | O_EXCL before printing anything, so a
winner is decided by the filesystem, and adds --finalize to atomically rename the claim
into place instead of the old rename-it-yourself convention. todo-status-audit.py adds the
matching checks: HOLD-/NEED_INPUT- lifecycle prefixes, a claim-stub audit (stale_claim,
unfinalized_claim_stub), the ready_but_wave_already_accepted cross-check against the wave
board, and re-derivation of declared Counts: claims -- both the equality baseline form and
the new floor form (an invariant count that must never drop below N) -- against committed
HEAD content rather than the working tree.
Both scripts also migrate off this repo's .work/todo/ and onto the shared
~/Project Work/todos/juneau-release-manager/ (+finished/) tracker, adopting the
TODO-Rnnnn id-letter convention already used by the other copies of this tooling.
Adds scripts/tests/, a hermetic pytest suite (205 tests) that never touches the real
~/Project Work trackers and derives the project's id letter and tracker slug from the
scripts themselves -- the test harness itself has never been committed in this repo
before now.
---
scripts/tests/.gitignore | 2 +
scripts/tests/README.md | 49 ++
scripts/tests/conftest.py | 62 ++
scripts/tests/test_todo_next_id.py | 417 +++++++++
scripts/tests/test_todo_status_audit.py | 1469 +++++++++++++++++++++++++++++++
scripts/todo-next-id.py | 465 ++++++++--
scripts/todo-status-audit.py | 1104 +++++++++++++++++++++--
7 files changed, 3405 insertions(+), 163 deletions(-)
diff --git a/scripts/tests/.gitignore b/scripts/tests/.gitignore
new file mode 100644
index 0000000000..6c56ff1bd9
--- /dev/null
+++ b/scripts/tests/.gitignore
@@ -0,0 +1,2 @@
+__pycache__/
+.pytest_cache/
diff --git a/scripts/tests/README.md b/scripts/tests/README.md
new file mode 100644
index 0000000000..a7473d06ba
--- /dev/null
+++ b/scripts/tests/README.md
@@ -0,0 +1,49 @@
+# scripts/tests/
+
+Pytest harness for `scripts/todo-next-id.py` and `scripts/todo-status-audit.py`. These two
+scripts have no other automated coverage today; everything here is hermetic (temp
+directories, synthetic fixture files) and never touches the real `~/Project Work` trackers.
+
+## Running
+
+`pytest` is not vendored anywhere in this repo (it's pure-stdlib elsewhere in `scripts/` on
+purpose). Two easy ways to run this suite without installing anything globally:
+
+```bash
+# Option A -- uv (fastest, no setup, nothing left behind):
+uv run --with pytest pytest scripts/tests
+
+# Option B -- a local venv:
+python3 -m venv .venv
+.venv/bin/pip install pytest
+.venv/bin/pytest scripts/tests
+```
+
+Do **not** `pip install --user pytest` / `pip install pytest` on a Homebrew-managed
+`python3` -- PEP 668 ("externally managed environment") will refuse it (or force
+`--break-system-packages`, which this repo's tooling should not depend on).
+
+## Layout
+
+- `conftest.py` -- loads the two hyphenated scripts as fresh module objects per test (they
+ can't be `import`ed by name) via `next_id_module` / `audit_module` fixtures.
+- `test_todo_next_id.py` -- the id allocator: concurrent-claim uniqueness (25/30/60-way),
+ every recognized filename prefix (including the `finished/` archive), the
+ unrecognized-prefix id-reuse hazard, and bounded retry/collision behavior.
+- `test_todo_status_audit.py` -- the status/header pre-filter: every documented prefix is
+ actually scanned (cross-checked against the module's own docstring, not hand-duplicated),
+ every documented reason code is actually emittable (same cross-check, one level up),
+ `empty_status_value` fires only on genuinely blank/decoration-only values, markdown
+ decoration doesn't defeat the lifecycle checks, and the TODO/READY/MAYBE/HOLD
+ lifecycle-mismatch signals. The `ready_but_wave_already_accepted` wave cross-check carries
+ both directions: it fires on an item still queued behind an accepted-and-closed wave, and
+ stays silent on the four false-positive shapes that exist in the real corpus -- a body-only
+ cross-reference to another wave's members, a non-`Ready to execute` umbrella that names the
+ wave in its header, a wave still in flight (live `waves/WAVE-nnnn-*.md`), and a negated
+ board line ("not accepted"). A regression test also pins that naming pre-existing symbols is
+ never a signal, since this check reads tracker records only and never the source tree.
+
+This file (and every file in this directory) is intended to be byte-for-byte identical
+across every repo that carries `todo-next-id.py` / `todo-status-audit.py` -- the tests derive
+the project's id letter and tracker slug from the scripts themselves rather than hardcoding
+them, so nothing here needs to differ between copies.
diff --git a/scripts/tests/conftest.py b/scripts/tests/conftest.py
new file mode 100644
index 0000000000..e04461f68c
--- /dev/null
+++ b/scripts/tests/conftest.py
@@ -0,0 +1,62 @@
+# ***************************************************************************************************************************
+# * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an
+# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# * specific language governing permissions and limitations under the License.
+# ***************************************************************************************************************************
+"""
+Shared pytest fixtures for scripts/tests/.
+
+scripts/todo-next-id.py and scripts/todo-status-audit.py have hyphens in their filenames, so
+they cannot be `import`ed by name -- they're loaded here by file path via importlib. Each
+fixture loads a brand-new module object per test (never cached/shared across tests), so a
+test that monkeypatches a module-level constant (e.g. MAX_CLAIM_ATTEMPTS) can never leak
+that mutation into an unrelated test.
+
+This file (and everything else under scripts/tests/) is identical across every repo that
+carries todo-next-id.py / todo-status-audit.py -- it derives the project's id letter and
+tracker slug from the scripts themselves (next_id_module.ID_PROJECT_LETTER / TRACKER_SLUG),
+never hardcodes them, so the same body works everywhere without per-repo edits.
+"""
+
+from __future__ import annotations
+
+import importlib.util
+from pathlib import Path
+
+import pytest
+
+SCRIPTS_DIR = Path(__file__).resolve().parent.parent
+
+
+def _load_script(script_stem: str):
+ """Load scripts/<script_stem>.py (e.g. "todo-next-id") as a fresh module object."""
+ path = SCRIPTS_DIR / f"{script_stem}.py"
+ spec = importlib.util.spec_from_file_location(f"_undertest_{script_stem.replace('-', '_')}", path)
+ module = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(module)
+ return module
+
+
+@pytest.fixture
+def scripts_dir() -> Path:
+ """Absolute path to the scripts/ directory containing the two tracker scripts under test."""
+ return SCRIPTS_DIR
+
+
+@pytest.fixture
+def next_id_module():
+ """A fresh scripts/todo-next-id.py module object (see module docstring for why "fresh")."""
+ return _load_script("todo-next-id")
+
+
+@pytest.fixture
+def audit_module():
+ """A fresh scripts/todo-status-audit.py module object (see module docstring for why "fresh")."""
+ return _load_script("todo-status-audit")
diff --git a/scripts/tests/test_todo_next_id.py b/scripts/tests/test_todo_next_id.py
new file mode 100644
index 0000000000..9cb339ae6c
--- /dev/null
+++ b/scripts/tests/test_todo_next_id.py
@@ -0,0 +1,417 @@
+# ***************************************************************************************************************************
+# * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an
+# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# * specific language governing permissions and limitations under the License.
+# ***************************************************************************************************************************
+"""
+Tests for scripts/todo-next-id.py -- the atomic TODO-id allocator.
+
+Two failure modes motivate this suite, both of the "produces no error, just wrong" kind:
+
+1. CONCURRENCY (the historical bug): the old allocator computed `next = 1 + max(existing
+ ids)` and simply returned it -- a read, not a reservation. Two sessions racing between
+ their scan and their write could be (and were) handed the SAME id. claim_next_id() fixes
+ this by atomically creating "TODO-<id>-CLAIMED.md" with os.open(O_CREAT | O_EXCL) before
+ ever returning a candidate; see TestConcurrentClaims below for why that makes the "no
+ duplicate ids" assertion deterministic rather than a matter of getting lucky with thread
+ scheduling.
+2. SCANNER COVERAGE: collect_ids() only "sees" a filename if FILENAME_RE recognizes its
+ prefix. A prefix it doesn't recognize isn't just skipped harmlessly -- it stops blocking
+ allocation of that number, so claim_next_id() can hand out an id an existing (possibly
+ archived) item already holds. See TestFilenamePrefixScanning.
+
+This file is byte-for-byte identical across every repo that carries todo-next-id.py: every
+project-specific value (id letter, tracker slug) is read from the loaded module itself
+(next_id_module fixture, from conftest.py), never hardcoded here.
+"""
+
+from __future__ import annotations
+
+import concurrent.futures
+import subprocess
+import sys
+
+import pytest
+
+# Every filename prefix todo-next-id.py's FILENAME_RE currently recognizes as "consumes an
+# id" -- kept as an independent, hand-maintained list (deliberately NOT derived from
+# FILENAME_RE itself, which would be circular and could never catch a scanner regression).
+# If a future prefix is added to FILENAME_RE, this list must be updated by hand for the new
+# prefix to get its own explicit coverage below.
+KNOWN_PREFIXES = ["TODO", "READY", "MAYBE", "HOLD", "FINISHED", "CANCELLED", "NEED_INPUT"]
+
+
+def _plan_filename(next_id_module, prefix: str, num: int, *, name: str = "sample") -> str:
+ letter = next_id_module.ID_PROJECT_LETTER
+ return f"{prefix}-{letter}{num:04d}-{name}.md"
+
+
+class TestFilenamePrefixScanning:
+ """Every lifecycle prefix must be scanned, in both the live tracker and the archive."""
+
+ @pytest.mark.parametrize("prefix", KNOWN_PREFIXES)
+ def test_prefix_recognized_in_live_tracker(self, next_id_module, tmp_path, prefix):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ (todo_dir / _plan_filename(next_id_module, prefix, 7)).write_text("placeholder\n")
+
+ _, numeric_ids = next_id_module.collect_ids(todo_dir, tmp_path / "finished")
+
+ assert 7 in numeric_ids, (
+ f"{prefix}-prefixed filename was not counted by collect_ids() -- a scanner gap "
+ f"here is exactly the 'silently skipped, no error' failure class this suite "
+ f"exists to catch."
+ )
+
+ @pytest.mark.parametrize("prefix", KNOWN_PREFIXES)
+ def test_prefix_recognized_in_finished_archive(self, next_id_module, tmp_path, prefix):
+ todo_dir = tmp_path / "todos"
+ finished_dir = tmp_path / "finished"
+ todo_dir.mkdir()
+ finished_dir.mkdir()
+ (finished_dir / _plan_filename(next_id_module, prefix, 11)).write_text("placeholder\n")
+
+ _, numeric_ids = next_id_module.collect_ids(todo_dir, finished_dir)
+
+ assert 11 in numeric_ids, f"{prefix}-prefixed archive filename was not counted."
+
+ def test_unrecognized_prefix_causes_real_id_collision(self, next_id_module, tmp_path):
+ """
+ Concrete reproduction of the "id REUSE" hazard: an id already in use under a prefix
+ FILENAME_RE doesn't recognize is invisible to the scan, so claim_next_id() computes a
+ candidate that collides with it -- two different files, same numeric id, both "real".
+ This is the same shape of bug as NEED_INPUT- being missing from
+ todo-status-audit.py's pattern, just with a worse consequence (silent id reuse
+ instead of a silently-skipped audit).
+ """
+ todo_dir = tmp_path / "todos"
+ finished_dir = tmp_path / "finished"
+ todo_dir.mkdir()
+ finished_dir.mkdir()
+ letter = next_id_module.ID_PROJECT_LETTER
+ for n in (1, 2, 3, 4):
+ (finished_dir / _plan_filename(next_id_module, "FINISHED", n)).write_text("x\n")
+ # An id that's genuinely in use, but under a prefix FILENAME_RE does not recognize.
+ collided_name = f"WONTFIX-{letter}0005-already-exists.md"
+ (finished_dir / collided_name).write_text("placeholder\n")
+
+ _, numeric_ids = next_id_module.collect_ids(todo_dir, finished_dir)
+ assert numeric_ids == {1, 2, 3, 4}, "id 5 should be invisible to the scanner -- WONTFIX- is not a recognized prefix"
+
+ claimed = next_id_module.claim_next_id(todo_dir, numeric_ids)
+
+ assert claimed == 5, (
+ "claim_next_id() computed a candidate that collides with an id already in use "
+ "under an unrecognized prefix -- this IS the severe id-reuse failure."
+ )
+ assert (finished_dir / collided_name).exists() # the pre-existing "5" is still there, now duplicated
+
+
+class TestClaimNextId:
+ """Basic claim behavior, collision handling, and the bounded-retry ceiling."""
+
+ def test_claims_first_free_id_on_empty_tracker(self, next_id_module, tmp_path):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+
+ claimed = next_id_module.claim_next_id(todo_dir, set())
+
+ assert claimed == 1
+ letter = next_id_module.ID_PROJECT_LETTER
+ claim_file = todo_dir / f"TODO-{letter}0001-CLAIMED.md"
+ assert claim_file.exists()
+ body = claim_file.read_text(encoding="utf-8")
+ assert "CLAIMED (placeholder, NOT a real plan)" in body
+ assert f"{letter}0001" in body
+
+ def test_claim_file_is_itself_scanned_as_taken(self, next_id_module, tmp_path):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ next_id_module.claim_next_id(todo_dir, set())
+
+ _, numeric_ids = next_id_module.collect_ids(todo_dir, tmp_path / "finished")
+
+ assert 1 in numeric_ids # retired permanently, even though never renamed to a real plan
+
+ def test_collision_retries_to_next_candidate(self, next_id_module, tmp_path):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ letter = next_id_module.ID_PROJECT_LETTER
+ # Simulate id 1 already claimed by someone else, moments earlier.
+ (todo_dir / f"TODO-{letter}0001-CLAIMED.md").write_text("already claimed\n")
+
+ claimed = next_id_module.claim_next_id(todo_dir, set())
+
+ assert claimed == 2
+
+ def test_bounded_retries_raise_instead_of_spinning_forever(self, next_id_module, tmp_path, monkeypatch):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ monkeypatch.setattr(next_id_module, "MAX_CLAIM_ATTEMPTS", 3)
+ letter = next_id_module.ID_PROJECT_LETTER
+ for n in (1, 2, 3):
+ (todo_dir / f"TODO-{letter}{n:04d}-CLAIMED.md").write_text("x\n")
+
+ with pytest.raises(RuntimeError, match="could not claim an id"):
+ next_id_module.claim_next_id(todo_dir, set())
+
+
+class TestConcurrentClaims:
+ """
+ Reproduces the exact failure mode that shipped: ids were each handed out more than once
+ in one session because the old code read `1 + max(existing ids)` and returned it -- a
+ read, not a reservation. claim_next_id() fixes this via os.open(O_CREAT | O_EXCL), which
+ the OS guarantees only one caller can ever win, even against another thread racing on the
+ exact same syscall at the exact same instant.
+
+ DETERMINISM: this is not timing-based -- there is no sleep, no retry-count assumption, no
+ "usually passes". The assertion (every returned id is unique, and the claimed set is
+ exactly {start, ..., start+N-1}) follows from the atomicity guarantee of O_CREAT|O_EXCL on
+ a local filesystem, so it holds on every run, on every machine, under any thread-
+ scheduling order -- it is not flaky. The only way it COULD flake is if the underlying
+ filesystem doesn't honor O_CREAT|O_EXCL atomically (true of some network filesystems,
+ never of a local disk/tmpfs, which is what pytest's tmp_path fixture uses). We are not
+ aware of a way to make this test meaningfully MORE deterministic than "rely on the OS's
+ own atomicity guarantee for the exact syscall the fix depends on" -- if that guarantee
+ doesn't hold, the fix itself doesn't hold either.
+ """
+
+ @pytest.mark.parametrize("concurrency", [25, 30, 60]) # the three manual-run sizes from the original incident
+ def test_concurrent_claims_are_all_unique(self, next_id_module, tmp_path, concurrency):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ numeric_ids: set[int] = set() # every caller races from the same empty snapshot
+
+ with concurrent.futures.ThreadPoolExecutor(max_workers=concurrency) as pool:
+ futures = [pool.submit(next_id_module.claim_next_id, todo_dir, numeric_ids) for _ in range(concurrency)]
+ results = [f.result() for f in futures]
+
+ assert len(results) == concurrency
+ assert len(set(results)) == concurrency, (
+ f"{concurrency}-way concurrent allocation produced duplicate ids: {sorted(results)} -- "
+ "this is the exact silent-duplicate-id bug this test exists to catch."
+ )
+ assert set(results) == set(range(1, concurrency + 1))
+
+ def test_concurrent_claims_respect_preexisting_ids(self, next_id_module, tmp_path):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ numeric_ids = {5, 7, 12} # as if a prior scan already found ids up to 12
+
+ with concurrent.futures.ThreadPoolExecutor(max_workers=30) as pool:
+ futures = [pool.submit(next_id_module.claim_next_id, todo_dir, numeric_ids) for _ in range(30)]
+ results = [f.result() for f in futures]
+
+ assert set(results) == set(range(13, 43))
+
+
+class TestFinalize:
+ """
+ Covers do_finalize() / --finalize: the atomic stub -> real-plan rename that closes the
+ finalization half of the atomicity hole (see the module docstring's "--finalize supersedes
+ the older 'rename it yourself' convention" paragraph). claim_next_id()'s O_CREAT|O_EXCL
+ already made the CLAIM half of a reservation race-proof (TestConcurrentClaims above); this
+ class covers the other half -- the caller no longer performs the rename themselves, so no
+ calling convention can leave both names on disk.
+ """
+
+ def test_finalize_renames_stub_to_final_path(self, next_id_module, tmp_path):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ letter = next_id_module.ID_PROJECT_LETTER
+ stub = todo_dir / f"TODO-{letter}0001-CLAIMED.md"
+ stub.write_text("# Real plan content, written in place over the stub\n", encoding="utf-8")
+
+ exit_code = next_id_module.do_finalize(todo_dir, "1", "short-slug")
+
+ assert exit_code == 0
+ assert not stub.exists()
+ final_path = todo_dir / f"TODO-{letter}0001-short-slug.md"
+ assert final_path.is_file()
+
+ def test_finalize_preserves_the_content_written_into_the_stub(self, next_id_module, tmp_path):
+ """The rename must carry over whatever the caller already wrote into the stub -- finalize
+ never touches content, only the filename."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ letter = next_id_module.ID_PROJECT_LETTER
+ stub = todo_dir / f"TODO-{letter}0001-CLAIMED.md"
+ stub.write_text("# The Real Plan\n\nCurrent status: Ready to execute.\n", encoding="utf-8")
+
+ next_id_module.do_finalize(todo_dir, "1", "short-slug")
+
+ final_path = todo_dir / f"TODO-{letter}0001-short-slug.md"
+ assert final_path.read_text(encoding="utf-8") == "# The Real Plan\n\nCurrent status: Ready to execute.\n"
+
+ def test_finalize_accepts_a_letter_prefixed_id(self, next_id_module, tmp_path):
+ """normalize_check_id()'s accepted forms (letter-prefixed, 'TODO-'-prefixed, ...) must
+ also work for --finalize, not just the bare digit form."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ letter = next_id_module.ID_PROJECT_LETTER
+ stub = todo_dir / f"TODO-{letter}0007-CLAIMED.md"
+ stub.write_text("placeholder\n", encoding="utf-8")
+
+ exit_code = next_id_module.do_finalize(todo_dir, f"{letter}7", "short-slug")
+
+ assert exit_code == 0
+ assert (todo_dir / f"TODO-{letter}0007-short-slug.md").is_file()
+
+ def test_finalize_fails_loudly_on_missing_stub(self, next_id_module, tmp_path, capsys):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+
+ exit_code = next_id_module.do_finalize(todo_dir, "1", "short-slug")
+
+ assert exit_code == 4
+ assert not (todo_dir / f"TODO-{next_id_module.ID_PROJECT_LETTER}0001-short-slug.md").exists()
+ assert "no claim stub" in capsys.readouterr().err
+
+ def test_finalize_rejects_a_malformed_id_without_touching_the_filesystem(self, next_id_module, tmp_path, capsys):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+
+ exit_code = next_id_module.do_finalize(todo_dir, "not-an-id!!", "short-slug")
+
+ assert exit_code == 4
+ assert "is not a valid id" in capsys.readouterr().err
+ assert list(todo_dir.iterdir()) == []
+
+ def test_finalize_rejects_a_lettered_child_id(self, next_id_module, tmp_path, capsys):
+ """Claim stubs are only ever issued for a bare numeric id -- todo-next-id.py never claims
+ on behalf of a lettered child -- so a lettered --finalize argument can never have a stub
+ and must be refused before even touching the filesystem, not treated as 'missing stub'."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+
+ exit_code = next_id_module.do_finalize(todo_dir, "7a", "short-slug")
+
+ assert exit_code == 4
+ detail = capsys.readouterr().err
+ assert "lettered child" in detail
+ assert list(todo_dir.iterdir()) == []
+
+ def test_finalize_rejects_a_malformed_slug(self, next_id_module, tmp_path, capsys):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ letter = next_id_module.ID_PROJECT_LETTER
+ stub = todo_dir / f"TODO-{letter}0001-CLAIMED.md"
+ stub.write_text("placeholder\n", encoding="utf-8")
+
+ exit_code = next_id_module.do_finalize(todo_dir, "1", "Not_A_Valid_Slug")
+
+ assert exit_code == 4
+ assert "not a valid slug" in capsys.readouterr().err
+ assert stub.exists() # refused before touching anything
+
+ def test_finalize_refuses_to_overwrite_an_existing_target(self, next_id_module, tmp_path, capsys):
+ """The exact shape of tonight's bug, caught at finalize time instead of only by the
+ audit: a real file already sits at the target name (e.g. because a caller already wrote
+ it directly instead of finalizing). finalize must not clobber it."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ letter = next_id_module.ID_PROJECT_LETTER
+ stub = todo_dir / f"TODO-{letter}0001-CLAIMED.md"
+ stub.write_text("placeholder\n", encoding="utf-8")
+ final_path = todo_dir / f"TODO-{letter}0001-short-slug.md"
+ final_path.write_text("# Already exists\n", encoding="utf-8")
+
+ exit_code = next_id_module.do_finalize(todo_dir, "1", "short-slug")
+
+ assert exit_code == 4
+ assert "already exists" in capsys.readouterr().err
+ assert stub.exists() # neither file touched -- both left exactly as they were
+ assert final_path.read_text(encoding="utf-8") == "# Already exists\n"
+
+ def test_finalize_does_not_disturb_an_unrelated_ids_stub(self, next_id_module, tmp_path):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ letter = next_id_module.ID_PROJECT_LETTER
+ stub_1 = todo_dir / f"TODO-{letter}0001-CLAIMED.md"
+ stub_1.write_text("placeholder for 1\n", encoding="utf-8")
+ stub_2 = todo_dir / f"TODO-{letter}0002-CLAIMED.md"
+ stub_2.write_text("real plan for 2\n", encoding="utf-8")
+
+ exit_code = next_id_module.do_finalize(todo_dir, "2", "short-slug")
+
+ assert exit_code == 0
+ assert stub_1.exists() # id 1's live reservation must be untouched
+ assert not stub_2.exists()
+
+
+class TestCliEndToEnd:
+ """A couple of real subprocess invocations, to catch bugs hiding in argument parsing /
+ main() wiring that direct function calls above can't see."""
+
+ def test_main_prints_next_id_and_creates_claim_file(self, scripts_dir, next_id_module, tmp_path):
+ project_work = tmp_path / "Project Work"
+ result = subprocess.run(
+ [sys.executable, str(scripts_dir / "todo-next-id.py"), "--root", str(project_work), "--allow-missing"],
+ capture_output=True,
+ text=True,
+ timeout=15,
+ )
+
+ assert result.returncode == 0, result.stderr
+ letter = next_id_module.ID_PROJECT_LETTER
+ assert result.stdout.strip() == f"{letter}0001"
+ claim_file = project_work / "todos" / next_id_module.TRACKER_SLUG / f"TODO-{letter}0001-CLAIMED.md"
+ assert claim_file.exists()
+
+ def test_missing_tracker_is_a_hard_error_without_allow_missing(self, scripts_dir, tmp_path):
+ result = subprocess.run(
+ [sys.executable, str(scripts_dir / "todo-next-id.py"), "--root", str(tmp_path / "Project Work")],
+ capture_output=True,
+ text=True,
+ timeout=15,
+ )
+
+ assert result.returncode == 2
+
+ def test_finalize_cli_end_to_end(self, scripts_dir, next_id_module, tmp_path):
+ """The full lifecycle through the CLI, not just do_finalize() directly: claim, overwrite
+ the stub in place, finalize, and confirm the real filename is what's left on disk."""
+ project_work = tmp_path / "Project Work"
+ claim_result = subprocess.run(
+ [sys.executable, str(scripts_dir / "todo-next-id.py"), "--root", str(project_work), "--allow-missing"],
+ capture_output=True,
+ text=True,
+ timeout=15,
+ )
+ assert claim_result.returncode == 0, claim_result.stderr
+ claimed_id = claim_result.stdout.strip()
+
+ todo_dir = project_work / "todos" / next_id_module.TRACKER_SLUG
+ stub = todo_dir / f"TODO-{claimed_id}-CLAIMED.md"
+ stub.write_text("# The real plan\n", encoding="utf-8")
+
+ finalize_result = subprocess.run(
+ [sys.executable, str(scripts_dir / "todo-next-id.py"), "--root", str(project_work), "--finalize", claimed_id, "short-slug"],
+ capture_output=True,
+ text=True,
+ timeout=15,
+ )
+
+ assert finalize_result.returncode == 0, finalize_result.stderr
+ assert not stub.exists()
+ final_path = todo_dir / f"TODO-{claimed_id}-short-slug.md"
+ assert final_path.is_file()
+ assert final_path.read_text(encoding="utf-8") == "# The real plan\n"
+
+ def test_finalize_cli_exits_4_on_missing_stub(self, scripts_dir, tmp_path):
+ project_work = tmp_path / "Project Work"
+ result = subprocess.run(
+ [sys.executable, str(scripts_dir / "todo-next-id.py"), "--root", str(project_work), "--allow-missing", "--finalize", "1", "short-slug"],
+ capture_output=True,
+ text=True,
+ timeout=15,
+ )
+
+ assert result.returncode == 4, result.stdout
diff --git a/scripts/tests/test_todo_status_audit.py b/scripts/tests/test_todo_status_audit.py
new file mode 100644
index 0000000000..97a8f2b9ce
--- /dev/null
+++ b/scripts/tests/test_todo_status_audit.py
@@ -0,0 +1,1469 @@
+# ***************************************************************************************************************************
+# * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an
+# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# * specific language governing permissions and limitations under the License.
+# ***************************************************************************************************************************
+"""
+Tests for scripts/todo-status-audit.py -- the status/header consistency pre-filter.
+
+The bug this suite guards against just shipped: NEED_INPUT-*.md was documented (module
+docstring) as scanned, but omitted from FILENAME_RE in all three copies of this script, so
+those files were silently skipped -- not flagged, not scanned, no output at all.
+TestDocumentedPrefixesAreScanned reproduces exactly that shape of bug: it parses the
+"promised" prefix list straight out of the module's own docstring and cross-checks it against
+what FILENAME_RE actually matches, so a future prefix added to the prose without being added
+to the pattern fails immediately, instead of staying silent until someone notices missing
+output.
+
+This file is byte-for-byte identical across every repo that carries todo-status-audit.py:
+the project-specific id letter is read from the companion todo-next-id.py module
+(next_id_module fixture), never hardcoded here.
+"""
+
+from __future__ import annotations
+
+import os
+import re
+import subprocess
+import sys
+import time
+from pathlib import Path
+
+import pytest
+
+_DOC_PREFIX_LIST_RE = re.compile(r"Checks every ([A-Za-z_/-]+?)\*\.md")
+
+# Reason codes are documented as the first token of a bullet in the "Checks performed" list,
+# either " - code Description" or " - code" with the description on the following line.
+_DOC_REASON_CODE_RE = re.compile(r"^ {2}- ([a-z][a-z0-9_]+)", re.MULTILINE)
+
+_LIFECYCLE_MISMATCH_CODES = {
+ "todo_prefix_marked_ready",
+ "on_hold_status_wrong_prefix",
+ "parked_status_wrong_prefix",
+ "ready_prefix_waiting_status",
+ "maybe_prefix_non_parked",
+ "hold_prefix_non_on_hold",
+}
+
+
+def _documented_prefixes(audit_module) -> list[str]:
+ """Parse the prefix list out of the module's own docstring (e.g. "Checks every
+ TODO-/READY-/MAYBE-/HOLD-/NEED_INPUT-*.md file..."), independently of FILENAME_RE, so the
+ comparison in test_documented_prefixes_are_all_recognized_by_filename_re below is a real
+ prose-vs-code cross-check, not a self-fulfilling one."""
+ match = _DOC_PREFIX_LIST_RE.search(audit_module.__doc__)
+ assert match, (
+ "module docstring no longer documents the scanned prefix list in the expected "
+ "'Checks every A-/B-/*.md' format -- update this regex to match the new wording."
+ )
+ return [p.rstrip("-") for p in match.group(1).split("/") if p]
+
+
+def _documented_reason_codes(audit_module) -> list[str]:
+ """Parse the reason codes out of the module docstring's "Checks performed" bullet list,
+ independently of the source, so the comparison in TestDocumentedChecksAreImplemented is a real
+ prose-vs-code cross-check rather than a self-fulfilling one."""
+ codes = _DOC_REASON_CODE_RE.findall(audit_module.__doc__)
+ assert codes, (
+ "module docstring no longer lists reason codes as ' - <code>' bullets under 'Checks "
+ "performed' -- update this regex to match the new wording."
+ )
+ return codes
+
+
+def _plan_filename(next_id_module, prefix: str, num: int, *, name: str = "sample") -> str:
+ letter = next_id_module.ID_PROJECT_LETTER
+ return f"{prefix}-{letter}{num:04d}-{name}.md"
+
+
+def _write_plan(path, *, status: str = "Ready to execute.", complexity: str = "Small", body: str = "", source: str = "") -> None:
+ source_line = f"{source}\n\n" if source else ""
+ path.write_text(
+ f"# Sample plan\n\n{source_line}Current status: {status}\nComplexity: {complexity}\n\n{body}",
+ encoding="utf-8",
+ )
+
+
+def _claim_filename(next_id_module, num: int) -> str:
+ """"TODO-<letter><num>-CLAIMED.md" -- the exact name claim_next_id() (todo-next-id.py) writes."""
+ letter = next_id_module.ID_PROJECT_LETTER
+ return f"TODO-{letter}{num:04d}-CLAIMED.md"
+
+
+def _write_claim_stub(path, *, claimed_by: str = "pid 99999 on test-host.example.com", claimed_at: str = "2026-01-01T00:00:00Z") -> None:
+ """Write a stub in the same shape claim_placeholder_text() (todo-next-id.py) produces: no
+ 'Current status:'/'Complexity:' line, a 'Claimed at:' timestamp, and a 'Claimed by: pid <n> on
+ <host>' line. Mirrors the real file's structure rather than reproducing its exact prose."""
+ formatted_id = path.name[len("TODO-"):-len("-CLAIMED.md")]
+ path.write_text(
+ f"# {formatted_id} -- CLAIMED (placeholder, NOT a real plan)\n"
+ "\n"
+ "This id was atomically reserved by todo-next-id.py so no other session\n"
+ "could be handed the same number. No plan has been written yet.\n"
+ "\n"
+ f"Claimed at: {claimed_at}\n"
+ f"Claimed by: {claimed_by}\n"
+ "Command: scripts/todo-next-id.py\n",
+ encoding="utf-8",
+ )
+
+
+def _age_stub(path, hours: float) -> None:
+ """Backdate path's mtime by `hours` so audit_claim_stub()'s age computation sees that age."""
+ when = time.time() - hours * 3600
+ os.utime(path, (when, when))
+
+
+def _write_board(root, text: str, *, live_waves=()) -> "object":
+ """Write a WAVE.md board at root and (optionally) live waves/WAVE-nnnn-*.md files, returning
+ the board path. Mirrors the real board layout: WAVE.md with waves/ as a sibling."""
+ board = root / "WAVE.md"
+ board.write_text(text, encoding="utf-8")
+ if live_waves:
+ waves_dir = root / "waves"
+ waves_dir.mkdir(exist_ok=True)
+ for wave in live_waves:
+ (waves_dir / f"{wave}-in-flight.md").write_text("# in flight\n", encoding="utf-8")
+ return board
+
+
+class TestDocumentedPrefixesAreScanned:
+ def test_documented_prefixes_are_all_recognized_by_filename_re(self, audit_module, next_id_module):
+ letter = next_id_module.ID_PROJECT_LETTER
+ prefixes = _documented_prefixes(audit_module)
+
+ assert prefixes, "expected at least one documented prefix"
+ for prefix in prefixes:
+ sample_name = f"{prefix}-{letter}0001-sample.md"
+ assert audit_module.FILENAME_RE.match(sample_name), (
+ f"'{prefix}' is documented as scanned in the module docstring, but FILENAME_RE "
+ f"does not match '{sample_name}' -- this is exactly the NEED_INPUT- bug: a "
+ f"prefix promised in prose, silently skipped by the actual scan pattern."
+ )
+
+ @pytest.mark.parametrize("prefix", ["TODO", "READY", "MAYBE", "HOLD", "NEED_INPUT"])
+ def test_each_prefix_is_found_and_audited_not_skipped(self, audit_module, next_id_module, tmp_path, prefix):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ path = todo_dir / _plan_filename(next_id_module, prefix, 3)
+ # Deliberately malformed (no status/complexity line at all) so it is guaranteed to be
+ # flagged -- the point here is "was it looked at", not "what did it get flagged for".
+ path.write_text("# Sample plan\n\nNo status or complexity line here.\n", encoding="utf-8")
+
+ found = audit_module.find_plan_files(todo_dir)
+
+ assert path in found, f"{prefix}-prefixed file was not returned by find_plan_files() -- silently skipped."
+ flags = audit_module.audit_file(path)
+ assert any(code == "missing_status_header" for code, _ in flags)
+
+ def test_finished_and_cancelled_are_explicitly_out_of_scope(self, audit_module, next_id_module, tmp_path):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ (todo_dir / _plan_filename(next_id_module, "FINISHED", 9, name="old")).write_text("no status line\n")
+ (todo_dir / _plan_filename(next_id_module, "CANCELLED", 10, name="old")).write_text("no status line\n")
+
+ assert audit_module.find_plan_files(todo_dir) == []
+
+ def test_unrecognized_prefix_is_silently_skipped_by_design(self, audit_module, next_id_module, tmp_path):
+ """Sanity check on find_plan_files() itself: an unrecognized prefix truly is invisible
+ to it, which is exactly why the documented-prefix cross-check above matters."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ letter = next_id_module.ID_PROJECT_LETTER
+ (todo_dir / f"SOMETHING_ELSE-{letter}0001-x.md").write_text("Current status: Done\nComplexity: Small\n")
+
+ assert audit_module.find_plan_files(todo_dir) == []
+
+
+class TestEmptyStatusValue:
+ """Both directions matter: a check that never fires is as useless as one that always
+ does. empty_status_value must fire on structurally-blank values and stay silent on the
+ free-form descriptive prose the trackers actually use."""
+
+ @pytest.mark.parametrize("status", ["", " ", "-", "---", "***", "``", "** **", "____", "-- -- --"])
+ def test_fires_on_genuinely_blank_or_decoration_only(self, audit_module, status):
+ path_text = f"# Sample\n\nCurrent status: {status}\nComplexity: Small\n"
+ assert audit_module.status_value_is_empty(status) # unit-level: the predicate itself
+ # And end-to-end, via audit_file(), on an actual file:
+ import tempfile
+ from pathlib import Path
+
+ with tempfile.TemporaryDirectory() as tmp:
+ path = Path(tmp) / "TODO-X0001-sample.md"
+ path.write_text(path_text, encoding="utf-8")
+ flags = audit_module.audit_file(path)
+ assert any(code == "empty_status_value" for code, _ in flags), f"status={status!r} should have flagged empty_status_value"
+
+ @pytest.mark.parametrize(
+ "status",
+ [
+ "Done",
+ "Done.",
+ "**COMPLETE (2026-07-16).**",
+ "Blocked on the pin bump.",
+ "Blocked on `TODO-J0042` landing first.",
+ "Ready to execute -- no open questions remain.",
+ "On hold pending operator decision; see notes.",
+ "**Parked** -- revisit after the next release.",
+ "This is a long, descriptive, bolded narrative paragraph about **why** this item "
+ "is where it is, with -- dashes -- and other punctuation sprinkled throughout it "
+ "for good measure.",
+ ],
+ )
+ def test_does_not_fire_on_descriptive_prose(self, audit_module, next_id_module, tmp_path, status):
+ path = tmp_path / _plan_filename(next_id_module, "TODO", 1)
+ _write_plan(path, status=status)
+
+ flags = audit_module.audit_file(path)
+
+ assert not any(code == "empty_status_value" for code, _ in flags), f"status={status!r} should NOT have flagged empty_status_value"
+
+
+class TestMarkdownDecorationDoesNotDefeatChecks:
+ def test_ready_to_execute_detected_under_bold_decoration(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "TODO", 1)
+ _write_plan(path, status="**Ready to execute** -- nothing left open.")
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "todo_prefix_marked_ready" in flags
+
+ def test_on_hold_detected_under_backtick_decoration(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "TODO", 1)
+ _write_plan(path, status="`On hold` pending a decision.")
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "on_hold_status_wrong_prefix" in flags
+
+ def test_parked_detected_under_leading_dash_decoration(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "TODO", 1)
+ _write_plan(path, status="- Parked until further notice.")
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "parked_status_wrong_prefix" in flags
+
+ def test_hold_prefix_ok_status_detected_under_blockquote_decoration(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "HOLD", 1)
+ _write_plan(path, status="> On hold, see thread for context.")
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "hold_prefix_non_on_hold" not in flags
+
+
+class TestLifecycleMismatchChecks:
+ def test_todo_prefixed_file_marked_ready_is_flagged(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "TODO", 1)
+ _write_plan(path, status="Ready to execute.")
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "todo_prefix_marked_ready" in flags
+
+ def test_todo_prefixed_file_says_on_hold_is_flagged(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "TODO", 1)
+ _write_plan(path, status="On hold pending review.")
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "on_hold_status_wrong_prefix" in flags
+
+ def test_todo_prefixed_file_says_parked_is_flagged(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "TODO", 1)
+ _write_plan(path, status="Parked -- revisit later.")
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "parked_status_wrong_prefix" in flags
+
+ def test_ready_prefixed_file_still_waiting_is_flagged(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "READY", 1)
+ _write_plan(path, status="Waiting for user input.")
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "ready_prefix_waiting_status" in flags
+
+ def test_maybe_prefixed_file_not_parked_is_flagged(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "MAYBE", 1)
+ _write_plan(path, status="Ready to execute.")
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "maybe_prefix_non_parked" in flags
+
+ def test_hold_prefixed_file_not_on_hold_is_flagged(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "HOLD", 1)
+ _write_plan(path, status="Ready to execute.")
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "hold_prefix_non_on_hold" in flags
+
+ def test_correctly_matched_prefixes_are_not_flagged(self, audit_module, next_id_module, tmp_path):
+ cases = [
+ ("TODO", "Not yet started."),
+ ("READY", "Ready to execute -- no open questions."),
+ ("MAYBE", "Parked -- low priority."),
+ ("HOLD", "On hold pending a decision."),
+ ]
+ for prefix, status in cases:
+ path = tmp_path / _plan_filename(next_id_module, prefix, 1, name=prefix.lower())
+ _write_plan(path, status=status)
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert not (flags & _LIFECYCLE_MISMATCH_CODES), f"{prefix} with status {status!r} should not trigger a lifecycle-mismatch flag, got {flags}"
+
+ def test_ready_but_open_questions_unresolved_is_flagged(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "READY", 1)
+ body = "## Open questions\n\n1. Should this use approach A or B?\n"
+ _write_plan(path, status="Ready to execute.", body=body)
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "ready_but_has_open_questions" in flags
+
+ def test_ready_with_resolved_open_questions_is_not_flagged(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "READY", 1)
+ body = "## Open questions\n\n1. Should this use approach A or B? Resolved: approach A.\n"
+ _write_plan(path, status="Ready to execute.", body=body)
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "ready_but_has_open_questions" not in flags
+
+ def test_need_input_prefix_is_exempt_from_lifecycle_mismatch_checks(self, audit_module, next_id_module, tmp_path):
+ """NEED_INPUT files are scanned (structural checks apply) but deliberately not
+ subject to the TODO/READY-specific Parked-/On-hold-prefix-mismatch checks."""
+ path = tmp_path / _plan_filename(next_id_module, "NEED_INPUT", 1)
+ _write_plan(path, status="On hold, parked, ready to execute -- all at once.")
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert not (flags & _LIFECYCLE_MISMATCH_CODES)
+
+
+class TestClosedOutWaves:
+ """The board side of the wave cross-check. A wave only counts as closed out when BOTH
+ records agree: the board's prose log says accepted AND the WAVE file is gone."""
+
+ def test_missing_board_yields_empty_set(self, audit_module, tmp_path):
+ assert audit_module.closed_out_waves(tmp_path / "nope" / "WAVE.md") == frozenset()
+
+ def test_none_board_yields_empty_set(self, audit_module):
+ assert audit_module.closed_out_waves(None) == frozenset()
+
+ def test_accepted_wave_with_no_live_file_is_closed_out(self, audit_module, tmp_path):
+ board = _write_board(tmp_path, "WAVE-0003 was accepted and pushed 2026-08-25.\n")
+
+ assert audit_module.closed_out_waves(board) == frozenset({"WAVE-0003"})
+
+ def test_multiple_accepted_waves_on_one_line_are_all_collected(self, audit_module, tmp_path):
+ board = _write_board(tmp_path, "WAVE-0003 was accepted and pushed; WAVE-0002 was accepted too.\n")
+
+ assert audit_module.closed_out_waves(board) == frozenset({"WAVE-0002", "WAVE-0003"})
+
+ def test_accepted_wave_with_a_live_wave_file_is_not_closed_out(self, audit_module, tmp_path):
+ """An in-flight wave still has its file on disk. The filesystem is the non-prose half of
+ the check, and it must veto a loose 'accepted' word match on the board."""
+ board = _write_board(tmp_path, "WAVE-0004 accepted in principle, still running.\n", live_waves=["WAVE-0004"])
+
+ assert audit_module.closed_out_waves(board) == frozenset()
+
+ @pytest.mark.parametrize(
+ "line",
+ [
+ "WAVE-0004 is not accepted yet.",
+ "WAVE-0004 was never accepted.",
+ "WAVE-0004 isn't accepted.",
+ "WAVE-0004 wasn't accepted.",
+ ],
+ )
+ def test_negated_acceptance_does_not_read_as_accepted(self, audit_module, tmp_path, line):
+ """Reading 'not accepted' as accepted would flag a CORRECTLY-queued item -- precisely the
+ cry-wolf failure this script was rewritten to eliminate, so it is guarded explicitly."""
+ board = _write_board(tmp_path, line + "\n")
+
+ assert audit_module.closed_out_waves(board) == frozenset()
+
+ def test_wave_mentioned_without_acceptance_is_not_closed_out(self, audit_module, tmp_path):
+ board = _write_board(tmp_path, "| WAVE-0005 | queued | 3 workers |\n")
+
+ assert audit_module.closed_out_waves(board) == frozenset()
+
+
+class TestHeaderRegion:
+ def test_header_region_stops_at_first_section_heading(self, audit_module):
+ text = "# Title\n\nCurrent status: Ready to execute.\n\n## Goal\n\nBody mentions WAVE-0003.\n"
+
+ header = audit_module.header_region(text)
+
+ assert "Current status" in header
+ assert "WAVE-0003" not in header
+
+ def test_header_region_is_whole_file_when_there_are_no_headings(self, audit_module):
+ text = "# Title\n\nCurrent status: Ready to execute.\n"
+
+ assert audit_module.header_region(text) == text
+
+
+class TestReadyButWaveAlreadyAccepted:
+ """The drift this check exists for: nine items shipped to master in one commit, stayed in the
+ active tracker for a day still saying 'Ready to execute ... do not farm until the operator
+ starts the WAVE queue', and the stale claim then propagated into downstream sequencing."""
+
+ def test_ready_item_queued_behind_an_accepted_wave_is_flagged(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "READY", 1)
+ _write_plan(
+ path,
+ status="Ready to execute. Do not farm until the operator starts the WAVE queue.",
+ source="Expanded to a reviewable spec (WAVE-0003 spec-review pass).",
+ body="## Goal\n\nShip the thing.\n",
+ )
+
+ flags = {code for code, _ in audit_module.audit_file(path, frozenset({"WAVE-0003"}))}
+
+ assert "ready_but_wave_already_accepted" in flags
+
+ def test_detail_names_the_offending_wave(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "READY", 1)
+ _write_plan(path, source="Filed under WAVE-0003.", body="## Goal\n\nx\n")
+
+ details = [d for code, d in audit_module.audit_file(path, frozenset({"WAVE-0003"})) if code == "ready_but_wave_already_accepted"]
+
+ assert details and "WAVE-0003" in details[0]
+
+ def test_no_board_means_the_check_is_inert(self, audit_module, next_id_module, tmp_path):
+ """Default closed_waves is empty, so every pre-existing single-argument caller keeps its
+ old behavior instead of getting a check silently evaluated against an unknown board."""
+ path = tmp_path / _plan_filename(next_id_module, "READY", 1)
+ _write_plan(path, source="Filed under WAVE-0003.", body="## Goal\n\nx\n")
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "ready_but_wave_already_accepted" not in flags
+
+ def test_wave_still_in_flight_is_not_flagged(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "READY", 1)
+ _write_plan(path, source="Filed under WAVE-0004.", body="## Goal\n\nx\n")
+
+ flags = {code for code, _ in audit_module.audit_file(path, frozenset({"WAVE-0003"}))}
+
+ assert "ready_but_wave_already_accepted" not in flags
+
+ def test_body_only_cross_reference_to_an_accepted_wave_is_not_flagged(self, audit_module, next_id_module, tmp_path):
+ """The corpus false-positive case. Four unstarted items reference the accepted wave only to
+ say they must be serialized against its members. That is a statement about OTHER items, it
+ lives in a body section, and it must not be read as this item's own membership."""
+ path = tmp_path / _plan_filename(next_id_module, "TODO", 2)
+ _write_plan(
+ path,
+ status="Ready to execute.",
+ body=(
+ "## Goal\n\nDo an unrelated thing.\n\n"
+ "## Sequencing\n\n"
+ "`READY-J0445u` / `READY-J0445v` all edit the same hot file, and the board records "
+ "WAVE-0003 as serial on it. This item must be serialized against those.\n"
+ ),
+ )
+
+ flags = {code for code, _ in audit_module.audit_file(path, frozenset({"WAVE-0003"}))}
+
+ assert "ready_but_wave_already_accepted" not in flags
+
+ def test_non_ready_status_is_not_flagged_even_with_a_header_wave(self, audit_module, next_id_module, tmp_path):
+ """The umbrella case: a parent item legitimately names the wave in its header while being
+ 'In progress' rather than queued. It never claimed not to have run, so there is no
+ contradiction to flag."""
+ path = tmp_path / _plan_filename(next_id_module, "TODO", 3)
+ _write_plan(
+ path,
+ status="In progress. Do not farm WAVE-0003 until operator kick-off.",
+ body="## Goal\n\nUmbrella.\n",
+ )
+
+ flags = {code for code, _ in audit_module.audit_file(path, frozenset({"WAVE-0003"}))}
+
+ assert "ready_but_wave_already_accepted" not in flags
+
+ def test_ready_item_with_no_wave_reference_at_all_is_not_flagged(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "READY", 4)
+ _write_plan(path, body="## Goal\n\nx\n")
+
+ flags = {code for code, _ in audit_module.audit_file(path, frozenset({"WAVE-0003"}))}
+
+ assert "ready_but_wave_already_accepted" not in flags
+
+ @pytest.mark.parametrize("prefix", ["MAYBE", "HOLD", "NEED_INPUT"])
+ def test_non_todo_ready_prefixes_are_exempt(self, audit_module, next_id_module, tmp_path, prefix):
+ """Same scoping as ready_but_has_open_questions: this is a TODO/READY lifecycle signal, and
+ the module docstring's prefix table exempts the others from those."""
+ path = tmp_path / _plan_filename(next_id_module, prefix, 5)
+ _write_plan(path, source="Filed under WAVE-0003.", body="## Goal\n\nx\n")
+
+ flags = {code for code, _ in audit_module.audit_file(path, frozenset({"WAVE-0003"}))}
+
+ assert "ready_but_wave_already_accepted" not in flags
+
+ def test_naming_symbols_that_already_exist_is_not_a_signal(self, audit_module, next_id_module, tmp_path):
+ """The inverse trap, pinned deliberately. An item that MODIFIES existing code names
+ symbols that already resolve in the tree; symbol-existence probing would flag it. This
+ check reads only tracker records, never the source tree, so a prose-heavy item full of
+ real class names cannot be flagged by it."""
+ path = tmp_path / _plan_filename(next_id_module, "READY", 6)
+ _write_plan(
+ path,
+ body=(
+ "## Scope\n\n"
+ "Widen `NestedTableDef.validate()`, extend `ViewTable.emitNestedTable`, and keep "
+ "`RowDetailDef.CONTRACT_VERSION` at `\"1\"`. Tests extend `ViewsJs_NestedTable_Test`.\n"
+ ),
+ )
+
+ flags = {code for code, _ in audit_module.audit_file(path, frozenset({"WAVE-0003"}))}
+
+ assert flags == set(), f"an item naming only pre-existing symbols must stay clean, got {flags}"
+
+
+class TestHeaderStructureChecks:
+ def test_missing_status_header_flagged(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "TODO", 1)
+ path.write_text("# Sample\n\nComplexity: Small\n", encoding="utf-8")
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "missing_status_header" in flags
+
+ def test_missing_complexity_header_flagged(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "TODO", 1)
+ path.write_text("# Sample\n\nCurrent status: Done\n", encoding="utf-8")
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "missing_complexity_header" in flags
+
+ def test_status_after_first_heading_is_misplaced(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "TODO", 1)
+ path.write_text(
+ "# Sample\n\nComplexity: Small\n\n## Goal\n\nCurrent status: Done\n",
+ encoding="utf-8",
+ )
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "status_header_misplaced" in flags
+
+ def test_status_before_first_heading_is_not_misplaced(self, audit_module, next_id_module, tmp_path):
+ path = tmp_path / _plan_filename(next_id_module, "TODO", 1)
+ _write_plan(path, status="Done")
+
+ flags = {code for code, _ in audit_module.audit_file(path)}
+
+ assert "status_header_misplaced" not in flags
+
+
+class TestClaimStubHandling:
+ """Covers the false-positive fix: an id-reservation stub from todo-next-id.py's
+ claim_next_id() must not be audited as an ordinary (malformed) plan file, and must instead be
+ checked for staleness on its own terms. See the module docstring's "Claim-stub handling"."""
+
+ def test_claim_stub_filename_matches_claim_stub_re(self, audit_module, next_id_module):
+ assert audit_module.CLAIM_STUB_RE.match(_claim_filename(next_id_module, 37))
+
+ def test_claim_stub_is_still_found_by_find_plan_files(self, audit_module, next_id_module, tmp_path):
+ """The reservation must keep counting as "taken" for numbering -- it is exempted from the
+ plan-file CHECKS, not removed from the scan entirely."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ _write_claim_stub(stub)
+
+ assert stub in audit_module.find_plan_files(todo_dir)
+
+ def test_fresh_claim_stub_would_have_failed_the_old_plan_file_checks(self, audit_module, next_id_module, tmp_path):
+ """Sanity check proving the bug shape is real: routed through audit_file() (the pre-fix
+ code path), a claim stub is unconditionally flagged on both checks it can never satisfy.
+ Guards the exemption test below against passing vacuously."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ _write_claim_stub(stub)
+
+ flags = {code for code, _ in audit_module.audit_file(stub)}
+
+ assert {"missing_status_header", "missing_complexity_header"} <= flags
+
+ def test_fresh_claim_stub_is_exempt_via_audit_claim_stub(self, audit_module, next_id_module, tmp_path):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ _write_claim_stub(stub) # just written: mtime is "now"
+
+ assert audit_module.audit_claim_stub(stub) == []
+
+ def test_claim_well_under_the_threshold_is_a_live_claim_negative_case(self, audit_module, next_id_module, tmp_path):
+ """The case that matters most: a claim mid-allocation -- exactly the shape a real WAVE id
+ reservation takes while its plan is being written -- must never trip the stale check."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ _write_claim_stub(stub)
+ _age_stub(stub, hours=1) # a normal claim-to-rename gap, nowhere near STALE_CLAIM_AGE_HOURS
+
+ assert audit_module.audit_claim_stub(stub) == []
+
+ def test_claim_just_under_the_threshold_is_not_flagged(self, audit_module, next_id_module, tmp_path):
+ """Boundary case one hour inside the threshold: must not fire."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ _write_claim_stub(stub)
+ _age_stub(stub, hours=audit_module.STALE_CLAIM_AGE_HOURS - 1)
+
+ assert audit_module.audit_claim_stub(stub) == []
+
+ def test_claim_past_the_threshold_is_flagged_stale(self, audit_module, next_id_module, tmp_path):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ _write_claim_stub(stub)
+ _age_stub(stub, hours=audit_module.STALE_CLAIM_AGE_HOURS + 1)
+
+ flags = {code for code, _ in audit_module.audit_claim_stub(stub)}
+
+ assert flags == {"stale_claim"}
+
+ def test_stale_detail_names_the_age_and_the_recorded_claimant(self, audit_module, next_id_module, tmp_path):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ _write_claim_stub(stub, claimed_by="pid 11895 on jamesbog-ltmjqf2.internal.salesforce.com")
+ _age_stub(stub, hours=48)
+
+ _, detail = audit_module.audit_claim_stub(stub)[0]
+
+ assert "pid 11895 on jamesbog-ltmjqf2.internal.salesforce.com" in detail
+ assert "48.0h" in detail
+
+ def test_stub_with_no_claimed_by_line_still_becomes_stale_from_age_alone(self, audit_module, next_id_module, tmp_path):
+ """If a stub records no pid (or a caller writes a hand-made one without it), staleness
+ must still be decidable from age alone -- the pid is never load-bearing here."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ stub.write_text("# J0037 -- CLAIMED (placeholder, NOT a real plan)\n\nNo metadata lines at all.\n", encoding="utf-8")
+ _age_stub(stub, hours=audit_module.STALE_CLAIM_AGE_HOURS + 1)
+
+ flags = audit_module.audit_claim_stub(stub)
+
+ assert flags and flags[0][0] == "stale_claim"
+ assert "unrecorded claimant" in flags[0][1]
+
+ def test_end_to_end_live_claim_does_not_dirty_the_audit(self, scripts_dir, next_id_module, tmp_path):
+ """The bug's exact symptom, reproduced and proven fixed at the CLI level: a just-claimed
+ id (the state every successful reservation is in until the caller writes the plan) must
+ no longer flag the scan at all."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ _write_claim_stub(stub) # fresh mtime: just claimed, exactly like a live reservation
+
+ result = subprocess.run(
+ [sys.executable, str(scripts_dir / "todo-status-audit.py"), "--dir", str(todo_dir)],
+ capture_output=True,
+ text=True,
+ timeout=15,
+ )
+
+ assert result.returncode == 0, result.stdout
+ assert "Scanned 1 file(s); 0 flagged." in result.stdout
+
+ def test_end_to_end_stale_claim_is_flagged_stale_not_missing_headers(self, audit_module, scripts_dir, next_id_module, tmp_path):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ _write_claim_stub(stub)
+ _age_stub(stub, hours=audit_module.STALE_CLAIM_AGE_HOURS + 1)
+
+ result = subprocess.run(
+ [sys.executable, str(scripts_dir / "todo-status-audit.py"), "--dir", str(todo_dir)],
+ capture_output=True,
+ text=True,
+ timeout=15,
+ )
+
+ assert result.returncode == 1, result.stdout
+ assert "stale_claim" in result.stdout
+ assert "missing_status_header" not in result.stdout
+ assert "missing_complexity_header" not in result.stdout
+
+
+class TestClaimStubCoexistence:
+ """Covers unfinalized_claim_stub: a claim stub and a real plan file coexisting under the
+ same numeric id. This is the regression tonight's actual incident produced -- an agent read
+ todo-next-id.py's finalize instruction, reconsidered mid-task, and wrote the real plan under
+ a fresh slugged filename ("TODO-<id>-dispatch-tool-enforcement.md") instead of finalizing
+ the stub ("TODO-<id>-CLAIMED.md") in place, leaving both on disk. Unlike stale_claim, this
+ check is deliberately NOT age-gated -- see the module docstring's "Claim-stub handling"."""
+
+ def test_stub_alone_is_not_flagged(self, audit_module, next_id_module, tmp_path):
+ """Negative case: a claim stub with no coexisting plan must not trip this check -- that
+ is the ordinary, in-progress reservation shape TestClaimStubHandling already covers."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ _write_claim_stub(stub)
+
+ files = audit_module.find_plan_files(todo_dir)
+ groups = audit_module.group_claims_by_numeric_id(files)
+ flags = {code for code, _ in audit_module.audit_claim_stub(stub, groups[37]["plans"])}
+
+ assert "unfinalized_claim_stub" not in flags
+
+ def test_plan_alone_is_not_flagged(self, audit_module, next_id_module, tmp_path):
+ """Negative case: a real plan file with no stub at all has nothing for the coexistence
+ logic to fire on -- audit_claim_stub() is never even called for a non-stub filename in
+ main()'s dispatch, and its own grouping records no stub for that id."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ plan = todo_dir / _plan_filename(next_id_module, "TODO", 37, name="some-plan")
+ _write_plan(plan)
+
+ files = audit_module.find_plan_files(todo_dir)
+ groups = audit_module.group_claims_by_numeric_id(files)
+
+ assert groups[37]["stub"] is None
+ assert not any(code == "unfinalized_claim_stub" for code, _ in audit_module.audit_file(plan))
+
+ def test_stub_and_plan_for_the_same_id_is_flagged(self, audit_module, next_id_module, tmp_path):
+ """Reproduces tonight's actual bug shape directly: a stub and a real plan file coexisting
+ under one id. This is the regression that must never come back silently."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ _write_claim_stub(stub)
+ plan = todo_dir / _plan_filename(next_id_module, "TODO", 37, name="dispatch-tool-enforcement")
+ _write_plan(plan)
+
+ files = audit_module.find_plan_files(todo_dir)
+ groups = audit_module.group_claims_by_numeric_id(files)
+ flags = {code for code, _ in audit_module.audit_claim_stub(stub, groups[37]["plans"])}
+
+ assert "unfinalized_claim_stub" in flags
+
+ def test_flag_fires_even_on_a_brand_new_stub_not_age_gated(self, audit_module, next_id_module, tmp_path):
+ """The distinguishing property from stale_claim: a stub claimed a second ago, sitting
+ beside a real plan for the same id, must still be flagged immediately -- and stale_claim
+ itself must NOT also fire, since the stub is nowhere near STALE_CLAIM_AGE_HOURS old."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ _write_claim_stub(stub) # fresh mtime -- would be silent under stale_claim alone
+ plan = todo_dir / _plan_filename(next_id_module, "TODO", 37, name="dispatch-tool-enforcement")
+ _write_plan(plan)
+
+ flags = audit_module.audit_claim_stub(stub, [plan])
+
+ assert len(flags) == 1
+ assert flags[0][0] == "unfinalized_claim_stub"
+
+ def test_detail_names_the_coexisting_plan_file_and_finalize(self, audit_module, next_id_module, tmp_path):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ _write_claim_stub(stub)
+ plan_name = _plan_filename(next_id_module, "TODO", 37, name="dispatch-tool-enforcement")
+ plan = todo_dir / plan_name
+ _write_plan(plan)
+
+ _, detail = audit_module.audit_claim_stub(stub, [plan])[0]
+
+ assert plan_name in detail
+ assert "--finalize" in detail
+
+ def test_stub_can_be_both_stale_and_unfinalized_simultaneously(self, audit_module, next_id_module, tmp_path):
+ """The two checks are independent and can both fire on the same stub: it can be old AND
+ coexist with a real plan at the same time -- neither suppresses the other."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ _write_claim_stub(stub)
+ _age_stub(stub, hours=audit_module.STALE_CLAIM_AGE_HOURS + 1)
+ plan = todo_dir / _plan_filename(next_id_module, "TODO", 37, name="dispatch-tool-enforcement")
+ _write_plan(plan)
+
+ flags = {code for code, _ in audit_module.audit_claim_stub(stub, [plan])}
+
+ assert flags == {"unfinalized_claim_stub", "stale_claim"}
+
+ def test_lettered_child_of_the_same_numeric_id_also_counts_as_coexisting(self, audit_module, next_id_module, tmp_path):
+ """A lettered child (e.g. TODO-<id>a-...) sharing the stub's numeric id is just as much
+ a plan-shaped file for that number as an unlettered one, so it must count too."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ _write_claim_stub(stub)
+ letter = next_id_module.ID_PROJECT_LETTER
+ child = todo_dir / f"TODO-{letter}0037a-child-of-37.md"
+ _write_plan(child)
+
+ files = audit_module.find_plan_files(todo_dir)
+ groups = audit_module.group_claims_by_numeric_id(files)
+ flags = {code for code, _ in audit_module.audit_claim_stub(stub, groups[37]["plans"])}
+
+ assert "unfinalized_claim_stub" in flags
+
+ def test_end_to_end_cli_reproduces_and_flags_tonights_incident(self, scripts_dir, next_id_module, tmp_path):
+ """The exact filename shape from tonight's incident, run through the real CLI end to
+ end: a claim stub and a real plan file, same numeric id, both left on disk."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ stub = todo_dir / _claim_filename(next_id_module, 37)
+ _write_claim_stub(stub)
+ plan = todo_dir / _plan_filename(next_id_module, "TODO", 37, name="dispatch-tool-enforcement")
+ _write_plan(plan)
+
+ result = subprocess.run(
+ [sys.executable, str(scripts_dir / "todo-status-audit.py"), "--dir", str(todo_dir)],
+ capture_output=True,
+ text=True,
+ timeout=15,
+ )
+
+ assert result.returncode == 1, result.stdout
+ assert "unfinalized_claim_stub" in result.stdout
+
+
+class TestDocumentedChecksAreImplemented:
+ """Same bug shape as TestDocumentedPrefixesAreScanned, one level up: a check promised in the
+ module docstring's "Checks performed" list but never wired into audit_file() is silent
+ forever. Parses the promised reason codes out of the docstring and cross-checks them against
+ the codes the module actually emits."""
+
+ def test_every_documented_reason_code_is_emitted_somewhere(self, audit_module, scripts_dir):
+ documented = _documented_reason_codes(audit_module)
+ source = (scripts_dir / "todo-status-audit.py").read_text(encoding="utf-8")
+
+ assert len(documented) >= 10, f"expected the docstring to promise the full check list, parsed {documented}"
+ for code in documented:
+ assert f'"{code}"' in source, (
+ f"'{code}' is documented in the 'Checks performed' list but never appears as a "
+ f"flag literal in the script -- a promised check that can never fire."
+ )
+
+
+class TestDeclaredCountsGrammar:
+ """The parse half of the declared-counts convention. Nothing here touches a repo: a grammar
+ error is a grammar error whether or not the source it names is reachable, which is what lets
+ --no-counts still report counts_malformed."""
+
+ def test_none_value_is_a_declaration_not_a_flag(self, audit_module):
+ assert audit_module.parse_counts_value(" None.") == ("none", None)
+
+ def test_none_value_is_case_insensitive(self, audit_module):
+ assert audit_module.parse_counts_value(" NONE.")[0] == "none"
+
+ def test_well_formed_claim_parses_into_its_parts(self, audit_module):
+ outcome, claim = audit_module.parse_counts_value(r" `console:matches:src/**/*.java:foo\(` = 12")
+ assert outcome == "claim"
+ assert (claim.repo, claim.kind, claim.pathspec, claim.expected) == ("console", "matches", "src/**/*.java", 12)
+ assert claim.pattern.pattern == r"foo\("
+
+ def test_paths_kind_parses_without_a_regex(self, audit_module):
+ outcome, claim = audit_module.parse_counts_value(" `board:paths:finished/juneau/FINISHED-*.md` = 23")
+ assert outcome == "claim"
+ assert claim.kind == "paths"
+ assert claim.pattern is None
+
+ def test_regex_may_contain_colons(self, audit_module):
+ """maxsplit=3 keeps a ':' inside the regex intact -- otherwise the commonest CSS/Java
+ patterns ('font-size:', 'Foo::bar') could not be declared at all."""
+ outcome, claim = audit_module.parse_counts_value(r" `juneau:matches:**/*.css:font-size:\s*[\d.]+em` = 146")
+ assert outcome == "claim"
+ assert claim.pattern.pattern == r"font-size:\s*[\d.]+em"
+
+ @pytest.mark.parametrize(
+ "value",
+ [
+ "",
+ " ",
+ "ten call sites in the rest package",
+ " `console:matches:src/**/*.java:foo` = twelve",
+ " `console:matches:src/**/*.java:foo` = 12 (excluding javadoc)",
+ " `console:matches:src/**/*.java:foo` = 12, `console:files:src/**/*.java:foo` = 9",
+ " `nosuchrepo:paths:x/*.md` = 1",
+ " `console:occurrences:src/**/*.java:foo` = 1",
+ " `console:matches:src/**/*.java` = 1",
+ " `console:matches:src/**/*.java:` = 1",
+ " `board:paths:finished/*.md:extra` = 1",
+ " `console:paths` = 1",
+ " `console:matches:/abs/path/*.java:foo` = 1",
+ " `console:matches:../outside/*.java:foo` = 1",
+ " `console:matches::foo` = 1",
+ " `console:matches:src/**/*.java:[unclosed` = 1",
+ # Operator forms that are NOT the two supported ones. Listed because a near-miss operator
+ # is the likeliest way to write a floor wrong, and tolerating any of them would mean
+ # guessing which comparison the author meant.
+ " `console:matches:src/**/*.java:foo` >= twelve",
+ " `console:matches:src/**/*.java:foo` > 5",
+ " `console:matches:src/**/*.java:foo` <= 5",
+ " `console:matches:src/**/*.java:foo` => 5",
+ " `console:matches:src/**/*.java:foo` >= 5 or so",
+ ],
+ )
+ def test_malformed_values_are_reported_as_malformed(self, audit_module, value):
+ assert audit_module.parse_counts_value(value)[0] == "malformed"
+
+ def test_paths_kind_cannot_declare_zero(self, audit_module):
+ """`paths ... = 0` is refused at parse time because a zero-file scope is counts_scope_empty
+ by rule 2 -- the claim could only ever be reported as a failure, so saying so plainly beats
+ letting an author write it."""
+ outcome, detail = audit_module.parse_counts_value(" `board:paths:finished/*.md` = 0")
+ assert outcome == "malformed"
+ assert "rotted" in detail
+
+ def test_matches_kind_may_declare_zero(self, audit_module):
+ """'no call sites remain' is a legitimate assertion, unlike 'no files match this glob'."""
+ assert audit_module.parse_counts_value(" `console:matches:src/**/*.java:foo` = 0")[0] == "claim"
+
+ def test_malformed_detail_names_the_known_tokens(self, audit_module):
+ _, detail = audit_module.parse_counts_value(" `nosuchrepo:paths:x/*.md` = 1")
+ for token in ["board", *audit_module.REPO_TOKENS]:
+ assert token in detail
+
+ def test_floor_form_parses_and_records_its_operator(self, audit_module):
+ """`>=` declares an invariant: "must never drop below N". Equality over a population that
+ can legitimately grow flags on benign growth, which invites correcting the number -- and
+ correcting an invariant is how a guard gets deleted."""
+ outcome, claim = audit_module.parse_counts_value(r" `juneau:matches:src/**/*.java:foo\(` >= 36")
+ assert outcome == "claim"
+ assert (claim.operator, claim.expected, claim.is_floor) == (">=", 36, True)
+
+ def test_equality_is_the_default_and_is_not_a_floor(self, audit_module):
+ _, claim = audit_module.parse_counts_value(r" `juneau:matches:src/**/*.java:foo\(` = 36")
+ assert (claim.operator, claim.is_floor) == ("=", False)
+
+ def test_floor_of_zero_is_refused(self, audit_module):
+ """The one decorative floor catchable without inventing a slack threshold: `>= 0` holds for
+ every possible derivation, so it looks like a guard while asserting nothing. Refused for the
+ same reason `paths ... = 0` is."""
+ outcome, detail = audit_module.parse_counts_value(" `juneau:matches:src/**/*.java:foo` >= 0")
+ assert outcome == "malformed"
+ assert "asserts nothing" in detail
+
+ def test_equality_may_still_declare_zero(self, audit_module):
+ """Refusing `>= 0` must not take `= 0` with it -- "no call sites remain" is a real claim."""
+ assert audit_module.parse_counts_value(" `juneau:matches:src/**/*.java:foo` = 0")[0] == "claim"
+
+ def test_rendered_shows_the_operator_actually_applied(self, audit_module):
+ """A detail message that printed `= 36` for a claim evaluated as a floor would be unactionable
+ in exactly the case where the operator response depends on knowing which it was."""
+ _, floor = audit_module.parse_counts_value(" `juneau:paths:src/*.java` >= 2")
+ _, exact = audit_module.parse_counts_value(" `juneau:paths:src/*.java` = 2")
+ assert floor.rendered().endswith(">= 2")
+ assert exact.rendered().endswith("= 2") and ">=" not in exact.rendered()
+
+ def test_malformed_detail_mentions_the_floor_form(self, audit_module):
+ """An author who wrote the operator wrong has to be told the floor form exists."""
+ _, detail = audit_module.parse_counts_value(" ten call sites")
+ assert ">=" in detail
+
+
+class TestGlobToRegex:
+ def test_single_star_does_not_cross_a_slash(self, audit_module):
+ """fnmatch's '*' matches '/', which would silently widen every pathspec past the directory
+ the author wrote -- a count over a wider file set is a wrong count that looks right."""
+ pattern = audit_module.glob_to_regex("src/*.java")
+ assert pattern.match("src/A.java")
+ assert not pattern.match("src/sub/A.java")
+
+ def test_double_star_slash_matches_zero_directories(self, audit_module):
+ pattern = audit_module.glob_to_regex("src/**/*.java")
+ assert pattern.match("src/A.java")
+ assert pattern.match("src/a/b/A.java")
+
+ def test_question_mark_matches_one_non_slash_character(self, audit_module):
+ pattern = audit_module.glob_to_regex("finished/FINISHED-J0445?-*.md")
+ assert pattern.match("finished/FINISHED-J0445a-slug.md")
+ assert not pattern.match("finished/FINISHED-J0445-slug.md")
+
+ def test_anchored_at_both_ends(self, audit_module):
+ pattern = audit_module.glob_to_regex("scripts/*.py")
+ assert not pattern.match("other/scripts/x.py")
+ assert not pattern.match("scripts/x.pyc")
+
+
+def _git(root, *args, check: bool = True):
+ """Run git in root with a throwaway identity supplied through the ENVIRONMENT.
+
+ Never `git config` (not even --local): the identity is passed per-process so nothing on this
+ machine's git configuration is read for authorship or written by these tests.
+ """
+ env = {
+ **os.environ,
+ "GIT_AUTHOR_NAME": "audit-test",
+ "GIT_AUTHOR_EMAIL": "audit-test@example.invalid",
+ "GIT_COMMITTER_NAME": "audit-test",
+ "GIT_COMMITTER_EMAIL": "audit-test@example.invalid",
+ }
+ done = subprocess.run(["git", "-C", str(root), *args], capture_output=True, text=True, env=env, timeout=30)
+ if check:
+ assert done.returncode == 0, done.stderr
+ return done
+
+
+def _make_repo(root, files: dict):
+ """Create a git repo at root with `files` committed, and return root."""
+ root.mkdir(parents=True, exist_ok=True)
+ _git(root, "init", "-q")
+ for name, content in files.items():
+ target = root / name
+ target.parent.mkdir(parents=True, exist_ok=True)
+ if isinstance(content, bytes):
+ target.write_bytes(content)
+ else:
+ target.write_text(content, encoding="utf-8")
+ _git(root, "add", "-A")
+ _git(root, "commit", "-q", "-m", "seed")
+ return root
+
+
+def _claim(audit_module, descriptor: str, expected: int):
+ outcome, claim = audit_module.parse_counts_value(f" `{descriptor}` = {expected}")
+ assert outcome == "claim", claim
+ return claim
+
+
+class TestDeclaredCountsDerivation:
+ """The re-derivation half: every number is counted again from committed content on every call."""
+
+ def test_matches_counts_every_occurrence(self, audit_module, tmp_path):
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "foo()\nfoo()\n", "src/B.java": "foo()\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ assert resolver.derive(_claim(audit_module, r"juneau:matches:src/**/*.java:foo\(\)", 3)) == ("ok", 3, "")
+
+ def test_files_counts_distinct_files_not_occurrences(self, audit_module, tmp_path):
+ """The 'nine REST classes across ten call sites' shape: two different numbers over one
+ pattern, which is why 'files' and 'matches' are separate kinds rather than one."""
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "foo()\nfoo()\n", "src/B.java": "foo()\n", "src/C.java": "bar()\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ assert resolver.derive(_claim(audit_module, r"juneau:files:src/**/*.java:foo\(\)", 2))[1] == 2
+ assert resolver.derive(_claim(audit_module, r"juneau:matches:src/**/*.java:foo\(\)", 3))[1] == 3
+
+ def test_paths_counts_files_in_scope(self, audit_module, tmp_path):
+ repo = _make_repo(tmp_path / "repo", {"a/x.md": "1", "a/y.md": "2", "a/z.txt": "3"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ assert resolver.derive(_claim(audit_module, "juneau:paths:a/*.md", 2))[1] == 2
+
+ def test_pattern_is_multiline_so_caret_anchors_per_line(self, audit_module, tmp_path):
+ repo = _make_repo(tmp_path / "repo", {"a/x.md": "- 2026-01-01 one\nnot a bullet\n- 2026-01-02 two\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ assert resolver.derive(_claim(audit_module, r"juneau:matches:a/*.md:^- \d{4}-\d{2}-\d{2}", 2))[1] == 2
+
+ def test_many_files_of_differing_sizes_are_read_correctly(self, audit_module, tmp_path):
+ """Blobs are fetched through one `git cat-file --batch` (per-file subprocesses turned a
+ sub-second check into a minute-long one over a 5500-file pathspec). Batch parsing is
+ offset arithmetic, so this covers an empty file and mixed sizes -- the classic off-by-one."""
+ files = {"a/empty.txt": "", "a/one.txt": "foo\n", "a/big.txt": "foo\n" * 500, "a/none.txt": "bar\n"}
+ repo = _make_repo(tmp_path / "repo", files)
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ assert resolver.derive(_claim(audit_module, "juneau:matches:a/*.txt:foo", 501))[1] == 501
+ assert resolver.derive(_claim(audit_module, "juneau:files:a/*.txt:foo", 2))[1] == 2
+
+ def test_binary_files_in_scope_are_skipped_not_decoded(self, audit_module, tmp_path):
+ repo = _make_repo(tmp_path / "repo", {"a/x.bin": b"\x00foo\x00foo", "a/y.txt": "foo\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ assert resolver.derive(_claim(audit_module, "juneau:matches:a/*:foo", 1))[1] == 1
+
+ def test_empty_scope_is_scope_empty_not_a_zero_count(self, audit_module, tmp_path):
+ """The specific defence against a stale declaration: a renamed or folded-away file leaves a
+ pattern that trivially matches nothing, and 0 == 0 would report success."""
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "foo()\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ outcome, value, detail = resolver.derive(_claim(audit_module, "juneau:matches:gone/**/*.java:foo", 0))
+ assert (outcome, value) == (audit_module.COUNT_SCOPE_EMPTY, None)
+ assert "gone/**/*.java" in detail
+
+ def test_missing_checkout_is_unverifiable_not_a_pass(self, audit_module, tmp_path):
+ resolver = audit_module.CountsResolver({"juneau": str(tmp_path / "absent")})
+ assert resolver.derive(_claim(audit_module, "juneau:paths:a/*.md", 1))[0] == audit_module.COUNT_UNVERIFIABLE
+
+ def test_directory_that_is_not_a_git_repo_is_unverifiable(self, audit_module, tmp_path):
+ plain = tmp_path / "plain"
+ plain.mkdir()
+ (plain / "x.md").write_text("foo\n", encoding="utf-8")
+ resolver = audit_module.CountsResolver({"juneau": str(plain)})
+ assert resolver.derive(_claim(audit_module, "juneau:paths:*.md", 1))[0] == audit_module.COUNT_UNVERIFIABLE
+
+
+class TestDeclaredCountsResolveAgainstHead:
+ """Counts resolve against committed content, never the working tree. A plan file citing
+ working-tree-only files produced a blocked item that two adversarial reviewers read past, and
+ these repos are cloned per-WAVE, so a tree-resolved count verifies differently per clone."""
+
+ def test_uncommitted_edit_does_not_change_the_derived_count(self, audit_module, tmp_path):
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "foo()\n"})
+ (repo / "src/A.java").write_text("foo()\nfoo()\nfoo()\n", encoding="utf-8")
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ assert resolver.derive(_claim(audit_module, r"juneau:matches:src/**/*.java:foo\(\)", 1))[1] == 1
+
+ def test_untracked_file_is_not_counted(self, audit_module, tmp_path):
+ """The exact 2026-08-25 shape: files present in this working tree and in nobody's HEAD."""
+ repo = _make_repo(tmp_path / "repo", {"scripts/a.py": "x\n"})
+ (repo / "scripts/b.py").write_text("x\n", encoding="utf-8")
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ assert resolver.derive(_claim(audit_module, "juneau:paths:scripts/*.py", 1))[1] == 1
+
+ def test_committing_changes_the_count_for_a_fresh_resolver(self, audit_module, tmp_path):
+ """Nothing is cached between runs: a new process (here, a new resolver) re-derives from git
+ and sees the new commit. This is what makes a stale declaration impossible to pass by
+ inertia -- there is no recorded result for it to coast on."""
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "foo()\n"})
+ claim = _claim(audit_module, r"juneau:matches:src/**/*.java:foo\(\)", 1)
+ assert audit_module.CountsResolver({"juneau": str(repo)}).derive(claim)[1] == 1
+
+ (repo / "src/B.java").write_text("foo()\nfoo()\n", encoding="utf-8")
+ _git(repo, "add", "-A")
+ _git(repo, "commit", "-q", "-m", "more")
+ assert audit_module.CountsResolver({"juneau": str(repo)}).derive(claim)[1] == 3
+
+ def test_board_token_resolves_against_the_filesystem(self, audit_module, tmp_path):
+ """The single documented exception: ~/Project Work is not a git repository, and is never
+ cloned, so the per-clone divergence that makes tree-resolution wrong cannot arise."""
+ board = tmp_path / "Project Work"
+ (board / "finished" / "juneau").mkdir(parents=True)
+ for name in ("FINISHED-J0001-a.md", "FINISHED-J0002-b.md"):
+ (board / "finished" / "juneau" / name).write_text("x\n", encoding="utf-8")
+ resolver = audit_module.CountsResolver(project_work=board)
+ assert resolver.derive(_claim(audit_module, "board:paths:finished/juneau/FINISHED-*.md", 2))[1] == 2
+
+
+class TestDeclaredCountsRepoResolution:
+ def test_token_naming_this_repo_resolves_to_this_scripts_own_root(self, audit_module):
+ """A filesystem fact rather than a guess, so the common case (an item counting inside its
+ own project) keeps working from a relocated checkout."""
+ self_token = next(t for t, (slug, _) in audit_module.REPO_TOKENS.items() if slug == audit_module.TRACKER_SLUG)
+ assert audit_module.CountsResolver().repo_root(self_token) == audit_module.SELF_REPO_ROOT
+
+ def test_other_tokens_resolve_below_home(self, audit_module):
+ other = next(t for t, (slug, _) in audit_module.REPO_TOKENS.items() if slug != audit_module.TRACKER_SLUG)
+ assert str(audit_module.CountsResolver().repo_root(other)).startswith(str(Path.home()))
+
+ def test_override_wins_over_the_default(self, audit_module, tmp_path):
+ resolver = audit_module.CountsResolver({"juneau": str(tmp_path / "elsewhere")})
+ assert resolver.repo_root("juneau") == tmp_path / "elsewhere"
+
+ def test_same_path_in_two_repos_is_disambiguated_by_the_token(self, audit_module, tmp_path):
+ """Citing the wrong repo was itself one of the 2026-08-25 miscounts: chrome.css exists in
+ two of these repos, so there is no implicit 'this repo' default to get wrong."""
+ one = _make_repo(tmp_path / "one", {"css/chrome.css": "a{}\n"})
+ two = _make_repo(tmp_path / "two", {"css/chrome.css": "a{}\nb{}\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(one), "release-manager": str(two)})
+ assert resolver.derive(_claim(audit_module, "juneau:matches:css/chrome.css:{}", 1))[1] == 1
+ assert resolver.derive(_claim(audit_module, "release-manager:matches:css/chrome.css:{}", 2))[1] == 2
+
+
+class TestDeclaredCountsFlags:
+ """audit_file() wiring: which reason code fires, and -- as importantly -- when none does."""
+
+ def _flags(self, audit_module, tmp_path, next_id_module, counts_lines, resolver=None):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir(exist_ok=True)
+ path = todo_dir / _plan_filename(next_id_module, "TODO", 1, name="counted")
+ # "In progress." keeps this fixture out of the Ready-gated lifecycle checks, so the
+ # assertions below are about the counts codes and nothing else.
+ _write_plan(path, status="In progress.", source="\n".join(counts_lines), body="## Goal\n\nx\n")
+ return {reason for reason, _ in audit_module.audit_file(path, frozenset(), resolver)}
+
+ def test_absent_counts_line_flags_nothing(self, audit_module, next_id_module, tmp_path):
+ """Absence is silence -- the property that lets this check run over 41 pre-convention files
+ without a single new flag, and the reason it is not a required field."""
+ resolver = audit_module.CountsResolver({"juneau": str(_make_repo(tmp_path / "repo", {"a.md": "x\n"}))})
+ assert self._flags(audit_module, tmp_path, next_id_module, [], resolver) == set()
+
+ def test_declared_none_flags_nothing(self, audit_module, next_id_module, tmp_path):
+ resolver = audit_module.CountsResolver({"juneau": str(_make_repo(tmp_path / "repo", {"a.md": "x\n"}))})
+ assert self._flags(audit_module, tmp_path, next_id_module, ["Counts: None."], resolver) == set()
+
+ def test_correct_count_flags_nothing(self, audit_module, next_id_module, tmp_path):
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "foo()\nfoo()\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ lines = [r"Counts: `juneau:matches:src/**/*.java:foo\(\)` = 2"]
+ assert self._flags(audit_module, tmp_path, next_id_module, lines, resolver) == set()
+
+ def test_wrong_count_is_flagged_as_mismatch(self, audit_module, next_id_module, tmp_path):
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "foo()\nfoo()\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ lines = [r"Counts: `juneau:matches:src/**/*.java:foo\(\)` = 3"]
+ assert self._flags(audit_module, tmp_path, next_id_module, lines, resolver) == {"counts_mismatch"}
+
+ def test_mismatch_detail_names_both_numbers_and_the_revision(self, audit_module, next_id_module, tmp_path):
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "foo()\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir(exist_ok=True)
+ path = todo_dir / _plan_filename(next_id_module, "TODO", 2, name="counted")
+ _write_plan(path, status="In progress.", source=r"Counts: `juneau:matches:src/**/*.java:foo\(\)` = 7", body="## Goal\n\nx\n")
+ detail = dict(audit_module.audit_file(path, frozenset(), resolver))["counts_mismatch"]
+ assert "= 7" in detail and "re-derived 1" in detail and audit_module.COUNTS_REV in detail
+
+ def test_unanchored_declaration_is_flagged_not_passed(self, audit_module, next_id_module, tmp_path):
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "foo()\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ lines = ["Counts: `juneau:matches:renamed/**/*.java:foo` = 0"]
+ assert self._flags(audit_module, tmp_path, next_id_module, lines, resolver) == {"counts_scope_empty"}
+
+ def test_floor_that_holds_flags_nothing(self, audit_module, next_id_module, tmp_path):
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "foo()\nfoo()\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ lines = [r"Counts: `juneau:matches:src/**/*.java:foo\(\)` >= 2"]
+ assert self._flags(audit_module, tmp_path, next_id_module, lines, resolver) == set()
+
+ def test_floor_breached_by_a_decrease_is_flagged(self, audit_module, next_id_module, tmp_path):
+ """The regression a floor exists to catch: the population shrank."""
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "foo()\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ lines = [r"Counts: `juneau:matches:src/**/*.java:foo\(\)` >= 2"]
+ assert self._flags(audit_module, tmp_path, next_id_module, lines, resolver) == {"counts_below_floor"}
+
+ def test_growth_above_a_floor_stays_green_where_equality_goes_red(self, audit_module, next_id_module, tmp_path):
+ """The whole point of the operator, asserted as a contrast rather than two separate tests:
+ identical source, identical pattern, and the only difference is `>=` versus `=`. The floor
+ passes because nothing was lost; the equality form flags because the number moved. Under the
+ equality-only grammar this benign growth is what invited an operator to 'correct' an
+ invariant."""
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "foo()\nfoo()\nfoo()\nfoo()\nfoo()\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ floor = [r"Counts: `juneau:matches:src/**/*.java:foo\(\)` >= 2"]
+ exact = [r"Counts: `juneau:matches:src/**/*.java:foo\(\)` = 2"]
+ assert self._flags(audit_module, tmp_path, next_id_module, floor, resolver) == set()
+ assert self._flags(audit_module, tmp_path, next_id_module, exact, resolver) == {"counts_mismatch"}
+
+ def test_empty_scope_under_a_floor_is_scope_empty_not_a_breach(self, audit_module, next_id_module, tmp_path):
+ """A rotted pathspec derives 0, which is below any floor -- so without the ordering in
+ _flag_declared_counts it would be reported as a regression and send the operator hunting in
+ code that is fine. It must be neither a breach nor a pass."""
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "foo()\nfoo()\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ lines = ["Counts: `juneau:matches:renamed/**/*.java:foo` >= 2"]
+ assert self._flags(audit_module, tmp_path, next_id_module, lines, resolver) == {"counts_scope_empty"}
+
+ def test_breach_and_mismatch_details_are_distinguishable(self, audit_module, next_id_module, tmp_path):
+ """The operator response differs -- investigate a regression versus possibly re-baseline -- so
+ the two details must not read alike."""
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "foo()\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir(exist_ok=True)
+
+ floor_path = todo_dir / _plan_filename(next_id_module, "TODO", 3, name="floored")
+ _write_plan(floor_path, status="In progress.", source=r"Counts: `juneau:matches:src/**/*.java:foo\(\)` >= 4", body="## Goal\n\nx\n")
+ breach = dict(audit_module.audit_file(floor_path, frozenset(), resolver))["counts_below_floor"]
+
+ exact_path = todo_dir / _plan_filename(next_id_module, "TODO", 4, name="exact")
+ _write_plan(exact_path, status="In progress.", source=r"Counts: `juneau:matches:src/**/*.java:foo\(\)` = 4", body="## Goal\n\nx\n")
+ mismatch = dict(audit_module.audit_file(exact_path, frozenset(), resolver))["counts_mismatch"]
+
+ assert ">= 4" in breach and "BREACHED" in breach and "Do NOT lower" in breach
+ assert "= 4" in mismatch and "BREACHED" not in mismatch and "floor" not in mismatch
+
+ def test_unreachable_repo_is_flagged_as_unverifiable(self, audit_module, next_id_module, tmp_path):
+ resolver = audit_module.CountsResolver({"juneau": str(tmp_path / "absent")})
+ lines = ["Counts: `juneau:paths:src/*.java` = 1"]
+ assert self._flags(audit_module, tmp_path, next_id_module, lines, resolver) == {"counts_unverifiable"}
+
+ def test_malformed_line_is_flagged(self, audit_module, next_id_module, tmp_path):
+ resolver = audit_module.CountsResolver({"juneau": str(_make_repo(tmp_path / "repo", {"a.md": "x\n"}))})
+ assert self._flags(audit_module, tmp_path, next_id_module, ["Counts: ten call sites"], resolver) == {"counts_malformed"}
+
+ def test_malformed_is_reported_without_a_resolver(self, audit_module, next_id_module, tmp_path):
+ """Grammar is checked with or without source access, so --no-counts cannot hide a broken
+ declaration -- only an unverified one."""
+ assert self._flags(audit_module, tmp_path, next_id_module, ["Counts: ten call sites"], None) == {"counts_malformed"}
+
+ def test_no_resolver_means_numbers_are_not_reported_as_agreeing(self, audit_module, next_id_module, tmp_path):
+ """Silence about a number that was never checked, rather than a manufactured pass."""
+ lines = [r"Counts: `juneau:matches:src/**/*.java:foo\(\)` = 999"]
+ assert self._flags(audit_module, tmp_path, next_id_module, lines, None) == set()
+
+ def test_every_declaration_on_a_repeated_field_is_checked(self, audit_module, next_id_module, tmp_path):
+ """One declaration per line, field repeated -- so a regex containing a comma is safe, and no
+ declaration can hide behind a sibling on the same line."""
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "foo()\nfoo()\n", "src/B.java": "foo()\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ lines = [
+ r"Counts: `juneau:matches:src/**/*.java:foo\(\)` = 3",
+ r"Counts: `juneau:files:src/**/*.java:foo\(\)` = 9",
+ ]
+ flags = [reason for reason, _ in audit_module.audit_file(
+ self._plan(audit_module, tmp_path, next_id_module, lines), frozenset(), resolver)]
+ assert flags == ["counts_mismatch"]
+
+ def _plan(self, audit_module, tmp_path, next_id_module, counts_lines):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir(exist_ok=True)
+ path = todo_dir / _plan_filename(next_id_module, "TODO", 3, name="counted")
+ _write_plan(path, status="In progress.", source="\n".join(counts_lines), body="## Goal\n\nx\n")
+ return path
+
+ def test_field_name_is_case_insensitive_like_the_other_headers(self, audit_module, next_id_module, tmp_path):
+ assert self._flags(audit_module, tmp_path, next_id_module, ["counts: ten call sites"], None) == {"counts_malformed"}
+
+ def test_zero_declared_and_zero_derived_passes_when_the_scope_still_exists(self, audit_module, next_id_module, tmp_path):
+ """'No call sites remain' is a real assertion. Documented residual: this cannot distinguish
+ a correctly-zero count from a pattern that has rotted while its files stayed put -- which is
+ why the convention advises declaring the positive baseline instead where one exists."""
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "bar()\n"})
+ resolver = audit_module.CountsResolver({"juneau": str(repo)})
+ lines = [r"Counts: `juneau:matches:src/**/*.java:foo\(\)` = 0"]
+ assert self._flags(audit_module, tmp_path, next_id_module, lines, resolver) == set()
+
+
+class TestMainCliEndToEnd:
+ def test_declared_count_mismatch_is_reported_end_to_end(self, scripts_dir, next_id_module, tmp_path):
+ """Whole-CLI wiring: --repo-root is honoured, the count is re-derived, and the stale number
+ is reported with a non-zero exit."""
+ repo = _make_repo(tmp_path / "repo", {"src/A.java": "foo()\n"})
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ filename = _plan_filename(next_id_module, "TODO", 1, name="counted")
+ _write_plan(todo_dir / filename, status="In progress.", source=r"Counts: `juneau:matches:src/**/*.java:foo\(\)` = 4", body="## Goal\n\nx\n")
+
+ result = subprocess.run(
+ [sys.executable, str(scripts_dir / "todo-status-audit.py"), "--dir", str(todo_dir), "--repo-root", f"juneau={repo}"],
+ capture_output=True,
+ text=True,
+ timeout=60,
+ )
+
+ assert result.returncode == 1, result.stdout
+ assert "counts_mismatch" in result.stdout
+ assert "re-derived 1" in result.stdout
+ assert "re-derived from HEAD this run" in result.stderr # never silently skipped
+
+ def test_no_counts_announces_itself_and_still_reports_bad_grammar(self, scripts_dir, next_id_module, tmp_path):
+ """"No count disagreed" and "no count was checked" must not look the same on the terminal."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ _write_plan(todo_dir / _plan_filename(next_id_module, "TODO", 1, name="counted"),
+ status="In progress.", source="Counts: eleven call sites", body="## Goal\n\nx\n")
+
+ result = subprocess.run(
+ [sys.executable, str(scripts_dir / "todo-status-audit.py"), "--dir", str(todo_dir), "--no-counts"],
+ capture_output=True,
+ text=True,
+ timeout=60,
+ )
+
+ assert result.returncode == 1, result.stdout
+ assert "counts_malformed" in result.stdout
+ assert "not re-derived" in result.stderr
+
+ def test_malformed_repo_root_override_is_a_hard_error(self, scripts_dir, tmp_path):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+
+ result = subprocess.run(
+ [sys.executable, str(scripts_dir / "todo-status-audit.py"), "--dir", str(todo_dir), "--repo-root", "juneau"],
+ capture_output=True,
+ text=True,
+ timeout=60,
+ )
+
+ assert result.returncode == 2
+ assert "TOKEN=PATH" in result.stderr
+
+ def test_accepted_wave_drift_is_reported_end_to_end(self, scripts_dir, next_id_module, tmp_path):
+ """Whole-CLI wiring: --board is read, the wave is resolved as accepted+closed, and the
+ stale item is reported with a non-zero exit."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ filename = _plan_filename(next_id_module, "READY", 1, name="shipped")
+ _write_plan(
+ todo_dir / filename,
+ status="Ready to execute. Do not farm until the operator starts the WAVE queue.",
+ source="Filed under WAVE-0003.",
+ body="## Goal\n\nx\n",
+ )
+ board = _write_board(tmp_path, "WAVE-0003 was accepted and pushed 2026-08-25. WAVE file deleted.\n")
+
+ result = subprocess.run(
+ [sys.executable, str(scripts_dir / "todo-status-audit.py"), "--dir", str(todo_dir), "--board", str(board)],
+ capture_output=True,
+ text=True,
+ timeout=15,
+ )
+
+ assert result.returncode == 1, result.stderr
+ assert "ready_but_wave_already_accepted" in result.stdout
+ assert filename in result.stdout
+ assert "WAVE-0003" in result.stderr # the board record is announced, never silently skipped
+
+ def test_missing_board_is_announced_as_inert_not_silently_skipped(self, scripts_dir, next_id_module, tmp_path):
+ """A silently-skipped wave check is indistinguishable from a clean one -- the same
+ silent-zero trap this script already guards for a missing scan directory."""
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ _write_plan(todo_dir / _plan_filename(next_id_module, "READY", 1), source="Filed under WAVE-0003.", body="## Goal\n\nx\n")
+
+ result = subprocess.run(
+ [sys.executable, str(scripts_dir / "todo-status-audit.py"), "--dir", str(todo_dir), "--board", str(tmp_path / "absent.md")],
+ capture_output=True,
+ text=True,
+ timeout=15,
+ )
+
+ assert result.returncode == 0, result.stdout
+ assert "inert" in result.stderr
+
+ def test_need_input_file_is_scanned_end_to_end(self, scripts_dir, next_id_module, tmp_path):
+ letter = next_id_module.ID_PROJECT_LETTER
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+ filename = f"NEED_INPUT-{letter}0001-blocked.md"
+ (todo_dir / filename).write_text("# Sample\n\nNo status or complexity header here.\n", encoding="utf-8")
+
+ result = subprocess.run(
+ [sys.executable, str(scripts_dir / "todo-status-audit.py"), "--dir", str(todo_dir)],
+ capture_output=True,
+ text=True,
+ timeout=15,
+ )
+
+ assert result.returncode == 1, result.stderr # at least one file flagged
+ assert filename in result.stdout
+ assert "Scanned 1 file(s); 1 flagged." in result.stdout
+
+ def test_clean_tracker_exits_zero(self, scripts_dir, tmp_path):
+ todo_dir = tmp_path / "todos"
+ todo_dir.mkdir()
+
+ result = subprocess.run(
+ [sys.executable, str(scripts_dir / "todo-status-audit.py"), "--dir", str(todo_dir)],
+ capture_output=True,
+ text=True,
+ timeout=15,
+ )
+
+ assert result.returncode == 0, result.stderr
+
+ def test_missing_directory_is_hard_error(self, scripts_dir, tmp_path):
+ result = subprocess.run(
+ [sys.executable, str(scripts_dir / "todo-status-audit.py"), "--dir", str(tmp_path / "does-not-exist")],
+ capture_output=True,
+ text=True,
+ timeout=15,
+ )
+
+ assert result.returncode == 2
diff --git a/scripts/todo-next-id.py b/scripts/todo-next-id.py
index 08554c3326..df10f2de3e 100755
--- a/scripts/todo-next-id.py
+++ b/scripts/todo-next-id.py
@@ -1,81 +1,130 @@
#!/usr/bin/env python3
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# ***************************************************************************************************************************
+# * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an
+# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# * specific language governing permissions and limitations under the License.
+# ***************************************************************************************************************************
"""
-Next-free TODO-id calculator for this repository's .work/todo/ tracker.
+Next-free TODO-id calculator for this project's tracker under ~/Project Work/todos/juneau-release-manager/.
-Repo-agnostic: the repository root is derived from this file's own location
-(<root>/scripts/todo-next-id.py -> <root>), never hardcoded, so the same body works in
-every repository that adopts the convention. Only the REPO_LABEL / SKILL_NAME constants
-below and the license header differ between copies.
+Trackers live in ~/Project Work (todos/<slug>/ + finished/<slug>/), not in the repo.
+Only REPO_LABEL / SKILL_NAME / ID_PROJECT_LETTER / TRACKER_SLUG differ between copies.
+CUs read ~/Project Work; do not copy .work/todo into clones.
-**Ids are per-repository and start at 1 in each.** This script only ever scans the tree it
-lives in. It prints the resolved root to stderr on every run precisely so that a run made
-from the wrong working tree is visible rather than silent.
+**Ids are per-project and start at 1 in each.** This script scans ~/Project Work/todos/<slug>
+and ~/Project Work/finished/<slug>. It prints those paths to stderr on every run.
Mirrors the exact scan scope documented in this repo's TODO-management skill, in its
"Adding a new item" and "MAYBE Numbering" sections:
- 1. Every "[TODO-n]" and bare "TODO-n" token in .work/todo/TODO.md (a trailing lowercase
+ 1. Every "[TODO-n]" and bare "TODO-n" token in ~/Project Work/todos/juneau-release-manager/TODO.md (a trailing lowercase
letter suffix, e.g. "TODO-174a", is stripped -- only the numeric part counts). A
- qualified cross-repo citation such as "juneau:TODO-42" is NOT counted; it names an id in
- another repo's tracker.
+ qualified cross-repo citation such as "otherproject:TODO-42" is NOT counted; it names
+ an id in another repo's tracker.
Note that this scan cannot distinguish an illustrative id from a live one: writing
"for example, TODO-5" anywhere in TODO.md permanently consumes id 5. Write "TODO-<n>" in
prose.
- 2. Every "TODO-"/"READY-"/"MAYBE-"/"FINISHED-"/"CANCELLED-<n>[<letter>]-*.md" filename
- directly under .work/todo/ and .work/todo/finished/.
-
-next = 1 + max(all numeric ids found). A child's letter suffix (TODO-174a, FINISHED-337f, ...)
-is ignored for this computation -- only its numeric part counts, per the skill's "Numbering"
-rule -- so promoting/splitting a lettered child never consumes a new sequential id.
-
-.work/ is gitignored, so this is pure filesystem/text scanning; no git needed.
-
-A MISSING .work/todo/ directory is a hard error (exit 2), not an empty scan. Silently
+ 2. Every "TODO-"/"READY-"/"MAYBE-"/"HOLD-"/"FINISHED-"/"CANCELLED-"/"NEED_INPUT-<n>[<letter>]-*.md"
+ filename directly under ~/Project Work/todos/juneau-release-manager/ and ~/Project Work/finished/juneau-release-manager/.
+
+next = 1 + max(all numeric ids found), but that candidate is not simply returned: it is
+ATOMICALLY CLAIMED first. Every id-granting call creates "TODO-<id>-CLAIMED.md" under the
+live tracker with an exclusive (O_CREAT | O_EXCL) file create before printing anything, so
+two sessions racing on the same tracker can never receive the same id -- one of them loses
+the create and silently retries the next candidate instead. See claim_next_id() below. A
+child's letter suffix (TODO-174a, FINISHED-337f, ...) is ignored for this computation --
+only its numeric part counts, per the skill's "Numbering" rule -- so promoting/splitting a
+lettered child never consumes a new sequential id, and never needs a claim of its own.
+
+The claim file IS the reservation, not a courtesy: it satisfies FILENAME_RE, so it is
+counted as "taken" by every future scan (this process's and everyone else's) exactly like a
+real plan file. The caller is expected to overwrite it in place -- write the real Goal /
+Steps / Notes into that same file (any tool, any number of edits) -- and then run
+`--finalize <id> <slug>` to rename it to add a slug
+("TODO-<id>-CLAIMED.md" -> "TODO-<id>-<short-slug>.md") as ONE atomic operation performed by
+this script (os.rename), not by the caller. A caller that crashes before finalizing leaves
+the stub sitting in the tracker, loudly labelled "CLAIMED (placeholder, NOT a real plan)"
+with the claiming pid/host/timestamp -- diagnosable, not a plausible-looking empty TODO.
+`--list-claims` finds every such stub and its age; nothing deletes them automatically, and
+the retired number is never reused even if the stub is later removed.
+
+`--finalize` supersedes the older "rename it yourself" convention, which is no longer
+documented as an option: a caller that renames the stub by hand, or -- the failure this
+closes -- writes the real plan under a brand-new filename and abandons the stub in place,
+produces exactly the bug this exists to prevent: a stub and a real plan coexisting under
+the same id. This script cannot intercept an arbitrary file write by the caller, so it
+cannot refuse that outright; todo-status-audit.py's `unfinalized_claim_stub` check is the
+backstop that catches it immediately (unconditionally of age) if a caller bypasses
+`--finalize` anyway. `--finalize` fails loudly rather than guessing if the id is malformed
+or lettered (claim stubs are only ever issued for a bare numeric id), if no stub exists for
+it, or if the final path already exists -- see do_finalize() below.
+
+Trackers are outside the git repo (~/Project Work); this is a pure filesystem scan.
+
+A MISSING ~/Project Work/todos/juneau-release-manager/ directory is a hard error (exit 2), not an empty scan. Silently
returning "1" from a tree that has no tracker is how ids get reused: it is the correct answer
in a freshly-seeded repo and a catastrophic one in a repo whose tracker you failed to find.
-Pass --allow-missing if you genuinely want the empty-tree answer.
+Pass --allow-missing if you genuinely want the empty-tree answer (this also creates the
+directory so the claim file below has somewhere to land).
Usage:
./scripts/todo-next-id.py
./scripts/todo-next-id.py --list
+ ./scripts/todo-next-id.py --list-claims
./scripts/todo-next-id.py --check 12
- ./scripts/todo-next-id.py --root /path/to/other/repo
+ ./scripts/todo-next-id.py --finalize 12 short-slug
+ ./scripts/todo-next-id.py --root ~/Project Work
Options:
- --root <path> Repository root to scan (default: the parent of this script's directory).
+ --root <path> Project Work root (default: ~/Project Work).
--list Print every id currently in use (letter suffixes preserved), one per
line, sorted numerically then by letter, instead of the next free id.
+ --list-claims Print every "<id>-CLAIMED.md" placeholder stub and its age, for auditing
+ sessions that claimed an id and never wrote (or renamed) the real plan.
+ Diagnostic only; never deletes anything.
--check <id> Exit 1 with a message if <id> (e.g. "12" or "7a"; a leading "TODO-"
is tolerated) is already in use; exit 0 with a message if it's free.
- --allow-missing Treat an absent .work/todo/ as an empty tracker instead of an error.
+ --finalize <id> <slug>
+ Atomically rename that id's claim stub ("TODO-<id>-CLAIMED.md") to its
+ final plan filename ("TODO-<id>-<slug>.md") via a single os.rename() --
+ the ONE supported way to complete a claim. The caller must have already
+ written the real plan into the stub file in place before calling this.
+ Fails loudly (never falls back to a permissive outcome) if <id> is
+ malformed or lettered, if no stub exists for it, or if the final path
+ already exists. See do_finalize() and the module docstring above.
+ --allow-missing Treat an absent ~/Project Work/todos/juneau-release-manager/ as empty instead of an error.
--help, -h Show this help message.
Exit status:
- 0 Success (or --check found the id free).
+ 0 Success (or --check found the id free, or --list-claims ran, or --finalize renamed
+ the stub into place).
1 --check found the id already taken, or a malformed --check argument.
- 2 .work/todo/ does not exist under the resolved root (and --allow-missing was not given).
+ 2 ~/Project Work/todos/juneau-release-manager/ does not exist (and --allow-missing was not given).
+ 3 Could not claim an id after MAX_CLAIM_ATTEMPTS collisions (the tracker directory is
+ likely unwritable -- this is not normal contention).
+ 4 --finalize could not complete -- malformed/lettered id, malformed slug, no claim
+ stub found for that id, or the final path already exists. The printed message says
+ which; see do_finalize()'s docstring for why each of these fails loudly instead of
+ guessing.
"""
from __future__ import annotations
import argparse
+import os
import re
+import socket
import sys
+import time
+from datetime import datetime, timezone
from pathlib import Path
# ---------------------------------------------------------------------------------------
@@ -83,25 +132,60 @@ from pathlib import Path
# copy of this script; keep it that way so a fix lands once and is copied verbatim.
# ---------------------------------------------------------------------------------------
REPO_LABEL = "Juneau Release Manager"
-SKILL_NAME = "release-manager-todo-management"
+SKILL_NAME = "todo-and-waves"
+# Project letter in the id: Juneau "J", Console "C", Release App "R", IRS "I" -- this
+# copy's project uses "R". Numeric part is always 4 digits. This scanner matches only
+# TODO-R<digits>; ids from the other three projects are not counted here.
+ID_PROJECT_LETTER = "R"
+TRACKER_SLUG = "juneau-release-manager"
+
+PROJECT_WORK = Path.home() / "Project Work"
+
+# "[TODO-R0042]" or bare "TODO-R0042" in TODO.md prose. Digits follow the project letter R.
+# A trailing child-letter suffix (TODO-R0174a) is captured separately so it can be preserved
+# for --list/--check but ignored for numbering.
+#
+# The leading lookbehind rejects a qualified cross-repo citation -- e.g.
+# "otherproject:TODO-R0042" -- naming an id in ANOTHER repo's tracker, which must not
+# consume one here.
+TODO_TOKEN_RE = re.compile(r"(?<![\w:])TODO-R(\d+)([a-z0-9]*)\b")
-DEFAULT_REPO_ROOT = Path(__file__).resolve().parent.parent
+# Every lifecycle-state filename under ~/Project Work/todos/juneau-release-manager/ or finished/juneau-release-manager/.
+FILENAME_RE = re.compile(
+ r"^(?:TODO|READY|MAYBE|HOLD|FINISHED|CANCELLED|NEED_INPUT)-R(\d+)([a-z0-9]*)-.*\.md$"
+)
-# "[TODO-42]" or bare "TODO-42" in TODO.md prose. A trailing run of lowercase letters (the
-# child-letter suffix, possibly more than one for grandchildren like "17fa") is captured
-# separately so it can be preserved for --list/--check but ignored for numbering.
+# Migration to TODO-Rnnnn is complete. Do not scan unprefixed TODO-<digits>.
+LEGACY_TODO_TOKEN_RE = None
+LEGACY_FILENAME_RE = None
+
+# ---------------------------------------------------------------------------------------
+# Atomic id claiming. The old code computed "next = 1 + max(existing ids)" from a scan
+# and simply returned it -- a read, not a reservation. Two sessions racing between their
+# scan and their write get handed the SAME number (this is exactly how TODO-C0021 and
+# TODO-C0022 were each handed out twice on 2026-08-25). Fix: before returning a candidate,
+# atomically create a placeholder claim file for it with os.O_CREAT | os.O_EXCL, which the
+# OS guarantees only one caller can ever succeed at, even against another process racing
+# on the same directory at the same instant. Losing that race just means someone else got
+# there first, so bump the candidate and try again -- bounded, so a systemic problem
+# (read-only tracker, exhausted disk, ...) fails loudly instead of spinning forever.
#
-# The leading lookbehind rejects a qualified cross-repo citation -- "juneau:TODO-42",
-# "support-console:TODO-42" -- which names an id in ANOTHER repo's tracker and must not
-# consume one here. Ids are bare and per-repository, so without this the act of writing down
-# that another repo's item blocks you would silently burn a local id.
-TODO_TOKEN_RE = re.compile(r"(?<![\w:])TODO-(\d+)([a-z]*)\b")
+# The claim file's name IS the reservation: "TODO-<id>-CLAIMED.md" matches FILENAME_RE, so
+# it is picked up by every subsequent scan (this run's and everyone else's) exactly like a
+# real plan file, permanently retiring that number until someone deletes the stub.
+# ---------------------------------------------------------------------------------------
+CLAIM_SUFFIX = "-CLAIMED.md"
+MAX_CLAIM_ATTEMPTS = 500 # ids only ever increase; this many collisions in a row means
+ # something else is wrong, not real contention.
-# Every lifecycle-state filename directly under .work/todo/ or .work/todo/finished/.
-FILENAME_RE = re.compile(r"^(?:TODO|READY|MAYBE|FINISHED|CANCELLED)-(\d+)([a-z]*)-.*\.md$")
+# A short-slug for --finalize: lowercase letters/digits, single hyphens between words, no
+# leading/trailing hyphen. Deliberately stricter than the "*" in FILENAME_RE's "-.*\.md" --
+# this is the one place a HUMAN OR AGENT is asked to type a slug by hand, so it is validated
+# before it is trusted to become a filename, rather than accepted verbatim.
+SLUG_RE = re.compile(r"^[a-z0-9]+(?:-[a-z0-9]+)*$")
-def collect_ids(todo_dir: Path) -> tuple[set, set]:
+def collect_ids(todo_dir: Path, finished_dir: Path | None = None) -> tuple[set, set]:
"""
Scan every source described in the module docstring.
@@ -114,66 +198,251 @@ def collect_ids(todo_dir: Path) -> tuple[set, set]:
raw_ids = set()
numeric_ids = set()
+ def _record(regex, source, *, filename=False):
+ if regex is None:
+ return
+ matches = (regex.match(source),) if filename else regex.finditer(source)
+ for m in matches:
+ if not m:
+ continue
+ raw_ids.add(str(int(m.group(1))) + m.group(2))
+ numeric_ids.add(int(m.group(1)))
+
todo_md = todo_dir / "TODO.md"
if todo_md.is_file():
text = todo_md.read_text(encoding="utf-8")
- for m in TODO_TOKEN_RE.finditer(text):
- raw_ids.add(m.group(1) + m.group(2))
- numeric_ids.add(int(m.group(1)))
+ _record(TODO_TOKEN_RE, text)
+ _record(LEGACY_TODO_TOKEN_RE, text)
- for directory in (todo_dir, todo_dir / "finished"):
+ dirs = [todo_dir, finished_dir if finished_dir is not None else todo_dir / "finished"]
+ for directory in dirs:
if not directory.is_dir():
continue
for entry in directory.iterdir():
if not entry.is_file():
continue
- m = FILENAME_RE.match(entry.name)
- if not m:
- continue
- raw_ids.add(m.group(1) + m.group(2))
- numeric_ids.add(int(m.group(1)))
+ _record(FILENAME_RE, entry.name, filename=True)
+ _record(LEGACY_FILENAME_RE, entry.name, filename=True)
return raw_ids, numeric_ids
def sort_key(raw_id: str):
"""Sort key for a raw id string: numeric part first, then its letter suffix."""
- m = re.match(r"^(\d+)([a-z]*)$", raw_id)
+ m = re.match(r"^(\d+)([a-z0-9]*)$", raw_id)
if not m:
return (0, raw_id)
return (int(m.group(1)), m.group(2))
+def format_id(raw_id: str) -> str:
+ """Format a stored raw id as letter + 4-digit number + child suffix (e.g. R0447, R0445o)."""
+ m = re.match(r"^(\d+)([a-z0-9]*)$", raw_id)
+ if not m:
+ return f"{ID_PROJECT_LETTER}{raw_id}" if ID_PROJECT_LETTER else raw_id
+ return f"{ID_PROJECT_LETTER}{int(m.group(1)):04d}{m.group(2)}"
+
+
def normalize_check_id(raw: str) -> str | None:
- """Normalize a --check argument (optionally "TODO-"-prefixed) to a bare "<digits><letters>" id, or None if malformed."""
+ """Normalize a --check argument to a bare "<digits><letters>" id, or None if malformed.
+
+ Accepts `12`, `7a`, `TODO-12`, and (for prefixed projects) `R12` / `TODO-R12`.
+ """
candidate = raw.strip()
if candidate.upper().startswith("TODO-"):
candidate = candidate[len("TODO-"):]
- m = re.match(r"^(\d+)([a-zA-Z]*)$", candidate)
+ if (
+ ID_PROJECT_LETTER
+ and len(candidate) >= 2
+ and candidate[0].upper() == ID_PROJECT_LETTER
+ and candidate[1].isdigit()
+ ):
+ candidate = candidate[1:]
+ m = re.match(r"^(\d+)([a-z0-9]*)$", candidate, re.I)
if not m:
return None
- return m.group(1) + m.group(2).lower()
+ return str(int(m.group(1))) + m.group(2).lower()
+
+
+def claim_placeholder_text(formatted_id: str) -> str:
+ """
+ Body written into a freshly claimed "<id>-CLAIMED.md" file.
+
+ Deliberately loud and un-plan-shaped: a crashed session must leave something a human
+ (or the next agent) immediately recognizes as an abandoned reservation, not a
+ plausible-looking empty TODO that gets mistaken for real, unstarted work.
+ """
+ now = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
+ return (
+ f"# {formatted_id} -- CLAIMED (placeholder, NOT a real plan)\n"
+ "\n"
+ f"This id was atomically reserved by {Path(sys.argv[0]).name} so no other session\n"
+ "could be handed the same number. No plan has been written yet.\n"
+ "\n"
+ f"Claimed at: {now}\n"
+ f"Claimed by: pid {os.getpid()} on {socket.gethostname()}\n"
+ f"Command: {' '.join(sys.argv)}\n"
+ "\n"
+ "If you are that session: overwrite this file in place with the real plan (Goal /\n"
+ "Steps / Notes per the todo-and-waves skill), then run\n"
+ f"`{Path(sys.argv[0]).name} --finalize {formatted_id} <short-slug>` to rename it into place --\n"
+ "do NOT rename it yourself, and do NOT write the real plan under a brand-new filename\n"
+ "while leaving this stub behind: either produces a stub and a real plan coexisting under\n"
+ "the same id, which the tracker audit now flags immediately (unfinalized_claim_stub),\n"
+ "regardless of age.\n"
+ "\n"
+ "If you are not that session and this file is stale (the claiming session crashed\n"
+ "or never followed up): the id stays retired regardless -- never hand out\n"
+ f"{formatted_id} again. Confirm nothing references it, then delete this stub so the\n"
+ "tracker stops showing a phantom entry; the next call to this script will move past\n"
+ "it on its own. `--list-claims` lists every stub of this kind with its age.\n"
+ )
+
+
+def claim_next_id(todo_dir: Path, numeric_ids: set) -> int:
+ """
+ Atomically claim and return the next free numeric id.
+
+ Starts at 1 + max(existing ids) from the scan -- the same formula the old code used --
+ but then proves the claim by creating "TODO-<id>-CLAIMED.md" under todo_dir with
+ O_CREAT | O_EXCL. That create either succeeds (we own the id; no one else ever will)
+ or raises FileExistsError (someone else claimed it first), in which case we bump the
+ candidate and retry. See the module docstring for what happens to the stub afterward.
+ """
+ candidate = 1 + max(numeric_ids, default=0)
+ for _ in range(MAX_CLAIM_ATTEMPTS):
+ formatted = format_id(str(candidate))
+ claim_path = todo_dir / f"TODO-{formatted}{CLAIM_SUFFIX}"
+ try:
+ fd = os.open(claim_path, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o644)
+ except FileExistsError:
+ candidate += 1
+ continue
+ try:
+ with os.fdopen(fd, "w", encoding="utf-8") as handle:
+ handle.write(claim_placeholder_text(formatted))
+ except BaseException:
+ claim_path.unlink(missing_ok=True)
+ raise
+ return candidate
+ raise RuntimeError(
+ f"could not claim an id after {MAX_CLAIM_ATTEMPTS} consecutive collisions "
+ f"starting at {format_id(str(candidate))} -- check that {todo_dir} is writable."
+ )
+
+
+def do_finalize(todo_dir: Path, raw_id: str, raw_slug: str) -> int:
+ """
+ Perform the stub -> real-plan transition as a single atomic operation (os.rename), so no
+ calling convention -- a manual rename, or (the bug this closes) writing the real plan
+ under a brand-new filename and abandoning the stub -- can produce a window where both
+ names exist on disk. Returns a process exit code (0 on success).
+
+ Takes a final PATH (id + slug), not final CONTENT: the caller is expected to have already
+ composed the real plan by editing the claim stub's own file in place (any tool, any number
+ of edits) before calling this. Funnelling an entire plan's markdown through a CLI argument
+ would be unwieldy and buys no extra safety -- the risky step was never "writing content"
+ (safe under a name only the claimant can rename), it was "which filename ends up holding
+ the final state", which this makes the one atomic step the script itself owns.
+
+ Fails loudly on each of the following -- an ambiguous or unreadable state must not resolve
+ to the permissive outcome -- rather than guessing or silently overwriting:
+ - raw_id is malformed, or names a lettered child (claim stubs are only ever issued for a
+ bare numeric id -- todo-next-id.py never claims on behalf of a lettered child; see
+ claim_next_id()'s docstring). Either way no such stub could ever exist.
+ - raw_slug is not a valid slug per SLUG_RE.
+ - no claim stub exists at the computed path (never claimed, already finalized, or
+ removed by someone else).
+ - the final path already exists (refuses to overwrite it; the caller must resolve the
+ collision by hand -- see the printed message for the two ways that can legitimately
+ happen).
+ """
+ normalized = normalize_check_id(raw_id)
+ if normalized is None:
+ example = f"{ID_PROJECT_LETTER}0012" if ID_PROJECT_LETTER else "0001"
+ print(f"ERROR: '{raw_id}' is not a valid id (expected e.g. '{example}' or '7a').", file=sys.stderr)
+ return 4
+
+ id_match = re.match(r"^(\d+)([a-z0-9]*)$", normalized)
+ digits, suffix = id_match.group(1), id_match.group(2)
+ if suffix:
+ print(
+ f"ERROR: '{raw_id}' names a lettered child (suffix '{suffix}'). Claim stubs are only "
+ f"ever issued for a bare numeric id -- todo-next-id.py never claims on behalf of a "
+ f"lettered child (see claim_next_id()'s docstring) -- so no such stub can exist. "
+ f"Finalize the parent id instead.",
+ file=sys.stderr,
+ )
+ return 4
+
+ if not SLUG_RE.match(raw_slug):
+ print(
+ f"ERROR: '{raw_slug}' is not a valid slug (expected lowercase letters, digits, and "
+ f"single hyphens between words, e.g. 'short-slug').",
+ file=sys.stderr,
+ )
+ return 4
+
+ formatted = format_id(digits)
+ stub_path = todo_dir / f"TODO-{formatted}{CLAIM_SUFFIX}"
+ final_path = todo_dir / f"TODO-{formatted}-{raw_slug}.md"
+
+ if not stub_path.is_file():
+ print(
+ f"ERROR: no claim stub at {stub_path} -- nothing to finalize. Either {formatted} was "
+ f"never claimed, was already finalized, or the stub was removed by someone else. Check "
+ f"{todo_dir} (or run --list-claims) before re-claiming.",
+ file=sys.stderr,
+ )
+ return 4
+
+ if final_path.exists():
+ print(
+ f"ERROR: {final_path} already exists -- refusing to overwrite it. If {formatted} was "
+ f"already finalized under this slug, there is nothing left to do here, but the stub at "
+ f"{stub_path} is now an orphan the tracker audit will flag (unfinalized_claim_stub) -- "
+ f"delete it once you've confirmed the existing file is the complete, correct plan. "
+ f"Otherwise, {final_path} belongs to something else; pick a different slug.",
+ file=sys.stderr,
+ )
+ return 4
+
+ os.rename(stub_path, final_path)
+ print(str(final_path))
+ return 0
def main():
parser = argparse.ArgumentParser(
- description=f"Compute the next free .work/todo/ TODO id for {REPO_LABEL} (see @{SKILL_NAME}).",
+ description=f"Compute the next free TODO id for {REPO_LABEL} under ~/Project Work (see @{SKILL_NAME}).",
epilog=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter,
)
- parser.add_argument("--root", metavar="PATH", help="Repository root to scan (default: parent of this script's directory).")
+ parser.add_argument("--root", metavar="PATH", help="Project Work root (default: ~/Project Work).")
parser.add_argument("--list", action="store_true", help="Print every id currently in use, sorted.")
+ parser.add_argument(
+ "--list-claims",
+ action="store_true",
+ help="List '<id>-CLAIMED.md' placeholder stubs (claimed but never turned into a "
+ "real plan), with their age. Diagnostic only -- never deletes anything.",
+ )
parser.add_argument("--check", metavar="ID", help="Exit non-zero if ID is already taken.")
- parser.add_argument("--allow-missing", action="store_true", help="Treat an absent .work/todo/ as empty instead of an error.")
+ parser.add_argument(
+ "--finalize",
+ nargs=2,
+ metavar=("ID", "SLUG"),
+ help="Atomically rename ID's claim stub to 'TODO-<ID>-<SLUG>.md' (see do_finalize()). "
+ "The real plan must already be written into the stub file in place.",
+ )
+ parser.add_argument("--allow-missing", action="store_true", help="Treat an absent tracker as empty instead of an error.")
args = parser.parse_args()
- repo_root = Path(args.root).resolve() if args.root else DEFAULT_REPO_ROOT
- todo_dir = repo_root / ".work" / "todo"
+ project_work = Path(args.root).expanduser().resolve() if args.root else PROJECT_WORK
+ todo_dir = project_work / "todos" / TRACKER_SLUG
+ finished_dir = project_work / "finished" / TRACKER_SLUG
- # Always announce the tree actually scanned. Ids are per-repository and bare, so
- # "TODO-5" is a different item in each repo; a wrong-tree run must not look identical
- # to a right-tree one. stderr, so `NEXT=$(./scripts/todo-next-id.py)` still works.
- print(f"[{REPO_LABEL}] scanning {todo_dir}", file=sys.stderr)
+ # Always announce the tree actually scanned. stderr, so NEXT=$(./scripts/todo-next-id.py) still works.
+ print(f"[{REPO_LABEL}] scanning {todo_dir} + {finished_dir}", file=sys.stderr)
if not todo_dir.is_dir():
if not args.allow_missing:
@@ -187,26 +456,56 @@ def main():
return 2
print(f"WARNING: {todo_dir} does not exist; treating as empty (--allow-missing).", file=sys.stderr)
- raw_ids, numeric_ids = collect_ids(todo_dir)
+ if args.finalize is not None:
+ return do_finalize(todo_dir, args.finalize[0], args.finalize[1])
+
+ if args.list_claims:
+ claim_re = re.compile(rf"^TODO-{re.escape(ID_PROJECT_LETTER)}(\d+){re.escape(CLAIM_SUFFIX)}$")
+ stubs = []
+ if todo_dir.is_dir():
+ stubs = sorted(
+ (entry for entry in todo_dir.iterdir() if entry.is_file() and claim_re.match(entry.name)),
+ key=lambda entry: entry.name,
+ )
+ if not stubs:
+ print("No claim stubs found.")
+ return 0
+ for entry in stubs:
+ age_minutes = (time.time() - entry.stat().st_mtime) / 60
+ age = f"{age_minutes:.0f}m" if age_minutes < 60 else f"{age_minutes / 60:.1f}h"
+ print(f"{entry.name}\t(age: {age})\t{entry}")
+ return 0
+
+ raw_ids, numeric_ids = collect_ids(todo_dir, finished_dir)
if args.check is not None:
normalized = normalize_check_id(args.check)
if normalized is None:
- print(f"ERROR: '{args.check}' is not a valid id (expected e.g. '12' or '7a').", file=sys.stderr)
+ example = f"{ID_PROJECT_LETTER}0012" if ID_PROJECT_LETTER else "0001"
+ print(f"ERROR: '{args.check}' is not a valid id (expected e.g. '{example}' or '7a').", file=sys.stderr)
return 1
if normalized in raw_ids:
- print(f"TAKEN: {normalized} is already in use.")
+ print(f"TAKEN: {format_id(normalized)} is already in use.")
return 1
- print(f"FREE: {normalized} is not currently in use.")
+ print(f"FREE: {format_id(normalized)} is not currently in use.")
return 0
if args.list:
for raw_id in sorted(raw_ids, key=sort_key):
- print(raw_id)
+ print(format_id(raw_id))
return 0
- next_id = 1 + max(numeric_ids, default=0)
- print(next_id)
+ if not todo_dir.is_dir():
+ # Only reachable via --allow-missing (the hard-error branch above already
+ # returned otherwise). Create it now so the claim below has somewhere to land.
+ todo_dir.mkdir(parents=True, exist_ok=True)
+
+ try:
+ next_id = claim_next_id(todo_dir, numeric_ids)
+ except RuntimeError as exc:
+ print(f"ERROR: {exc}", file=sys.stderr)
+ return 3
+ print(format_id(str(next_id)))
return 0
diff --git a/scripts/todo-status-audit.py b/scripts/todo-status-audit.py
index 7472f77c1e..f12ac0705f 100755
--- a/scripts/todo-status-audit.py
+++ b/scripts/todo-status-audit.py
@@ -1,32 +1,33 @@
#!/usr/bin/env python3
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-"""
-Best-effort status/header consistency pre-filter for this repository's .work/todo/ plan files.
+# ***************************************************************************************************************************
+# * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an
+# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# * specific language governing permissions and limitations under the License.
+# ***************************************************************************************************************************
+# The docstring is a raw string so the declared-counts example below can show a real regex (with its
+# backslash escapes) verbatim, instead of a de-escaped one that would not work if copied out.
+r"""
+Best-effort status/header consistency pre-filter for this project's plan files under ~/Project Work/todos/juneau-release-manager/.
Repo-agnostic: the repository root is derived from this file's own location
(<root>/scripts/todo-status-audit.py -> <root>), never hardcoded, so the same body works in
every repository that adopts the convention. Only the REPO_LABEL / SKILL_NAME constants
below and the license header differ between copies.
-Checks every TODO-<id>-*.md / READY-<id>-*.md / MAYBE-<id>-*.md file directly under .work/todo/
-(FINISHED-/CANCELLED-*.md archives are explicitly out of scope -- per this repo's
-TODO-management skill, "status line is not required in FINISHED archives") against that
-skill's "Per-file `Current status:` and `Complexity:` header" rules, and flags candidate
-inconsistencies. This is a PRE-FILTER, not a validator: it flags candidates for a human
-(or agent) to look at, and will not catch everything on format-drifted files -- tolerant,
+Checks every TODO-/READY-/MAYBE-/HOLD-/NEED_INPUT-*.md file directly under
+~/Project Work/todos/juneau-release-manager/ (FINISHED-/CANCELLED-*.md archives are explicitly
+out of scope -- per this repo's TODO-management skill, "status line is not required in FINISHED
+archives")
+against that skill's "Per-file `Current status:` and `Complexity:` header" rules, and flags
+candidate inconsistencies. This is a PRE-FILTER, not a validator: it flags candidates for a
+human (or agent) to look at, and will not catch everything on format-drifted files -- tolerant,
best-effort markdown-header parsing throughout.
Checks performed (each file may accumulate multiple flags):
@@ -34,17 +35,32 @@ Checks performed (each file may accumulate multiple flags):
- missing_complexity_header No "Complexity:" line found anywhere in the file.
- status_header_misplaced "Current status:" appears at or after the first "## " section
heading (it must come before it, per the skill's "Placement" rule).
- - unrecognized_status_phrase The status text doesn't start with one of the skill's documented
- phrases for this file's prefix (TODO/READY: "Waiting for user
- input on open questions.", "Ready to execute.", "In progress.";
- MAYBE: must start with "Parked"). Free-form variants that legitimately
- extend a recognized prefix (e.g. "Ready to execute (all items
- independently actionable)." ) are NOT flagged -- only prefix
- mismatches are.
+ - empty_status_value The "Current status:" field is present but has no real content once
+ markdown/punctuation decoration is stripped (e.g. the header line was
+ never filled in). Deliberately permissive about WHAT the value says --
+ the trackers have settled on free-form descriptive prose ("Done.",
+ "Blocked on the Juneau pin bump.", "**COMPLETE (2026-07-16).**", etc.)
+ rather than a fixed phrase set, so only a structurally-absent value is
+ flagged here. (This replaces a former unrecognized_status_phrase check
+ that enumerated a fixed set of exact phrases and consequently flagged
+ the majority of files once the trackers settled on descriptive prose
+ instead of that enumerated set -- an enumeration is exactly what rots.)
- ready_but_has_open_questions
Status starts with "Ready to execute" but the file still has a
"## Open questions" section containing at least one numbered item
whose text doesn't look answered/resolved.
+ - ready_but_wave_already_accepted
+ Status starts with "Ready to execute" and the file's header region
+ (everything before the first "## " heading) declares membership in a
+ WAVE-nnnn that the wave board records as ALREADY ACCEPTED and whose
+ WAVE file is gone. The item is claiming to be queued behind a wave
+ that has already been accepted and pushed, which cannot both be
+ true: either the item shipped in that wave and was never archived
+ (the "shipped but not archived" drift this check exists for), or it
+ was dropped from the wave and its header reference is stale. Both
+ readings need an operator edit, so the flag is actionable either way.
+ See "Wave-acceptance cross-check" below for why this is a
+ cross-record consistency check and not a guess at what shipped.
- ready_prefix_waiting_status A READY-*.md file whose status still says "Waiting for user input"
(READY files should have no unresolved open questions left).
- todo_prefix_marked_ready A TODO-*.md file (not yet renamed) whose status already says "Ready
@@ -53,6 +69,201 @@ Checks performed (each file may accumulate multiple flags):
- parked_status_wrong_prefix A TODO-*.md/READY-*.md file whose status starts with "Parked" (that
wording is reserved for MAYBE-*.md files).
- maybe_prefix_non_parked A MAYBE-*.md file whose status does NOT start with "Parked".
+ - on_hold_status_wrong_prefix A TODO-*.md/READY-*.md file whose status starts with "On hold" (that
+ wording is reserved for HOLD-*.md files).
+ - hold_prefix_non_on_hold A HOLD-*.md file whose status does NOT start with "On hold".
+ - counts_malformed A "Counts:" line exists but its value is neither the literal "None." nor
+ the declared-counts grammar `<repo>:<kind>:<pathspec>[:<regex>]` = <int>.
+ Kept distinct from an absent line for the same reason the Key-symbols
+ tri-state is: "the author wrote nothing" and "the author wrote something
+ that did not parse" are different facts.
+ - counts_mismatch A declared count written with '=' does not equal the count re-derived
+ from source THIS RUN. Often routine re-baselining: an '=' declaration is
+ usually a BASELINE, and the item's own work is what changed the number.
+ - counts_below_floor A declared count written with '>=' re-derived BELOW its floor. Kept
+ distinct from counts_mismatch because the operator response differs: a
+ floor breach is a regression to investigate and never a number to
+ correct, where an equality mismatch frequently is exactly that.
+ - counts_scope_empty A declared count's pathspec resolves to ZERO files, so the declaration is
+ no longer anchored to anything and its number cannot be tested. Flagged
+ loudly rather than passing vacuously -- this is the specific defence
+ against a declaration that outlives the code it described.
+ - counts_unverifiable A declared count names a repo that cannot be resolved to a git worktree
+ with a readable revision (missing checkout, no git, unreadable HEAD).
+ - stale_claim An id-reservation stub ("TODO-<id>-CLAIMED.md", written by
+ todo-next-id.py's claim_next_id() to atomically reserve an id) is
+ older than STALE_CLAIM_AGE_HOURS with no real plan ever written in
+ its place -- the claiming session most likely crashed. See "Claim-
+ stub handling" below for what the stub records and why this check
+ is age-based, not liveness-based. Claim stubs are otherwise EXEMPT
+ from every other check above -- see that section for why.
+ - unfinalized_claim_stub A "TODO-<id>-CLAIMED.md" reservation stub (see stale_claim just
+ above) coexists with a real (non-stub) plan file under the SAME
+ numeric id. Deliberately NOT age-gated, unlike stale_claim: an id
+ must never simultaneously name a live reservation and a finished
+ plan, so this is wrong the instant it exists, no matter how fresh
+ the stub is. See "Claim-stub handling" below for how this happens
+ and how to resolve it.
+
+NEED_INPUT-*.md files (blocked on operator answers, per the skill's prefix table) are scanned like
+TODO-/READY- files for the structural checks above, but are deliberately not subject to the
+Parked-/On-hold-prefix-mismatch checks, or to ready_but_wave_already_accepted, which are all
+TODO/READY-specific signals.
+
+Claim-stub handling
+--------------------
+todo-next-id.py's claim_next_id() reserves an id by creating "TODO-<id>-CLAIMED.md" BEFORE any
+plan content exists (see that module's docstring and claim_placeholder_text()). The stub is
+deliberately un-plan-shaped -- headed "CLAIMED (placeholder, NOT a real plan)", with no "Current
+status:" or "Complexity:" line -- so a human immediately recognizes an abandoned reservation
+instead of mistaking it for a plausible-looking empty TODO. That filename satisfies FILENAME_RE by
+design: the whole point of the claim is that it counts as "taken" by every future scan, this
+process's and everyone else's, exactly like a real plan file.
+
+Treating a claim stub as an ordinary plan file was itself the bug this handling exists to fix: it
+can never carry a "Current status:" or "Complexity:" line, so EVERY successful id reservation
+flagged missing_status_header and missing_complexity_header for as long as the reservation
+lasted -- which is by design, not a defect, for however long the caller has not yet written the
+real plan. A claim stub (recognized separately via CLAIM_STUB_RE) is therefore exempt from every
+plan-file check above and instead audited by audit_claim_stub() for two things: whether it
+coexists with a real plan under the same id (unfinalized_claim_stub, unconditional -- see below),
+and whether the reservation looks abandoned by age (stale_claim).
+
+unfinalized_claim_stub fires the instant a real (non-stub) file is found sharing the stub's exact
+numeric id -- group_claims_by_numeric_id() below builds that grouping once per run from the same
+file list find_plan_files() already returned, so this costs one extra pass over filenames already
+in hand, not a second directory scan. This is what tonight's actual incident looked like on disk:
+todo-next-id.py's finalize step was, at the time, only a documented convention ("overwrite this
+file in place ... then rename it"), not code -- an agent read that instruction, reconsidered
+mid-task, and wrote the real plan under a fresh slugged filename instead of renaming the stub,
+leaving both on disk. todo-next-id.py's `--finalize` (see that module) now performs the rename as
+one atomic operation the SCRIPT owns, closing that window; this check is the backstop for any
+caller that still bypasses it -- deliberately not age-gated, because the coexistence is wrong
+immediately, not just once it has sat around long enough to look abandoned like a lone stub does.
+
+What the stub records (claim_placeholder_text() in todo-next-id.py): a "Claimed at:" ISO-8601 UTC
+timestamp, and a "Claimed by: pid <n> on <hostname>" line. The pid and hostname are NOT used to
+decide staleness here, despite being logged for exactly that purpose in the stub's own "if this
+file is stale" paragraph: the recorded pid belongs to the one-shot todo-next-id.py CLI invocation
+itself, which prints the id and exits within a fraction of a second of writing the stub -- in the
+SUCCESSFUL case every bit as much as in the crashed one. Checking whether that pid is still
+running would therefore report "dead" for essentially every claim, live or not, moments after it
+was made: a guaranteed false positive, not a hardened signal. A recycled pid or a stub written on
+another host (the failure modes liveness checks are usually distrusted for) would only make such a
+check less reliable on top of that; the structural problem above is sufficient on its own to rule
+it out. Age -- the stub's file mtime, the same signal todo-next-id.py's own --list-claims already
+reports -- is therefore the only input to staleness.
+
+STALE_CLAIM_AGE_HOURS is chosen so a normal, in-progress reservation never trips it: per
+todo-next-id.py's own docstring the caller claims the id and writes the real plan into that same
+file in one sitting ("overwrite it in place ... and then run `--finalize`"), and nothing in this
+tracker's corpus -- including its largest plan files, drafted in a single session -- shows that
+sitting outlasting a working day. The threshold is comfortably longer than any such session,
+including one paused for a lunch break, while still catching a stub that has survived past the
+session that claimed it.
+
+Wave-acceptance cross-check (ready_but_wave_already_accepted)
+------------------------------------------------------------
+This is the only check that reads anything outside the scanned file, and it deliberately reads
+the OTHER tracker record rather than the source tree. It exists because a whole wave of items
+shipped to master and stayed in the active tracker for a day, silently, still saying "Ready to
+execute ... do not farm until the operator starts the WAVE queue" -- and the stale claim then
+propagated into downstream planning, which built serialization constraints against items that
+had already shipped.
+
+It is a CROSS-RECORD CONSISTENCY check, structurally the same as the prefix-vs-status checks
+above: two records that the convention requires to agree (the wave board's accepted-wave log,
+and the item's own "I am queued behind wave N" claim) are compared, and a contradiction is
+flagged. It does NOT try to infer from prose which code an item was supposed to produce and
+then look for that code in the tree. That was evaluated and rejected -- see the notes at
+closed_out_waves() and header_region() -- because deliverables live in free prose with no
+machine-readable field, and an item that MODIFIES an existing symbol names a symbol that
+already resolves, which makes symbol-existence probing a false-positive generator.
+
+Three independent conditions must all hold, so the check is inert wherever any of them is
+absent (no board file, no wave reference, or a status that isn't a pre-kickoff claim):
+ 1. A wave board (WAVE.md, resolved as a sibling of todos/, or --board) records the wave as
+ accepted, AND no live waves/WAVE-nnnn*.md file remains. Per the board's own documented
+ lifecycle the WAVE file is deleted at accept, so requiring BOTH a prose "accepted" mention
+ and the file's absence keeps a merely-discussed wave from reading as accepted.
+ 2. The item names that WAVE-nnnn in its header region (before the first "## " heading), i.e.
+ as part of its own identity, not as a cross-reference buried in a body section. Items that
+ merely mention another wave's members ("must be serialized against ...") reference the wave
+ deep in the body and are not flagged.
+ 3. The item's status still starts with "Ready to execute" (the same narrow, already-documented
+ prefix the ready_but_has_open_questions check uses) -- i.e. it still claims not to have run.
+
+Declared counts (counts_* checks)
+--------------------------------
+The second check family that reads outside the scanned file, and the only one that reads source.
+It exists because numeric claims in plan files rot invisibly: a count written once ("ten call
+sites", "23 archives", "146 `em` declarations") is prose from the moment it is typed, and a reader
+has no way to tell a still-true number from a stale one without counting the thing again. Seven
+separate counts were re-derived across these trackers on 2026-08-25 and every one came back
+different from what the tracker said. None was found by reading; each was found by counting again.
+
+This check does NOT infer what to count. That was the failure mode of the rejected
+deliverable-probing approach (see closed_out_waves() and header_region()): prose does not say
+what it is asserting, so a tool guessing at it manufactures false positives. Instead the ITEM
+DECLARES what to count and what it expects, exactly the way `Key symbols:` has the item declare
+its new symbols, and the audit does the counting. A declaration is a `Counts:` header line:
+
+ Counts: `console:files:src/main/java/**/rest/**/*.java:requiredConfirmation\(|armPhrase\(` = 10
+
+read as "<repo token>:<kind>:<pathspec>[:<regex>] <op> <expected integer>", where <op> is `=` or
+`>=`. Equality is the default and is the right shape for a BASELINE -- a fact about the state before
+the item's work, which the item exists to change. `>=` declares a FLOOR ("this must never drop below
+N'"), which is what an INVARIANT actually means. The floor form exists because an equality test over a
+growable population is actively harmful: benign growth flags, and the operator is then invited to
+"correct" the number, which is how a regression guard gets deleted by accident. The field may repeat,
+one declaration per line; the literal value `None.` declares "nothing here is mechanically countable".
+Kinds: `matches` (total regex occurrences), `files` (files with at least one occurrence), `paths`
+(files matching the pathspec, no regex). Repo tokens: the three tracker repos plus `board` for
+~/Project Work itself. Full grammar and rationale: the "Declared counts" section of @todo-and-waves.
+
+Three properties keep a STALE declaration from passing, which is the risk the convention had to be
+designed against -- a declaration nobody re-checks is no better than the sentence it replaced:
+
+ 1. Every number is re-derived from source on every run. The declared integer is only ever the
+ EXPECTED side of a comparison; it is never read as an answer, never cached to disk, and no
+ derived result is written back anywhere. There is no state for a previous run to leave behind.
+ (Within a single run, blob reads and path listings for one immutable revision are memoized --
+ that is deduplication of identical reads of the same commit, not a cached verdict.)
+ 2. A pathspec that resolves to zero files is counts_scope_empty, NOT a pass. This is the case
+ that would otherwise rot silently: a file renamed or folded away leaves a declaration whose
+ pattern trivially matches nothing, and comparing 0 to 0 would report success. One of the seven
+ real failures (a claim about N per-letter FINISHED archives, after those archives had been
+ folded into one file and deleted) is exactly this shape. Because a zero-file scope is always
+ loud, a `paths ... = 0` declaration is deliberately NOT expressible -- "this glob matches
+ nothing" and "this glob has rotted" are indistinguishable, so the convention refuses the claim
+ instead of guessing which one it is.
+ 3. The grammar is strict, so a half-edited declaration becomes counts_malformed rather than being
+ skipped. A line that no longer parses is reported, never quietly ignored.
+
+DELIBERATE NON-GOAL: a floor set far below reality is NOT detected. `>= 1` over a population of 500
+never fails, and nothing here reports that. Detecting it would need a threshold on how much slack is
+legitimate, and any such threshold recreates the exact failure the floor form was added to remove --
+it would flag populations that had merely grown, and invite raising the floor to match reality, which
+is re-baselining a guard. The one unambiguous case IS refused: `>= 0` is rejected at parse time,
+because it holds for every possible derivation and so asserts nothing. Past that, a floor's slack is
+a judgment the author makes and the audit does not second-guess it. What still protects a floor is
+property 2 above: a pathspec that has rotted to zero matches is counts_scope_empty -- never a pass,
+and never the misreading "well, 0 is below the floor, so it fails as a regression".
+
+Counts resolve against COMMITTED CONTENT (`HEAD`) in the named repo, never the working tree. Two
+reasons, one of them paid for the hard way: a plan file citing working-tree-only files (three .py
+files present locally and in nobody's HEAD) produced a blocked item on 2026-08-25 that two
+adversarial reviewers both read past; and these repos are cloned per-WAVE, so a tree-resolved count
+means the same declaration verifies differently in juneau-3 than in juneau-7 -- which is a coin
+flip, not a re-derivation. HEAD is the one referent the operator, a subagent and a reviewer share.
+The `board` token is the single documented exception: ~/Project Work is not a git repository, so
+its counts necessarily resolve against the filesystem. That is sound rather than a concession --
+the board is never cloned, so the per-clone divergence that makes tree-resolution wrong for a repo
+cannot arise, and there is no HEAD to resolve against in the first place.
+
+Absent is silence. No `Counts:` line means no claim, and nothing is flagged -- the field is
+optional by design (see the skill), and a check that fired on absence would flag every
+pre-convention file, which is precisely the cry-wolf failure this script was rewritten to remove.
A MISSING scan directory is a hard error (exit 2). An EMPTY-but-present one is a clean pass
(exit 0). The original version conflated the two and returned 0 for both, so pointing the
@@ -62,14 +273,29 @@ trap as running `rg` over the gitignored .work/ without --no-ignore.
Usage:
./scripts/todo-status-audit.py
./scripts/todo-status-audit.py --verbose
- ./scripts/todo-status-audit.py --root /path/to/other/repo
+ ./scripts/todo-status-audit.py --root ~/Project Work
./scripts/todo-status-audit.py --dir /path/to/alternate/todo/dir
+ ./scripts/todo-status-audit.py --board /path/to/WAVE.md
+ ./scripts/todo-status-audit.py --repo-root juneau=/path/to/juneau
+ ./scripts/todo-status-audit.py --no-counts
Options:
- --root <path> Repository root; scans <root>/.work/todo/ (default: parent of this
- script's directory). Ignored if --dir is given.
+ --root <path> Project Work root (default: ~/Project Work). Ignored if --dir is given.
--dir <path> Exact directory to scan, overriding --root. Non-recursive -- only *.md
files directly in this directory are considered.
+ --board <path> Wave board file for the ready_but_wave_already_accepted check. Defaults to
+ WAVE.md beside the scanned tracker's parent (i.e. <project-work>/WAVE.md).
+ If the board cannot be found, that one check is skipped and every other
+ check still runs.
+ --repo-root <token>=<path>
+ Override where a `Counts:` repo token resolves to. Repeatable. Without it a
+ token naming THIS repo resolves to this script's own repo root (a filesystem
+ fact, not a guess) and any other token to its default $HOME-relative path.
+ --no-counts Do not re-derive declared counts. Grammar errors are still reported --
+ counts_malformed needs no source access -- but counts_mismatch /
+ counts_scope_empty / counts_unverifiable are not evaluated. For running
+ without the checkouts (or without git) present; NOT a way to silence a
+ failing count.
--verbose, -v Also print files that passed every check (default: only print flagged files).
--help, -h Show this help message.
@@ -83,28 +309,47 @@ from __future__ import annotations
import argparse
import re
+import subprocess
import sys
+import time
from pathlib import Path
# ---------------------------------------------------------------------------------------
-# The ONLY repo-specific values in this file. Everything below is identical across every
-# copy of this script; keep it that way so a fix lands once and is copied verbatim.
+# The ONLY repo-specific values in this file: REPO_LABEL / TRACKER_SLUG / FILENAME_RE (which
+# embeds this project's id letter) and CLAIM_STUB_RE (the same letter, one more place it
+# appears). Everything below is identical across every copy of this script; keep it that way
+# so a fix lands once and is copied verbatim.
# ---------------------------------------------------------------------------------------
REPO_LABEL = "Juneau Release Manager"
-SKILL_NAME = "release-manager-todo-management"
+SKILL_NAME = "todo-and-waves"
+TRACKER_SLUG = "juneau-release-manager"
+
+PROJECT_WORK = Path.home() / "Project Work"
-DEFAULT_REPO_ROOT = Path(__file__).resolve().parent.parent
+FILENAME_RE = re.compile(r"^(TODO|READY|MAYBE|HOLD|NEED_INPUT)-R\d+[a-z0-9]*-.*\.md$")
-FILENAME_RE = re.compile(r"^(TODO|READY|MAYBE)-\d+[a-z]*-.*\.md$")
+# An id-reservation stub written by todo-next-id.py's claim_next_id() -- see the module
+# docstring's "Claim-stub handling" section. Always a bare numeric TODO- id (todo-next-id.py
+# never claims on behalf of a lettered child) with no slug -- "-CLAIMED.md" is the whole
+# filename tail, not something a claimant appends -- so this matches a strict subset of
+# FILENAME_RE's TODO- branch. A claim stub is still found by find_plan_files() (the
+# reservation must keep counting as "taken"), but is routed to audit_claim_stub() below
+# instead of audit_file(): it is a distinct file kind, not a malformed plan file.
+CLAIM_STUB_RE = re.compile(r"^TODO-R\d+-CLAIMED\.md$")
-# Each trailing capture used to be "\s*(.*)$"/"\s+(.*)$" -- a mandatory-or-optional whitespace run directly
-# adjacent to a "rest of line" run, both of which can consume the same space characters. That overlap gives
-# the regex engine multiple equivalent ways to split a run of whitespace between the two groups, which is
-# super-linear (not just linear) to explore on a failing match. Every caller below already re-derives the
-# "meaningful" text with .strip() (or a case-insensitive substring test on the stripped text), so the leading
-# whitespace only ever needs to be consumed ONCE, deterministically -- as a single non-repeating "\s" where a
-# separator is mandatory (SECTION_HEADING_RE, NUMBERED_ITEM_RE), or dropped entirely where it was already
-# optional and redundant with the caller's own .strip() (STATUS_LINE_RE, COMPLEXITY_LINE_RE).
+# The leading numeric id shared by a claim stub and any real (non-stub) plan filename for the
+# same numbered item -- letter suffix and slug excluded. Used only to group files by id for the
+# unfinalized_claim_stub check below (see group_claims_by_numeric_id()); never for numbering,
+# which is todo-next-id.py's job, not this script's.
+ID_FROM_FILENAME_RE = re.compile(r"^(?:TODO|READY|MAYBE|HOLD|NEED_INPUT)-R(\d+)")
+
+# Each pattern's trailing "whatever follows" group used to be preceded by its own separate `\s*`/`\s+`
+# quantifier (e.g. `\s*(.*)$`), which is superlinear-backtracking-prone: `\s` and `.` overlap (both match a
+# space/tab), so on a non-matching line the engine can retry the split point between the two quantifiers
+# across every position before giving up. Every caller already `.strip()`s the captured group (or, for
+# NUMBERED_ITEM_RE, never inspects its content at all -- only the match's start position matters), so the
+# extra quantifier was redundant: dropping it (STATUS_LINE_RE/COMPLEXITY_LINE_RE) or collapsing it to a single
+# mandatory character (SECTION_HEADING_RE/NUMBERED_ITEM_RE) removes the overlap without changing what matches.
STATUS_LINE_RE = re.compile(r"^\s*Current status:(.*)$", re.IGNORECASE | re.MULTILINE)
COMPLEXITY_LINE_RE = re.compile(r"^\s*Complexity:(.*)$", re.IGNORECASE | re.MULTILINE)
SECTION_HEADING_RE = re.compile(r"^##\s(.*)$", re.MULTILINE)
@@ -116,25 +361,465 @@ NUMBERED_ITEM_RE = re.compile(r"^\s*\d+[.)]\s(.*)$", re.MULTILINE)
# catch. Still best-effort: an explicit negation like "not resolved" reads as resolved.
RESOLVED_MARKER_RE = re.compile(r"\b(?:resolved|answered|decided)\b")
-# Recognized status prefixes (case-insensitive, checked with str.startswith after lowercasing) per
-# the skill's "Status wording rules" -- kept separate from TODO/READY vs MAYBE since the two file
-# families use disjoint wording.
-READY_TO_EXECUTE_STATUS = "ready to execute"
-TODO_READY_STATUS_PREFIXES = (
- "waiting for user input on open questions",
- READY_TO_EXECUTE_STATUS,
- "in progress",
-)
+# The corpus commonly wraps the start of a status value in markdown emphasis/quoting (bold
+# "**Done.**", a leading backtick, a leading "-"/em dash before "Parked" or "On hold"). Stripping
+# that decoration before a phrase-prefix comparison is a generic, content-agnostic normalization --
+# it encodes which PUNCTUATION is decoration, not which WORDS are expected -- so it doesn't
+# reintroduce the kind of phrase enumeration that rotted the old unrecognized_status_phrase check.
+STATUS_DECORATION_RE = re.compile(r"^[\s*_`\"'>-]+")
+
+# Recognized status-prefix phrases (case-insensitive, checked with str.startswith after
+# normalize_status_text()) for the specific lifecycle-mismatch signals below -- narrow, targeted
+# substrings tied 1:1 to a documented filename-prefix convention, not an exhaustive enumeration of
+# every valid status wording (that enumeration is exactly what status_value_is_empty() replaces).
+READY_TO_EXECUTE_PREFIX = "ready to execute"
MAYBE_STATUS_PREFIX = "parked"
+HOLD_STATUS_PREFIX = "on hold"
+
+# --- Wave-acceptance cross-check inputs (see the module docstring) ----------------------------
+# The board file and the live-WAVE-file directory, both resolved relative to the Project Work
+# root rather than hardcoded, so the check is inert (not wrong) when pointed at a scratch tree.
+BOARD_FILENAME = "WAVE.md"
+WAVES_DIRNAME = "waves"
+
+# A wave id is a fixed-width literal token, which is the whole reason this check can be exact:
+# there is no prose to interpret on either side of the comparison, just the same token appearing
+# in two records that the convention requires to agree.
+WAVE_ID_RE = re.compile(r"WAVE-\d{4}")
+
+# "accepted" is matched on a word boundary on the SAME board line as the wave id. Deliberately
+# not a phrase enumeration: the board's accepted-wave log is free prose ("was accepted and pushed
+# 2026-08-25", "WAVE-0002 ... was accepted and pushed"), so pinning an exact sentence would rot
+# the way the removed unrecognized_status_phrase check did. The negation guard below plus the
+# live-WAVE-file check in closed_out_waves() are what keep a loose word match from over-firing.
+BOARD_ACCEPTED_RE = re.compile(r"\baccepted\b", re.IGNORECASE)
+
+# A board line saying a wave is NOT accepted must not read as accepted. RESOLVED_MARKER_RE above
+# documents the same hazard and accepts it; here the cost of being wrong is flagging a
+# CORRECTLY-queued item, i.e. exactly the cry-wolf failure this script was rewritten to remove,
+# so the common negations are matched explicitly instead.
+BOARD_NOT_ACCEPTED_RE = re.compile(r"\b(?:not|never|un-?|isn't|aren't|wasn't|weren't)\s*accepted\b", re.IGNORECASE)
+
+# --- Claim-stub inputs (see the module docstring's "Claim-stub handling" section) --------------
+# Tolerant parses of the two fields claim_placeholder_text() (todo-next-id.py) writes, used only
+# to name the claimant/timestamp in a flag's detail message -- NOT to decide staleness; an
+# unparseable or absent field here still leaves age (file mtime) as the deciding signal.
+CLAIM_CLAIMED_AT_RE = re.compile(r"^Claimed at:\s*(\S+)", re.MULTILINE)
+CLAIM_CLAIMED_BY_RE = re.compile(r"^Claimed by:\s*pid\s+(\d+)\s+on\s+(\S+)", re.MULTILINE)
+
+# How old a claim stub must be, by file mtime, before it is flagged stale rather than treated as
+# a normal in-progress reservation. See "Claim-stub handling" in the module docstring for why
+# this is age alone (not the pid/hostname the stub also records) and why 24h specifically.
+STALE_CLAIM_AGE_HOURS = 24
+
+# --- Declared-counts inputs (see the module docstring) ----------------------------------------
+# Parsed with the same tolerant "^\s*<Field>:" shape as the other header fields, and deliberately
+# NOT restricted to the header region: a count declaration is a checkable assertion wherever it is
+# written, and the check's own strictness (the descriptor must parse, the pathspec must resolve)
+# is what keeps it from over-firing, so there is no need to also guess at placement.
+COUNTS_LINE_RE = re.compile(r"^\s*Counts:(.*)$", re.IGNORECASE | re.MULTILINE)
+
+# The literal empty-declaration value, spelled exactly as `Key symbols:` spells its own -- one
+# convention, one way to say "nothing", so a reader never has to learn a second spelling.
+COUNTS_NONE_VALUE = "none."
+
+# `<descriptor>` <op> <integer>, and nothing else on the line. Strict on purpose: a trailing human
+# label or a second claim on the same line would have to be tolerated by ignoring text, and
+# ignoring text is how a half-edited declaration passes. Anything unparseable is counts_malformed.
+# '>=' is listed before '=' in the alternation so the two-character operator is never read as a
+# bare '=' with a stray '>' in front of it.
+COUNT_CLAIM_RE = re.compile(r"^`([^`]+)`\s*(>=|=)\s*(\d+)$")
+
+# matches = total regex occurrences; files = files with at least one occurrence; paths = files
+# matching the pathspec (no regex). These three are not a guess at what items might want to count:
+# they are the three shapes the real mis-counted claims actually took -- occurrence counts ("ten
+# call sites", "146 `em` declarations", "six tracker references"), distinct-file counts ("nine REST
+# classes"), and file-set sizes ("23 archives").
+COUNT_KIND_MATCHES = "matches"
+COUNT_KIND_FILES = "files"
+COUNT_KIND_PATHS = "paths"
+COUNT_KINDS = (COUNT_KIND_MATCHES, COUNT_KIND_FILES, COUNT_KIND_PATHS)
+
+# Committed content, always. See the module docstring's "Counts resolve against COMMITTED CONTENT"
+# paragraph for why the working tree is not offered as an option rather than merely not defaulted.
+COUNTS_REV = "HEAD"
+
+# ~/Project Work itself. Verified not to be a git repository, which is why it is a separate token
+# with its own documented resolution rule rather than a fourth entry in REPO_TOKENS.
+BOARD_REPO_TOKEN = "board"
+
+# token -> (tracker slug, $HOME-relative default path). Identical in every copy of this script, so
+# a cross-repo declaration means the same thing whichever copy evaluates it. The token is MANDATORY
+# in a descriptor and there is no implicit default, deliberately: citing the wrong repo was itself
+# one of the 2026-08-25 miscounts (a `chrome.css` path that exists in two of these repos), and an
+# implicit "this repo" default would have hidden exactly that choice instead of exposing it.
+REPO_TOKENS = {
+ "juneau": ("juneau", "git/apache/juneau"),
+ "console": ("sandbox-support-console", "git/sandbox/sandbox-support-console"),
+ "release-manager": ("juneau-release-manager", "git/apache/release-manager"),
+}
+
+# This script lives at <repo-root>/scripts/, so its own repo root is a filesystem fact. Used in
+# preference to the $HOME-relative default whenever a declaration names THIS repo, which keeps the
+# common case (an item counting inside its own project) working from a relocated checkout.
+SELF_REPO_ROOT = Path(__file__).resolve().parent.parent
+
+GIT_TIMEOUT_SECONDS = 60
+
+# The two comparison forms. '=' is the default and the right shape for a baseline; '>=' declares a
+# floor, which is what an invariant means. Two operators rather than a separate `kind` because the
+# thing being counted is unchanged -- only the assertion about the number differs.
+COUNT_OP_EQ = "="
+COUNT_OP_FLOOR = ">="
+COUNT_OPS = (COUNT_OP_EQ, COUNT_OP_FLOOR)
+
+# Derivation outcomes, distinguished for the same reason the tri-state is: a number that could not
+# be derived must never be reported as a number that matched.
+COUNT_OK = "ok"
+COUNT_SCOPE_EMPTY = "scope_empty"
+COUNT_UNVERIFIABLE = "unverifiable"
+
+
+class CountClaim:
+ """One parsed `Counts:` declaration: what to count, where, the operator, and the expected total."""
+
+ def __init__(self, repo: str, kind: str, pathspec: str, pattern: "re.Pattern | None", expected: int, descriptor: str, operator: str = COUNT_OP_EQ):
+ self.repo = repo
+ self.kind = kind
+ self.pathspec = pathspec
+ self.pattern = pattern
+ self.expected = expected
+ self.descriptor = descriptor
+ self.operator = operator
+
+ @property
+ def is_floor(self) -> bool:
+ return self.operator == COUNT_OP_FLOOR
+
+ def rendered(self) -> str:
+ """The declaration as the author wrote it, for every flag detail.
+
+ Built from the parsed parts rather than the raw line so a detail message can never show an
+ operator the checker did not actually apply -- reporting '= 36' for a claim evaluated as a
+ floor would make the flag unactionable in the one case where the response depends on it.
+ """
+ return f"`{self.descriptor}` {self.operator} {self.expected}"
+
+
+def glob_to_regex(pathspec: str) -> "re.Pattern":
+ """
+ Compile a slash-delimited glob into an anchored regex over repo-relative POSIX paths.
+
+ Hand-translated rather than delegated to fnmatch because fnmatch's `*` matches `/`, which would
+ silently widen every pathspec past the directory the author wrote -- a count over a wider file
+ set than declared is a wrong count that looks right. Supported: `**/` (zero or more directories),
+ a trailing/bare `**`, `*` and `?` (within one segment), and `[...]`/`[!...]` character classes.
+ """
+ out = ["^"]
+ i, n = 0, len(pathspec)
+ while i < n:
+ if pathspec.startswith("**/", i):
+ out.append("(?:[^/]*/)*")
+ i += 3
+ elif pathspec.startswith("**", i):
+ out.append(".*")
+ i += 2
+ elif pathspec[i] == "*":
+ out.append("[^/]*")
+ i += 1
+ elif pathspec[i] == "?":
+ out.append("[^/]")
+ i += 1
+ elif pathspec[i] == "[":
+ close = pathspec.find("]", i + 1)
+ if close == -1:
+ out.append(re.escape(pathspec[i]))
+ i += 1
+ else:
+ body = pathspec[i + 1 : close]
+ out.append("[" + ("^" + body[1:] if body.startswith("!") else body) + "]")
+ i = close + 1
+ else:
+ out.append(re.escape(pathspec[i]))
+ i += 1
+ out.append("$")
+ return re.compile("".join(out))
+
+
+def parse_counts_value(value: str) -> tuple:
+ """
+ Parse one `Counts:` field value into ("none", None) / ("claim", CountClaim) / ("malformed", reason).
+
+ Three outcomes, mirroring the Key-symbols tri-state, with Absent handled by the caller (no line
+ at all). Nothing here touches the filesystem: a grammar error is a grammar error whether or not
+ the repos are reachable, which is what lets --no-counts still report counts_malformed.
+ """
+ stripped = value.strip()
+ if stripped == "":
+ return ("malformed", "'Counts:' line has no value -- write 'None.' to declare that nothing here is mechanically countable.")
+ if stripped.lower() == COUNTS_NONE_VALUE:
+ return ("none", None)
+
+ claim_match = COUNT_CLAIM_RE.match(stripped)
+ if claim_match is None:
+ return ("malformed", f"expected `<repo>:<kind>:<pathspec>[:<regex>]` = <integer> (or >= <integer> for a floor), got: {stripped}")
+
+ descriptor, operator, expected = claim_match.group(1), claim_match.group(2), int(claim_match.group(3))
+ # maxsplit=3 so a regex containing ':' stays intact; the pathspec therefore may not contain one,
+ # which also excludes git's magic-pathspec syntax (":(glob)...") -- documented, not accidental.
+ fields = descriptor.split(":", 3)
+ if len(fields) < 3:
+ return ("malformed", f"descriptor needs at least <repo>:<kind>:<pathspec>, got: {descriptor}")
+
+ repo, kind, pathspec = fields[0].strip(), fields[1].strip(), fields[2].strip()
+ pattern_source = fields[3] if len(fields) == 4 else None
+
+ if repo != BOARD_REPO_TOKEN and repo not in REPO_TOKENS:
+ known = ", ".join(sorted(list(REPO_TOKENS) + [BOARD_REPO_TOKEN]))
+ return ("malformed", f"unknown repo token '{repo}' (known: {known})")
+ if kind not in COUNT_KINDS:
+ return ("malformed", f"unknown kind '{kind}' (known: {', '.join(COUNT_KINDS)})")
+ if pathspec == "":
+ return ("malformed", f"empty pathspec in: {descriptor}")
+ if pathspec.startswith("/") or ".." in pathspec.split("/"):
+ return ("malformed", f"pathspec must be repo-relative with no '..': {pathspec}")
+
+ if kind == COUNT_KIND_PATHS:
+ if pattern_source is not None:
+ return ("malformed", f"kind 'paths' counts files and takes no regex, but one was given: {descriptor}")
+ if expected == 0:
+ # A zero-file scope is counts_scope_empty by rule 2, so `paths ... = 0` could only ever
+ # be reported as a failure. Rejecting it at parse time says so plainly instead of
+ # letting an author write a claim that is defined to flag.
+ return ("malformed", "kind 'paths' cannot declare 0 -- a glob matching nothing is indistinguishable from a glob that has rotted (see counts_scope_empty).")
+ pattern = None
+ else:
+ if not pattern_source:
+ return ("malformed", f"kind '{kind}' needs a regex after the pathspec: {descriptor}")
+ try:
+ pattern = re.compile(pattern_source, re.MULTILINE)
+ except re.error as exc:
+ return ("malformed", f"regex does not compile ({exc}): {pattern_source}")
+
+ # A floor of 0 holds for every possible derivation, so it asserts nothing while looking like a
+ # guard -- the one decorative-floor case that can be caught without inventing a slack threshold
+ # (see the module docstring's DELIBERATE NON-GOAL). Refused for the same reason `paths ... = 0` is.
+ if operator == COUNT_OP_FLOOR and expected == 0:
+ return ("malformed", "a floor of 0 asserts nothing -- every derivation satisfies `>= 0`. Declare the floor you actually mean, or use '=' if the claim is that the count is exactly 0.")
+
+ return ("claim", CountClaim(repo, kind, pathspec, pattern, expected, descriptor, operator))
+
+
+class CountsResolver:
+ """
+ Re-derives declared counts from committed content.
+
+ Holds no persistent state and writes nothing: the only thing it remembers is, within one run,
+ the path listing and blob text of a single immutable revision, so that two declarations over the
+ same commit do not re-read it. That is deduplication of identical reads, not a cached verdict --
+ a fresh process re-derives every number from git from scratch, which is the property that makes
+ a stale declaration impossible to pass by inertia.
+ """
+
+ def __init__(self, repo_overrides: "dict | None" = None, project_work: "Path | None" = None):
+ self._overrides = dict(repo_overrides or {})
+ self._project_work = project_work or PROJECT_WORK
+ self._paths_cache = {}
+ self._blob_cache = {}
+
+ def repo_root(self, token: str) -> "Path | None":
+ """Where a repo token resolves to, or None if the token is unknown."""
+ if token in self._overrides:
+ return Path(self._overrides[token]).expanduser()
+ if token == BOARD_REPO_TOKEN:
+ return self._project_work
+ entry = REPO_TOKENS.get(token)
+ if entry is None:
+ return None
+ slug, home_relative = entry
+ # A token naming THIS repo resolves to this script's own root, which is derived rather than
+ # assumed; other tokens fall back to the shared default location.
+ return SELF_REPO_ROOT if slug == TRACKER_SLUG else Path.home() / home_relative
+
+ def _git(self, root: Path, args: list) -> "str | None":
+ """Run a read-only git command in root, returning stdout or None on any failure."""
+ try:
+ done = subprocess.run(
+ ["git", "-C", str(root), *args],
+ capture_output=True,
+ text=True,
+ timeout=GIT_TIMEOUT_SECONDS,
+ check=False,
+ )
+ except (OSError, subprocess.SubprocessError):
+ return None
+ return done.stdout if done.returncode == 0 else None
+
+ def _repo_paths(self, token: str, root: Path) -> "list | None":
+ """Every file path in scope for this token: the revision's tree, or the board's filesystem."""
+ if token in self._paths_cache:
+ return self._paths_cache[token]
+
+ if token == BOARD_REPO_TOKEN:
+ # Not a git repository (verified), so the filesystem IS the record here -- see the
+ # module docstring for why that is sound for the board and wrong for a cloned repo.
+ if not root.is_dir():
+ paths = None
+ else:
+ paths = sorted(str(p.relative_to(root).as_posix()) for p in root.rglob("*") if p.is_file())
+ else:
+ listing = self._git(root, ["ls-tree", "-r", "--name-only", "-z", COUNTS_REV])
+ paths = None if listing is None else sorted(p for p in listing.split("\0") if p)
+
+ self._paths_cache[token] = paths
+ return paths
+
+ @staticmethod
+ def _decode(raw: "bytes | None") -> "str | None":
+ """Decoded text, or None for unreadable/binary content.
+
+ A NUL byte means binary; counting a regex over decoded binary is meaningless, and skipping
+ it matches what `git grep -I` / `rg` already do, so a declared count never silently picks
+ up noise from a jar or an image.
+ """
+ if raw is None or b"\0" in raw:
+ return None
+ return raw.decode("utf-8", errors="replace")
+
+ def _read_all(self, token: str, root: Path, paths: list) -> dict:
+ """
+ {path: text-or-None} for every requested path, populating the per-run blob cache.
+
+ For a git repo the blobs are fetched through a SINGLE `git cat-file --batch` process rather
+ than one `git show` per file: a pathspec like "**/*.java" over a large repo selects thousands
+ of files, and per-file subprocesses turned a sub-second check into a minute-long one.
+ """
+ wanted = [p for p in paths if (token, p) not in self._blob_cache]
+
+ if wanted and token == BOARD_REPO_TOKEN:
+ for path in wanted:
+ try:
+ raw = (root / path).read_bytes()
+ except OSError:
+ raw = None
+ self._blob_cache[(token, path)] = self._decode(raw)
+ elif wanted:
+ for path, raw in self._cat_file_batch(root, wanted).items():
+ self._blob_cache[(token, path)] = self._decode(raw)
+
+ # Anything cat-file declined to answer for is recorded as unreadable rather than retried, so
+ # one bad path cannot make the whole declaration silently count fewer files than it names.
+ for path in wanted:
+ self._blob_cache.setdefault((token, path), None)
+
+ return {p: self._blob_cache[(token, p)] for p in paths}
+
+ def _cat_file_batch(self, root: Path, paths: list) -> dict:
+ """{path: bytes} for the given paths at COUNTS_REV, via one `git cat-file --batch`."""
+ request = "".join(f"{COUNTS_REV}:{p}\n" for p in paths).encode("utf-8")
+ try:
+ done = subprocess.run(
+ ["git", "-C", str(root), "cat-file", "--batch"],
+ input=request,
+ capture_output=True,
+ timeout=GIT_TIMEOUT_SECONDS,
+ check=False,
+ )
+ except (OSError, subprocess.SubprocessError):
+ return {}
+ if done.returncode != 0:
+ return {}
+
+ out, offset, blobs = done.stdout, 0, {}
+ for path in paths:
+ newline = out.find(b"\n", offset)
+ if newline == -1:
+ break
+ header = out[offset:newline].split(b" ")
+ offset = newline + 1
+ # "<oid> <type> <size>" for a hit; "<request> missing" (or "ambiguous") otherwise, and a
+ # non-blob (a path that resolves to a tree) carries no content to skip past either.
+ if len(header) != 3 or header[1] != b"blob":
+ continue
+ size = int(header[2])
+ blobs[path] = out[offset : offset + size]
+ offset += size + 1 # git writes a trailing newline after each object's contents
+ return blobs
+
+ def derive(self, claim: CountClaim) -> tuple:
+ """
+ Re-derive this claim's count NOW. Returns (outcome, value, detail).
+
+ outcome is COUNT_OK (value is the freshly counted integer), COUNT_SCOPE_EMPTY (the pathspec
+ matched no file, so there is nothing to count and the declaration is unanchored), or
+ COUNT_UNVERIFIABLE (the repo or revision could not be read at all).
+ """
+ root = self.repo_root(claim.repo)
+ if root is None:
+ return (COUNT_UNVERIFIABLE, None, f"unknown repo token '{claim.repo}'")
+ if not root.is_dir():
+ return (COUNT_UNVERIFIABLE, None, f"'{claim.repo}' does not resolve to a directory ({root})")
+
+ paths = self._repo_paths(claim.repo, root)
+ if paths is None:
+ return (COUNT_UNVERIFIABLE, None, f"cannot read {claim.repo} at {COUNTS_REV} ({root})")
+
+ matcher = glob_to_regex(claim.pathspec)
+ in_scope = [p for p in paths if matcher.match(p)]
+ if not in_scope:
+ return (COUNT_SCOPE_EMPTY, None, f"pathspec '{claim.pathspec}' matches no file in {claim.repo}")
+
+ if claim.kind == COUNT_KIND_PATHS:
+ return (COUNT_OK, len(in_scope), "")
+
+ occurrences, hit_files = 0, 0
+ for text in self._read_all(claim.repo, root, in_scope).values():
+ if text is None:
+ continue
+ found = len(claim.pattern.findall(text))
+ if found:
+ hit_files += 1
+ occurrences += found
+ return (COUNT_OK, occurrences if claim.kind == COUNT_KIND_MATCHES else hit_files, "")
def find_plan_files(todo_dir: Path) -> list:
- """Every TODO-/READY-/MAYBE-<id>[<letter>]-*.md file directly under todo_dir, sorted by name."""
+ """Every TODO-/READY-/MAYBE-/HOLD-/NEED_INPUT-<id>[<letter>]-*.md file directly under todo_dir,
+ sorted by name. Includes claim stubs (CLAIM_STUB_RE) -- an id reservation must keep counting
+ as "taken" -- but the caller routes those to audit_claim_stub() instead of audit_file(); see
+ the module docstring's "Claim-stub handling" section."""
return sorted(p for p in todo_dir.glob("*.md") if FILENAME_RE.match(p.name))
+def group_claims_by_numeric_id(files: list) -> dict:
+ """
+ Map numeric id (int, letter suffix stripped) -> {"stub": Path | None, "plans": [Path, ...]},
+ from files (as returned by find_plan_files(), which already includes claim stubs).
+
+ Feeds exactly one check: unfinalized_claim_stub in audit_claim_stub() below. A stub and ANY
+ real (non-stub) file sharing the same numeric id -- including a lettered child of that id --
+ means the id was reserved but the reservation was never consumed by finalizing it, because a
+ plan-shaped file for that same number already exists under a different name. Grouping by the
+ bare numeric id (not the raw digit string) matters here for the same reason todo-next-id.py
+ itself keys everything off int(...): "TODO-R0037-..." and a hypothetically-unpadded
+ "TODO-R37-..." name the same id and must land in the same bucket.
+ """
+ groups: dict[int, dict] = {}
+ for path in files:
+ m = ID_FROM_FILENAME_RE.match(path.name)
+ if not m:
+ continue
+ numeric_id = int(m.group(1))
+ entry = groups.setdefault(numeric_id, {"stub": None, "plans": []})
+ if CLAIM_STUB_RE.match(path.name):
+ entry["stub"] = path
+ else:
+ entry["plans"].append(path)
+ return groups
+
+
def file_prefix(path: Path) -> str:
- """"TODO", "READY", or "MAYBE" (the filename already matched FILENAME_RE)."""
+ """"TODO", "READY", "MAYBE", "HOLD", or "NEED_INPUT" (the filename already matched FILENAME_RE)."""
return path.name.split("-", 1)[0]
@@ -153,6 +838,61 @@ def extract_section(text: str, heading_text: str) -> str | None:
return None
+def header_region(text: str) -> str:
+ """
+ Everything before the first "## " section heading (the whole file if there is none).
+
+ This is the region where a plan file states what it IS -- title, Source, Current status,
+ Complexity, Parent -- as opposed to the body sections, where it discusses other items. That
+ split is what separates "I am a member of WAVE-nnnn" from "I must be serialized against
+ WAVE-nnnn's members": in the real corpus the four items that only cross-reference an accepted
+ wave do so from a body section ("## Sequencing", "## Notes"-adjacent prose), several hundred
+ lines in, while every actual member names its wave in the header region.
+
+ Body-vs-header alone is not load-bearing, and is not meant to be: the status-prefix condition
+ in _flag_ready_but_wave_accepted() independently excludes all four of those cross-referencing
+ items, so the two conditions are belt-and-braces rather than a single fuzzy guess.
+ """
+ first_heading = SECTION_HEADING_RE.search(text)
+ return text[: first_heading.start()] if first_heading is not None else text
+
+
+def closed_out_waves(board_path: "Path | None") -> frozenset:
+ """
+ The set of WAVE-nnnn ids the board records as accepted AND that have no live WAVE file left.
+
+ Both halves are required, and each covers the other's failure mode:
+ - The board's accepted-wave log is prose, so a loose "accepted" word match could in
+ principle pick up a wave that is only being discussed.
+ - The waves/ directory is a filesystem fact, not prose: per the board's own documented
+ lifecycle ("After the operator accepts ... delete that WAVE-nnnn-*.md"), an in-flight or
+ merely-planned wave still has its file on disk. Requiring the file to be GONE means a
+ wave has to be closed out in both records before any item can be flagged against it.
+
+ Returns an empty set for a missing/unreadable board, which makes the whole
+ ready_but_wave_already_accepted check inert rather than wrong -- the same reason this script
+ treats a missing scan directory as a hard error but an empty one as a clean pass: never
+ manufacture a verdict from an absent input.
+ """
+ if board_path is None or not board_path.is_file():
+ return frozenset()
+ try:
+ board_text = board_path.read_text(encoding="utf-8")
+ except OSError:
+ return frozenset()
+
+ accepted = set()
+ for line in board_text.splitlines():
+ if BOARD_ACCEPTED_RE.search(line) and not BOARD_NOT_ACCEPTED_RE.search(line):
+ accepted.update(WAVE_ID_RE.findall(line))
+
+ waves_dir = board_path.parent / WAVES_DIRNAME
+ if waves_dir.is_dir():
+ live = {w for p in waves_dir.glob("*.md") for w in WAVE_ID_RE.findall(p.name)}
+ accepted -= live
+ return frozenset(accepted)
+
+
def open_questions_are_unresolved(section_body: str) -> bool:
"""True if the "## Open questions" section has at least one numbered item that doesn't look answered/resolved."""
items = list(NUMBERED_ITEM_RE.finditer(section_body))
@@ -169,15 +909,26 @@ def open_questions_are_unresolved(section_body: str) -> bool:
return False
-def status_prefix_ok(prefix: str, status: str) -> bool:
- """True if status's wording matches one of the recognized phrases for this file's TODO/READY/MAYBE prefix."""
- normalized = status.strip().lower()
- if prefix == "MAYBE":
- return normalized.startswith(MAYBE_STATUS_PREFIX)
- return any(normalized.startswith(p) for p in TODO_READY_STATUS_PREFIXES)
+def normalize_status_text(status: str) -> str:
+ """Status text with leading markdown emphasis/quote decoration stripped and case-folded, used by
+ the phrase-prefix lifecycle-mismatch checks below (e.g. recognizing 'ready to execute' under a
+ leading '**')."""
+ return STATUS_DECORATION_RE.sub("", status).strip().lower()
-def _flag_missing_headers(status_match, complexity_match, flags: list) -> None:
+def status_value_is_empty(status: str) -> bool:
+ """
+ True if the "Current status:" field is present but carries no real content once markdown/
+ punctuation decoration is stripped -- e.g. a header line left unfilled, or a value that is only
+ dashes/asterisks. Deliberately permissive about WHAT the remaining text says: the trackers have
+ settled on free-form descriptive prose rather than a fixed phrase set, so this only catches the
+ value being structurally absent, never any particular wording.
+ """
+ stripped = re.sub(r"[\s*_`\"'\u2010-\u2015.,:;()\[\]-]+", "", status)
+ return stripped == ""
+
+
+def _flag_missing_headers(status_match: "re.Match | None", complexity_match: "re.Match | None", flags: list) -> None:
"""Appends missing_status_header / missing_complexity_header for headers absent from the file."""
if status_match is None:
flags.append(("missing_status_header", "No 'Current status:' line found."))
@@ -185,7 +936,7 @@ def _flag_missing_headers(status_match, complexity_match, flags: list) -> None:
flags.append(("missing_complexity_header", "No 'Complexity:' line found."))
-def _flag_status_placement(text: str, status_match, flags: list) -> None:
+def _flag_status_placement(text: str, status_match: "re.Match", flags: list) -> None:
"""Appends status_header_misplaced if the status line appears at/after the first '##' heading."""
first_heading = SECTION_HEADING_RE.search(text)
if first_heading is not None and status_match.start() >= first_heading.start():
@@ -194,48 +945,200 @@ def _flag_status_placement(text: str, status_match, flags: list) -> None:
def _flag_open_questions_if_ready(prefix: str, normalized_status: str, text: str, flags: list) -> None:
"""Appends ready_but_has_open_questions if a Ready-to-execute file still has an unresolved '## Open questions'."""
- if prefix not in ("TODO", "READY") or not normalized_status.startswith(READY_TO_EXECUTE_STATUS):
+ if prefix not in ("TODO", "READY") or not normalized_status.startswith(READY_TO_EXECUTE_PREFIX):
return
oq_section = extract_section(text, "Open questions")
if oq_section is not None and open_questions_are_unresolved(oq_section):
flags.append(("ready_but_has_open_questions", "Status says 'Ready to execute' but '## Open questions' still has unresolved item(s)."))
+def _flag_ready_but_wave_accepted(prefix: str, normalized_status: str, text: str, closed_waves: frozenset, flags: list) -> None:
+ """
+ Appends ready_but_wave_already_accepted for a Ready-to-execute file still queued behind a wave
+ the board has already accepted and closed out.
+
+ Scoped to TODO-/READY- for the same reason the other Ready-gated check is: this is a
+ TODO/READY lifecycle signal, and the docstring's prefix table exempts NEED_INPUT- from those.
+ """
+ if not closed_waves or prefix not in ("TODO", "READY"):
+ return
+ if not normalized_status.startswith(READY_TO_EXECUTE_PREFIX):
+ return
+ declared = sorted(set(WAVE_ID_RE.findall(header_region(text))) & closed_waves)
+ if declared:
+ waves = ", ".join(declared)
+ flags.append((
+ "ready_but_wave_already_accepted",
+ f"Status still says 'Ready to execute' but the board records {waves} as accepted (WAVE file gone) -- "
+ f"archive the item if it shipped in that wave, or drop the stale wave reference.",
+ ))
+
+
def _flag_status_prefix_transitions(prefix: str, status: str, normalized_status: str, flags: list) -> None:
- """Appends the READY/TODO/MAYBE prefix-vs-status-wording mismatch flags (missed renames, wrong prefix)."""
+ """Appends the READY/TODO/MAYBE/HOLD prefix-vs-status-wording mismatch flags (missed renames, wrong prefix)."""
if prefix == "READY" and normalized_status.startswith("waiting for user input"):
flags.append(("ready_prefix_waiting_status", "READY-prefixed file but status still says 'Waiting for user input'."))
- if prefix == "TODO" and normalized_status.startswith(READY_TO_EXECUTE_STATUS):
+ if prefix == "TODO" and normalized_status.startswith(READY_TO_EXECUTE_PREFIX):
flags.append(("todo_prefix_marked_ready", "TODO-prefixed file already marked 'Ready to execute' -- possible missed rename to READY-*.md."))
- if prefix in ("TODO", "READY") and normalized_status.startswith("parked"):
+ if prefix in ("TODO", "READY") and normalized_status.startswith(MAYBE_STATUS_PREFIX):
flags.append(("parked_status_wrong_prefix", "Status says 'Parked...' but filename is not MAYBE-prefixed."))
- if prefix == "MAYBE" and not normalized_status.startswith("parked"):
+ if prefix == "MAYBE" and not normalized_status.startswith(MAYBE_STATUS_PREFIX):
flags.append(("maybe_prefix_non_parked", f"MAYBE-prefixed file but status doesn't start with 'Parked': '{status}'"))
+ if prefix in ("TODO", "READY") and normalized_status.startswith(HOLD_STATUS_PREFIX):
+ flags.append(("on_hold_status_wrong_prefix", "Status says 'On hold...' but filename is not HOLD-prefixed."))
+
+ if prefix == "HOLD" and not normalized_status.startswith(HOLD_STATUS_PREFIX):
+ flags.append(("hold_prefix_non_on_hold", f"HOLD-prefixed file but status doesn't start with 'On hold': '{status}'"))
+
+
+def _flag_declared_counts(text: str, counts_resolver: "CountsResolver | None", flags: list) -> None:
+ """
+ Appends counts_malformed / counts_mismatch / counts_below_floor / counts_scope_empty /
+ counts_unverifiable.
+
+ An ABSENT `Counts:` line produces nothing, which is the whole reason this check can be turned on
+ over an existing corpus: a file that makes no declaration is making no claim. Grammar errors are
+ reported with or without a resolver, because a line that does not parse is wrong regardless of
+ whether the source it names is reachable.
+ """
+ for line_match in COUNTS_LINE_RE.finditer(text):
+ outcome, parsed = parse_counts_value(line_match.group(1))
+ if outcome == "none":
+ continue
+ if outcome == "malformed":
+ flags.append(("counts_malformed", f"'Counts:' line does not parse -- {parsed}"))
+ continue
+ if counts_resolver is None:
+ continue
+
+ derived_outcome, derived, detail = counts_resolver.derive(parsed)
+ if derived_outcome == COUNT_UNVERIFIABLE:
+ flags.append((
+ "counts_unverifiable",
+ f"declared count {parsed.rendered()} could not be re-derived: {detail}. "
+ f"An unverified count is not a verified one -- fix the repo token/path, or pass --repo-root.",
+ ))
+ elif derived_outcome == COUNT_SCOPE_EMPTY:
+ # Checked BEFORE either comparison, and that ordering is load-bearing for floors: a
+ # pathspec that has rotted to nothing would otherwise derive 0 and be reported as a floor
+ # breach, sending the operator to hunt a regression in code that is fine. An unanchored
+ # declaration is its own failure whichever operator it was written with.
+ flags.append((
+ "counts_scope_empty",
+ f"declared count {parsed.rendered()} is no longer anchored: {detail} "
+ f"at {COUNTS_REV}. The number cannot be tested, so it is flagged rather than passed.",
+ ))
+ elif parsed.is_floor:
+ if derived < parsed.expected:
+ flags.append((
+ "counts_below_floor",
+ f"declared floor {parsed.rendered()} BREACHED -- re-derived {derived} "
+ f"from {parsed.repo} at {COUNTS_REV}. A floor is an invariant: investigate the drop. "
+ f"Do NOT lower the floor to match -- that deletes the guard.",
+ ))
+ elif derived != parsed.expected:
+ flags.append((
+ "counts_mismatch",
+ f"declared count {parsed.rendered()} but re-derived {derived} "
+ f"from {parsed.repo} at {COUNTS_REV}.",
+ ))
+
+
+def audit_claim_stub(path: Path, coexisting_plans: "list | tuple" = ()) -> list:
+ """
+ Return [(reason_code, detail)] for one "TODO-<id>-CLAIMED.md" claim stub (CLAIM_STUB_RE).
+
+ A claim stub is exempt from every check in audit_file() -- it is a reservation, not a plan,
+ and is deliberately un-plan-shaped by claim_placeholder_text() (todo-next-id.py) so a human
+ recognizes an abandoned reservation instead of a plausible-looking empty TODO. Two things are
+ checked here instead:
+ - unfinalized_claim_stub (coexisting_plans, from group_claims_by_numeric_id()): whether a
+ real plan file already exists for this SAME numeric id. Unconditional -- checked
+ regardless of the stub's age, because it is wrong the instant it exists. coexisting_plans
+ defaults to empty so every existing single-argument caller keeps its previous behavior.
+ - stale_claim: whether the reservation looks abandoned by age. See STALE_CLAIM_AGE_HOURS and
+ the module docstring's "Claim-stub handling" section for why age -- not the pid/hostname
+ the stub also records -- is the signal.
+ """
+ flags = []
+
+ if coexisting_plans:
+ names = ", ".join(p.name for p in sorted(coexisting_plans))
+ plural = len(coexisting_plans) > 1
+ flags.append((
+ "unfinalized_claim_stub",
+ f"this id-reservation stub coexists with {'real plan files' if plural else 'a real plan file'} "
+ f"under the SAME numeric id: {names}. An id must never simultaneously name a reservation and a "
+ f"finished plan -- this is wrong regardless of age (distinct from 'stale_claim' below, if that is "
+ f"also flagged). Most likely the claiming session wrote the real plan under a brand-new filename "
+ f"instead of finalizing this stub in place (see todo-next-id.py's --finalize). If {names} is the "
+ f"complete, correct plan for this id, delete this stub -- do not run --finalize, it will refuse "
+ f"since the target already exists. If you are not certain the plan is complete, investigate "
+ f"before deleting anything.",
+ ))
+
+ age_hours = (time.time() - path.stat().st_mtime) / 3600
+ if age_hours < STALE_CLAIM_AGE_HOURS:
+ return flags
-def audit_file(path: Path) -> list:
- """Return a list of (reason_code, detail) tuples for this plan file. Empty list means no flags."""
+ text = path.read_text(encoding="utf-8")
+ claimed_at = CLAIM_CLAIMED_AT_RE.search(text)
+ claimed_by = CLAIM_CLAIMED_BY_RE.search(text)
+ when = claimed_at.group(1) if claimed_at else "an unrecorded time (no 'Claimed at:' line)"
+ who = (
+ f"pid {claimed_by.group(1)} on {claimed_by.group(2)}"
+ if claimed_by
+ else "an unrecorded claimant (no 'Claimed by:' line)"
+ )
+
+ flags.append((
+ "stale_claim",
+ f"id-reservation stub is {age_hours:.1f}h old (threshold {STALE_CLAIM_AGE_HOURS}h) -- "
+ f"claimed at {when} by {who} -- and was never turned into a real plan or removed. The "
+ f"claiming session likely crashed; confirm nothing references this id, then delete the "
+ f"stub per its own instructions ('todo-next-id.py --list-claims' finds every stub of "
+ f"this kind).",
+ ))
+ return flags
+
+
+def audit_file(path: Path, closed_waves: frozenset = frozenset(), counts_resolver: "CountsResolver | None" = None) -> list:
+ """
+ Return a list of (reason_code, detail) tuples for this plan file. Empty list means no flags.
+
+ closed_waves defaults to empty, which makes ready_but_wave_already_accepted inert: every
+ caller that has no wave board (and every existing single-argument caller) gets exactly the
+ previous behavior instead of a check silently running against an unknown board.
+
+ counts_resolver defaults to None on the same principle: without one, declared counts are checked
+ for GRAMMAR only and never reported as agreeing, so a caller that cannot reach the source trees
+ gets silence about the numbers rather than a manufactured pass.
+ """
text = path.read_text(encoding="utf-8")
prefix = file_prefix(path)
flags = []
+ _flag_declared_counts(text, counts_resolver, flags)
+
status_match = STATUS_LINE_RE.search(text)
complexity_match = COMPLEXITY_LINE_RE.search(text)
_flag_missing_headers(status_match, complexity_match, flags)
if status_match is not None:
status = status_match.group(1).strip()
- normalized = status.lower()
+ normalized = normalize_status_text(status)
_flag_status_placement(text, status_match, flags)
- if not status_prefix_ok(prefix, status):
- flags.append(("unrecognized_status_phrase", f"Status text doesn't match the {prefix} wording rules: '{status}'"))
+ if status_value_is_empty(status):
+ flags.append(("empty_status_value", "'Current status:' field is present but has no content."))
_flag_open_questions_if_ready(prefix, normalized, text, flags)
+ _flag_ready_but_wave_accepted(prefix, normalized, text, closed_waves, flags)
_flag_status_prefix_transitions(prefix, status, normalized, flags)
return flags
@@ -243,20 +1146,37 @@ def audit_file(path: Path) -> list:
def main() -> int:
parser = argparse.ArgumentParser(
- description=f"Best-effort pre-filter for {REPO_LABEL}'s .work/todo/ header inconsistencies (see @{SKILL_NAME}).",
+ description=f"Best-effort pre-filter for {REPO_LABEL} tracker header inconsistencies (see @{SKILL_NAME}).",
epilog=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter,
)
- parser.add_argument("--root", metavar="PATH", help="Repository root; scans <root>/.work/todo/ (default: parent of this script's directory).")
+ parser.add_argument("--root", metavar="PATH", help="Project Work root (default: ~/Project Work).")
parser.add_argument("--dir", metavar="PATH", help="Exact directory to scan, overriding --root.")
+ parser.add_argument("--board", metavar="PATH", help=f"Wave board file (default: {BOARD_FILENAME} beside the tracker's parent).")
+ parser.add_argument("--repo-root", metavar="TOKEN=PATH", action="append", default=[], help="Override where a Counts: repo token resolves to. Repeatable.")
+ parser.add_argument("--no-counts", action="store_true", help="Do not re-derive declared counts (grammar errors are still reported).")
parser.add_argument("--verbose", "-v", action="store_true", help="Also print files that passed every check.")
args = parser.parse_args()
+ repo_overrides = {}
+ for override in args.repo_root:
+ token, _, override_path = override.partition("=")
+ if not token or not override_path:
+ print(f"ERROR: --repo-root expects TOKEN=PATH, got {override!r}.", file=sys.stderr)
+ return 2
+ repo_overrides[token] = override_path
+
if args.dir:
todo_dir = Path(args.dir).resolve()
else:
- repo_root = Path(args.root).resolve() if args.root else DEFAULT_REPO_ROOT
- todo_dir = repo_root / ".work" / "todo"
+ project_work = Path(args.root).expanduser().resolve() if args.root else PROJECT_WORK
+ todo_dir = project_work / "todos" / TRACKER_SLUG
+
+ # Inferred as a sibling of todos/, i.e. <project-work>/WAVE.md, which holds for --root and for
+ # the default. Under --dir the inference can miss, and missing is the correct outcome there:
+ # closed_out_waves() then returns empty and the wave check stays inert instead of comparing
+ # against some unrelated board that happened to sit two levels up.
+ board_path = Path(args.board).expanduser().resolve() if args.board else todo_dir.parent.parent / BOARD_FILENAME
# Always announce the tree actually scanned, for the same reason todo-next-id.py does:
# bare per-repository ids make a wrong-tree run indistinguishable from a right-tree one.
@@ -272,12 +1192,36 @@ def main() -> int:
files = find_plan_files(todo_dir)
if not files:
- print(f"No TODO-/READY-/MAYBE-*.md files found under {todo_dir} (directory exists and is empty of plan files).")
+ print(f"No TODO-/READY-/MAYBE-/HOLD-/NEED_INPUT-*.md files found under {todo_dir} (directory exists and is empty of plan files).")
return 0
+ # Announced for the same reason the scan directory is: a silently-skipped wave check looks
+ # identical to a clean one, which is the trap this script keeps trying to close.
+ closed_waves = closed_out_waves(board_path)
+ if closed_waves:
+ print(f"[{REPO_LABEL}] wave board {board_path}: accepted+closed {', '.join(sorted(closed_waves))}", file=sys.stderr)
+ else:
+ print(f"[{REPO_LABEL}] no accepted+closed waves from {board_path} -- ready_but_wave_already_accepted is inert this run.", file=sys.stderr)
+
+ # Announced for the same reason the wave check is: "no count disagreed" and "no count was
+ # checked" must not look the same on the terminal.
+ if args.no_counts:
+ counts_resolver = None
+ print(f"[{REPO_LABEL}] --no-counts: declared counts are grammar-checked only, not re-derived.", file=sys.stderr)
+ else:
+ counts_resolver = CountsResolver(repo_overrides, project_work=todo_dir.parent.parent)
+ print(f"[{REPO_LABEL}] declared counts re-derived from {COUNTS_REV} this run (nothing cached between runs).", file=sys.stderr)
+
+ claim_groups = group_claims_by_numeric_id(files)
+
flagged_count = 0
for path in files:
- flags = audit_file(path)
+ if CLAIM_STUB_RE.match(path.name):
+ numeric_id = int(ID_FROM_FILENAME_RE.match(path.name).group(1))
+ coexisting_plans = claim_groups.get(numeric_id, {}).get("plans", [])
+ flags = audit_claim_stub(path, coexisting_plans)
+ else:
+ flags = audit_file(path, closed_waves, counts_resolver)
if flags:
flagged_count += 1
print(f"{path.name}")