You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@magpie.apache.org by po...@apache.org on 2026/10/23 00:26:12 UTC
(magpie) branch main updated: fix(dev): test only changed workspace members, in parallel, in the local pytest hook (#1764)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/magpie.git
The following commit(s) were added to refs/heads/main by this push:
new d23116e2f fix(dev): test only changed workspace members, in parallel, in the local pytest hook (#1764)
d23116e2f is described below
commit d23116e2f02dcf7467e287855a2b1c848d28a96f
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Fri Oct 23 02:26:05 2026 +0200
fix(dev): test only changed workspace members, in parallel, in the local pytest hook (#1764)
The workspace-pytest hook ran every member's suite whenever any file it
matched changed, the root pyproject.toml included, so each dev-version bump
ran everything. It now passes the changed files to run-workspace-check.sh,
which runs only the members ci-pytest-batches.py selects, the same selection
the tests workflow makes on a pull request.
A member is picked when any of its files changed other than Markdown and
plugin manifests (tests also run shell scripts and read data files), also
through a symlinked source directory. Every member depending on a picked one
is added too, transitively, through [project] dependencies, optional
dependencies and dependency groups; the shared magpie-dev toolchain is not
followed.
pytest-xdist joins the shared toolchain, and both the hook and the tests
workflow run each member's tests with -n auto. The adversarial-review
parallelism test no longer asserts a wall-clock bound, which failed on a
loaded machine; it checks that the two stub runs overlapped.
AGENTS.md, CONTRIBUTING.md and the AI contribution policy now say to run
the hooks with --all-files --skip workspace-pytest, and the tests with
--from-ref origin/main --to-ref HEAD.
Generated-by: Claude Opus 5
---
.github/workflows/tests.yml | 5 +-
.pre-commit-config.yaml | 15 +++-
AGENTS.md | 24 ++++--
CONTRIBUTING.md | 33 +++++---
docs/ai-contribution-policy.md | 2 +-
pyproject.toml | 2 +-
tools/adversarial-review/tests/test_runner.py | 21 +++--
tools/dev/ci-pytest-batches.py | 109 +++++++++++++++++++++++---
tools/dev/pyproject.toml | 1 +
tools/dev/run-workspace-check.sh | 19 ++++-
tools/dev/tests/test_ci_pytest_batches.py | 59 ++++++++++++++
uv.lock | 24 ++++++
12 files changed, 278 insertions(+), 36 deletions(-)
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
index ed889923f..bc50a534f 100644
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -168,6 +168,9 @@ jobs:
# `--group` flag here because the sync above already
# installed the dev tools into the shared workspace venv.
#
+ # `-n auto` runs each member's tests in parallel (`pytest-xdist`,
+ # from the shared `magpie-dev` toolchain).
+ #
# Every member of the batch runs even after one fails; the job
# fails at the end, and the summary table names which member did.
env:
@@ -181,7 +184,7 @@ jobs:
for path in ${MEMBER_PATHS}; do
name="${path##*/}"
echo "::group::pytest (${name})"
- if uv run --directory "${path}" pytest --color=yes; then
+ if uv run --directory "${path}" pytest --color=yes -n auto; then
result="passed"
else
result="**failed**"
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 7e3c72a26..b72670774 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -658,9 +658,18 @@ repos:
- id: workspace-pytest
name: pytest (workspace)
language: system
- entry: tools/dev/run-workspace-check.sh pytest "pytest --color=yes"
- files: ^(tools/[^/]+(/[^/]+)?/(src|tests|pyproject\.toml)|pyproject\.toml|tools/dev/run-workspace-check\.sh)
- pass_filenames: false
+ entry: tools/dev/run-workspace-check.sh pytest "pytest --color=yes -n auto"
+ # Each member's tests run in parallel (`pytest-xdist`, from the
+ # shared `magpie-dev` toolchain). Only the members the changed
+ # files touch run, with the members depending on them
+ # (`tools/dev/ci-pytest-batches.py`); the root `pyproject.toml`,
+ # `uv.lock` and the plugin manifests are left out so a
+ # dev-version bump runs nothing. CI's `tests` workflow still runs
+ # them all when either root file changes. Serial, so one
+ # invocation sees every file.
+ files: ^(tools|plugins|ai-tutors)/
+ exclude: (\.md$|/\.claude-plugin/)
+ require_serial: true
# Skills' sibling scripts (plugins/*/skills/*/scripts) are not
# workspace members; their stdlib unittest suites run here.
- id: skill-script-tests
diff --git a/AGENTS.md b/AGENTS.md
index 715893a3e..665e3b521 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -461,11 +461,25 @@ line:
test -x .git/hooks/pre-commit || prek install
```
-**Before opening or updating a PR, run `prek run --all-files`** as a
-hard pre-flight gate. The commit hook only sees the files in that
-commit, so issues in files committed earlier on the branch can slip
-past it; and the PR's CI run is scoped to the PR's diff, so a file the
-branch did not touch but broke anyway (a moved anchor, a renamed
+**Before opening or updating a PR, run every hook but the tests on every
+file, then the tests for what the branch changed**, as a hard pre-flight
+gate:
+
+```bash
+prek run --all-files --skip workspace-pytest
+prek run workspace-pytest --from-ref origin/main --to-ref HEAD
+```
+
+The second run tests only the workspace members with a file the branch
+changed (Markdown and plugin manifests aside), and every member that depends
+on one of them (`tools/dev/ci-pytest-batches.py` picks them, as the PR's
+`tests` workflow does). Do not run the whole suite locally (`prek run --all-files`
+with the tests included) unless asked: a change to a root build input such
+as `uv.lock` already gets a full run in CI.
+
+The commit hook only sees the files in that commit, so issues in files
+committed earlier on the branch can slip past it; and the PR's CI run is
+scoped to the PR's diff, so a file the branch did not touch but broke anyway (a moved anchor, a renamed
heading, a hook whose config changed) goes green on the PR and fails on
the `main` build after merge. A whole-tree run locally mirrors that
`main` build and surfaces both classes before you push. If a hook modifies
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 08c25153f..0fcde42fc 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -538,7 +538,7 @@ git clone git@github.com:apache/magpie.git
cd magpie
uv tool install prek
prek install # wire the hooks into .git/hooks
-prek run --all-files # runs every hook on every file
+prek run --all-files --skip workspace-pytest # every hook but the tests, on every file
```
Note that if you are already using `prek` for some other project, you
@@ -744,8 +744,10 @@ the proposal as a draft, not a final. The loop is:
restates the function name"*. Specific corrections converge fast;
vague ones produce mush.
4. **Agent revises** — and the loop repeats.
-5. **Tests run** — `prek run --all-files`, `uv run pytest` in the
- affected Python package, the eval suite for the affected skill.
+5. **Tests run** — `prek run --all-files --skip workspace-pytest`,
+ then `prek run workspace-pytest --from-ref origin/main --to-ref HEAD`
+ for the affected Python packages and the packages depending on them,
+ and the eval suite for the affected skill.
When something fails, the failure is part of the conversation —
show the agent the error, let it diagnose.
@@ -840,8 +842,15 @@ substitutes for the other.
## Running the dev loop
-Every change should pass `prek run --all-files` locally before you
-open a PR — CI runs the same config. The hook set:
+Every change should pass the hooks locally before you open a PR — CI
+runs the same config. Run the tests only for what the branch changed:
+
+```bash
+prek run --all-files --skip workspace-pytest
+prek run workspace-pytest --from-ref origin/main --to-ref HEAD
+```
+
+The hook set:
- **`doctoc`** — regenerates TOCs on every `.md` file (except
skill `SKILL.md` files, which keep YAML frontmatter at the top,
@@ -862,7 +871,12 @@ open a PR — CI runs the same config. The hook set:
- **Per-package Python checks** — `ruff check`, `ruff format --check`,
`mypy`, `pytest` against each `tools/*/` Python package, scoped
to changes inside that package directory via the hook's `files:`
- pattern.
+ pattern. `pytest` runs only for the packages with a changed file
+ other than Markdown or a plugin manifest (also through a symlinked
+ source) and the packages depending on them; a change to the root
+ `pyproject.toml` or `uv.lock` alone, such as a dev-version bump,
+ runs no tests locally. Each package's tests run in parallel
+ (`pytest -n auto`, via `pytest-xdist`).
Separate GitHub workflows:
@@ -870,8 +884,8 @@ Separate GitHub workflows:
request the hooks are scoped to the PR's own diff (`--from-ref` /
`--to-ref`) on `ubuntu-slim`; on a push to `main` they run
`--all-files` on `ubuntu-latest`. So a green PR check is not a
- whole-repo result — run `prek run --all-files` locally before you
- push.
+ whole-repo result — run `prek run --all-files --skip workspace-pytest`
+ locally before you push.
- **`zizmor.yml`** — lints GitHub Actions workflows for known-bad
patterns; runs on every PR. zizmor is declared in the root
`pyproject.toml` dev group, so `uv run zizmor --config .zizmor.yml .`
@@ -1041,7 +1055,8 @@ git diff main..<branch> | grep '^[-+]' | grep -v '^[-+][-+][-+]' \
```bash
git fetch origin main
git rebase origin/main
- prek run --all-files
+ prek run --all-files --skip workspace-pytest
+ prek run workspace-pytest --from-ref origin/main --to-ref HEAD
git push
```
diff --git a/docs/ai-contribution-policy.md b/docs/ai-contribution-policy.md
index 3e4ae46c5..a9f6cac83 100644
--- a/docs/ai-contribution-policy.md
+++ b/docs/ai-contribution-policy.md
@@ -72,7 +72,7 @@ A contribution that includes AI-generated content is acceptable only when:
They have read every line of the diff, can explain it,
and answer review questions themselves rather than relaying them to the agent.
2. **Verification before review.**
- `prek run --all-files`, the affected package's tests,
+ `prek run --all-files --skip workspace-pytest`, the tests of the affected packages and their dependents,
and — for a skill change — the affected eval suite are run before review is requested.
3. **Review comments carry the reviewer's own judgement.**
A comment that only restates the PR description, the diff,
diff --git a/pyproject.toml b/pyproject.toml
index 6b2d364c2..b117418d7 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -42,7 +42,7 @@ requires-python = ">=3.11"
# - `prek` is the static-check / pre-commit runner. CI installs it
# via `uv sync --group dev` and invokes `uv run prek …`.
# - `magpie-dev` (tools/dev) carries the shared static-analysis +
-# test toolchain — ruff, mypy, pytest — declared in ONE place and
+# test toolchain — ruff, mypy, pytest, pytest-xdist — declared in ONE place and
# depended on by every workspace member's own `dev` group, so each
# sub-project's environment is still self-contained. The workspace
# checks run each tool against the member's own project via
diff --git a/tools/adversarial-review/tests/test_runner.py b/tools/adversarial-review/tests/test_runner.py
index 380f1c790..f859112ab 100644
--- a/tools/adversarial-review/tests/test_runner.py
+++ b/tools/adversarial-review/tests/test_runner.py
@@ -104,18 +104,27 @@ def test_timeout_kills_the_whole_process_group(stub_bin, tmp_path):
def test_reviewers_run_in_parallel_and_keep_order(stub_bin, tmp_path):
bin_dir, make = stub_bin
- make("codex", "import time; time.sleep(3)\n" + CODEX_OK)
+
+ # Each stub records when it was running. A total wall-clock bound failed
+ # on a loaded machine (pytest-xdist workers, pre-commit), so the test
+ # checks the two runs overlapped instead.
+ def timed(name: str, body: str) -> str:
+ log = tmp_path / f"{name}.times"
+ return (
+ f"import time; s = time.time(); time.sleep(3)\n"
+ f"open({str(log)!r}, 'w').write(f'{{s}} {{time.time()}}')\n" + body
+ )
+
+ make("codex", timed("codex", CODEX_OK))
make(
"gemini",
- "import json, time; time.sleep(3)\nprint(json.dumps({'response': json.dumps({'findings': []})}))\n",
+ timed("gemini", "import json\nprint(json.dumps({'response': json.dumps({'findings': []})}))\n"),
)
contexts = {n: ctx(tmp_path, n) for n in ("gemini", "codex")}
- start = time.monotonic()
results = run_all(["gemini", "codex"], contexts, 30, {"PATH": str(bin_dir)})
- # One after the other they take 6 s; the 2.5 s left over is for starting
- # both stubs on a loaded machine (a 1.3 s margin failed under pre-commit).
- assert time.monotonic() - start < 5.5
assert [(r.reviewer, r.status) for r in results] == [("gemini", "ok"), ("codex", "ok")]
+ spans = [tuple(map(float, (tmp_path / f"{n}.times").read_text().split())) for n in ("gemini", "codex")]
+ assert max(start for start, _ in spans) < min(end for _, end in spans)
def test_one_timeout_does_not_block_the_others(stub_bin, tmp_path):
diff --git a/tools/dev/ci-pytest-batches.py b/tools/dev/ci-pytest-batches.py
index d39d068b3..565f1cd4e 100644
--- a/tools/dev/ci-pytest-batches.py
+++ b/tools/dev/ci-pytest-batches.py
@@ -19,9 +19,10 @@
A member gets pytest when it has a `[tool.pytest.ini_options]` section and does
not opt out with `[tool.magpie.checks] skip = [..., "pytest"]` — the same rule
-as `tools/dev/run-workspace-check.sh`. On a pull request only the members whose
-Python or `tests/` files changed are picked, unless a shared build input changed;
-on any other event every member is.
+as `tools/dev/run-workspace-check.sh`. On a pull request only the members with a
+changed file other than Markdown (directly or through a symlinked source) are
+picked, together with every member that depends on one of them, unless
+a shared build input changed; on any other event every member is.
Most of a pytest job is setup (checkout, uv, a workspace sync), so the picked
members are grouped:
@@ -35,12 +36,16 @@ Prints a JSON array of `{"key", "name", "paths", "packages"}` for the workflow's
matrix (`paths` and `packages` space-separated, in the same order; `packages` are
the members' project names, so a job syncs only what its members declare).
`--changed-file` names a file listing the changed paths; omit it for a full run.
+`--paths` prints the picked member paths one per line instead, for the local
+`workspace-pytest` hook, so a commit runs the same members a pull request would.
"""
from __future__ import annotations
import argparse
import json
+import os
+import re
import sys
import tomllib
from pathlib import Path
@@ -48,6 +53,9 @@ from pathlib import Path
SOLO_MAX = 6
MAX_BATCHES = 8
+#: The shared toolchain every member's `dev` group lists; not a dependency to follow.
+TOOLCHAIN = "magpie-dev"
+
# A change to any of these can affect every member's tests.
GLOBAL = frozenset(
{
@@ -68,17 +76,95 @@ def has_pytest(root: Path, path: str) -> bool:
return "pytest" not in tool.get("magpie", {}).get("checks", {}).get("skip", [])
-def member_changed(path: str, changed: list[str]) -> bool:
- """A `.py` file anywhere under the member, or any file under its `tests/`."""
- prefix, tests = path + "/", path + "/tests/"
- return any(c.startswith(prefix) and (c.endswith(".py") or c.startswith(tests)) for c in changed)
+def find_linked_sources(root: Path, path: str) -> list[str]:
+ """Repo-relative targets of the directory symlinks under a member.
+
+ A member such as `tools/setup-preflight` tests code that lives elsewhere in
+ the tree through a `src/` symlink; a change at the target is a change to
+ the member.
+ """
+ resolved_root = root.resolve()
+ linked = []
+ for current, dirs, _ in os.walk(root / path):
+ dirs[:] = [d for d in dirs if not d.startswith(".") and d != "node_modules"]
+ for name in dirs:
+ entry = Path(current) / name
+ if not entry.is_symlink():
+ continue
+ try:
+ linked.append(entry.resolve().relative_to(resolved_root).as_posix())
+ except ValueError:
+ continue
+ return sorted(linked)
+
+
+def member_changed(path: str, changed: list[str], linked: list[str] | None = None) -> bool:
+ """Any file under the member or its linked sources, but Markdown and plugin manifests.
+
+ Not only Python: tests also run shell scripts and read data files beside
+ the code (`tools/agent-isolation` tests `agent-iso.sh`). A dev-version bump
+ rewrites every `.claude-plugin/plugin.json`, which no member's tests read;
+ the `check-family-plugins` hook checks them.
+ """
+ prefixes = tuple(p + "/" for p in [path, *(linked or [])])
+ return any(
+ c.startswith(prefixes) and not c.endswith(".md") and "/.claude-plugin/" not in c for c in changed
+ )
+
+
+def _normalize(name: str) -> str:
+ return re.sub(r"[-_.]+", "-", name).lower()
+
+
+def find_dependents(root: Path, paths: list[str]) -> dict[str, set[str]]:
+ """Map each member to the members that depend on it, at runtime or in a dependency group.
+
+ Groups count because a test-only dependency is still exercised by the
+ dependent's tests (`release-management` tests run `release-config`).
+ `TOOLCHAIN` does not: every member lists it in its `dev` group for the shared
+ toolchain, and following it would turn any change under `tools/dev` into a
+ full run.
+ """
+ pyprojects = {}
+ for path in paths:
+ with (root / path / "pyproject.toml").open("rb") as f:
+ pyprojects[path] = tomllib.load(f)
+ by_name = {
+ _normalize(data["project"]["name"]): path
+ for path, data in pyprojects.items()
+ if "name" in data.get("project", {})
+ }
+ by_name.pop(TOOLCHAIN, None)
+ dependents: dict[str, set[str]] = {path: set() for path in paths}
+ for path, data in pyprojects.items():
+ project = data.get("project", {})
+ requirements = list(project.get("dependencies", []))
+ for extra in project.get("optional-dependencies", {}).values():
+ requirements += extra
+ for group in data.get("dependency-groups", {}).values():
+ requirements += [r for r in group if isinstance(r, str)]
+ for requirement in requirements:
+ target = by_name.get(_normalize(re.split(r"[\s\[<>=!~;@(]", requirement, maxsplit=1)[0]))
+ if target is not None and target != path:
+ dependents[target].add(path)
+ return dependents
def select(root: Path, changed: list[str] | None) -> list[str]:
+ """The members to test: those a change touches, plus every member depending on them."""
with (root / "pyproject.toml").open("rb") as f:
paths = tomllib.load(f)["tool"]["uv"]["workspace"]["members"]
- run_all = changed is None or any(c in GLOBAL for c in changed)
- picked = [p for p in paths if has_pytest(root, p) and (run_all or member_changed(p, changed or []))]
+ if changed is None or any(c in GLOBAL for c in changed):
+ affected = set(paths)
+ else:
+ affected = {p for p in paths if member_changed(p, changed, find_linked_sources(root, p))}
+ dependents = find_dependents(root, paths)
+ pending = list(affected)
+ while pending:
+ for dependent in dependents[pending.pop()] - affected:
+ affected.add(dependent)
+ pending.append(dependent)
+ picked = [p for p in paths if p in affected and has_pytest(root, p)]
return sorted(picked, key=lambda p: p.rsplit("/", 1)[-1])
@@ -124,6 +210,7 @@ def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("--root", type=Path, default=Path("."))
parser.add_argument("--changed-file", type=Path, help="file listing a pull request's changed paths")
+ parser.add_argument("--paths", action="store_true", help="print the picked member paths, one per line")
args = parser.parse_args(argv)
changed = None
if args.changed_file:
@@ -133,6 +220,10 @@ def main(argv: list[str] | None = None) -> int:
if line.strip()
]
paths = select(args.root, changed)
+ if args.paths:
+ for path in paths:
+ print(path)
+ return 0
out = batches(paths, {p: weight(args.root, p) for p in paths})
for batch in out:
batch["packages"] = " ".join(package(args.root, p) for p in batch["paths"].split())
diff --git a/tools/dev/pyproject.toml b/tools/dev/pyproject.toml
index 44e1b2e61..0574f6bc8 100644
--- a/tools/dev/pyproject.toml
+++ b/tools/dev/pyproject.toml
@@ -44,6 +44,7 @@ license = { text = "Apache-2.0" }
dependencies = [
"mypy>=2.3.0",
"pytest>=9.1.1",
+ "pytest-xdist>=3.8.0",
"ruff>=0.16.2",
]
diff --git a/tools/dev/run-workspace-check.sh b/tools/dev/run-workspace-check.sh
index 58c0a05aa..2457370e8 100755
--- a/tools/dev/run-workspace-check.sh
+++ b/tools/dev/run-workspace-check.sh
@@ -38,6 +38,12 @@
# may be a multi-token string (e.g. "ruff check") quoted as a
# single argv item.
#
+# For `pytest`, any further arguments are the changed files (the
+# `workspace-pytest` hook passes them): only the members those files
+# touch run, picked by `tools/dev/ci-pytest-batches.py`, the same
+# selection a pull request's `tests` workflow makes. With no files
+# every applicable member runs.
+#
# Per-member opt-out: a workspace member can skip one or more
# checks by adding a `[tool.magpie.checks]` block:
#
@@ -48,7 +54,7 @@
# tools/dev/run-workspace-check.sh ruff "ruff check"
# tools/dev/run-workspace-check.sh ruff-format "ruff format --check"
# tools/dev/run-workspace-check.sh mypy mypy
-# tools/dev/run-workspace-check.sh pytest "pytest --color=yes"
+# tools/dev/run-workspace-check.sh pytest "pytest --color=yes -n auto"
#
# Exit code: 0 iff the check succeeded on every applicable
# member; 1 otherwise. The script does NOT short-circuit on the
@@ -153,6 +159,17 @@ if [ "$status" -ne 0 ]; then
exit "$status"
fi
+if [ "$CHECK_KEY" = "pytest" ] && [ "$#" -gt 0 ]; then
+ # The list goes through stdin, not a temp file: a sandboxed agent may
+ # not be able to write the system temp directory.
+ applicable=$(printf '%s\n' "$@" | python3 tools/dev/ci-pytest-batches.py --changed-file /dev/stdin --paths)
+ status=$?
+ if [ "$status" -ne 0 ]; then
+ exit "$status"
+ fi
+ set --
+fi
+
if [ -z "$applicable" ]; then
echo "→ ${CHECK_KEY}: no applicable workspace members (nothing to run)"
exit 0
diff --git a/tools/dev/tests/test_ci_pytest_batches.py b/tools/dev/tests/test_ci_pytest_batches.py
index 7f39ce938..6959a8e3c 100644
--- a/tools/dev/tests/test_ci_pytest_batches.py
+++ b/tools/dev/tests/test_ci_pytest_batches.py
@@ -74,6 +74,9 @@ def test_only_members_with_pytest_and_no_opt_out_are_picked(tmp_path):
(["tools/a/src/x.py"], ["tools/a"]),
(["tools/a/tests/fixtures/data.json"], ["tools/a"]),
(["tools/a/README.md"], []),
+ (["tools/a/scripts/run.sh"], ["tools/a"]),
+ (["tools/a/pyproject.toml"], ["tools/a"]),
+ (["tools/a/.claude-plugin/plugin.json"], []),
(["uv.lock"], ["tools/a", "tools/b"]),
(["tools/dev/ci-pytest-batches.py"], ["tools/a", "tools/b"]),
([], []),
@@ -84,6 +87,62 @@ def test_a_pull_request_picks_only_what_it_changed(tmp_path, changed, picked):
assert cpb.select(root, changed) == picked
+def test_a_change_behind_a_member_symlink_picks_the_member(tmp_path):
+ root = _workspace(tmp_path, {"tools/a": PYTEST, "tools/b": PYTEST})
+ (root / "plugins" / "shared").mkdir(parents=True)
+ (root / "tools" / "a" / "src").mkdir()
+ (root / "tools" / "a" / "src" / "shared").symlink_to(root / "plugins" / "shared")
+ assert cpb.select(root, ["plugins/shared/x.py"]) == ["tools/a"]
+
+
+@pytest.mark.parametrize(
+ ("changed", "picked"),
+ [
+ (["tools/c/src/z.py"], ["tools/a", "tools/b", "tools/c"]),
+ (["tools/b/src/y.py"], ["tools/a", "tools/b"]),
+ (["tools/a/src/x.py"], ["tools/a"]),
+ ],
+)
+def test_a_change_picks_the_members_depending_on_it(tmp_path, changed, picked):
+ root = _workspace(
+ tmp_path,
+ {
+ "tools/a": '[project.optional-dependencies]\nall = ["pkg-b>=1"]\n' + PYTEST,
+ "tools/b": 'dependencies = ["Pkg_C"]\n' + PYTEST,
+ "tools/c": PYTEST,
+ },
+ )
+ assert cpb.select(root, changed) == picked
+
+
+def test_a_dependency_group_makes_a_dependent(tmp_path):
+ root = _workspace(
+ tmp_path,
+ {"tools/a": '[dependency-groups]\ndev = ["pkg-b"]\n' + PYTEST, "tools/b": PYTEST},
+ )
+ assert cpb.select(root, ["tools/b/src/y.py"]) == ["tools/a", "tools/b"]
+
+
+def test_the_shared_toolchain_does_not_make_a_dependent(tmp_path):
+ root = _workspace(
+ tmp_path,
+ {
+ "tools/a": '[dependency-groups]\ndev = ["magpie-dev"]\n' + PYTEST,
+ "tools/dev": PYTEST,
+ },
+ )
+ (root / "tools" / "dev" / "pyproject.toml").write_text('[project]\nname = "magpie-dev"\n' + PYTEST)
+ assert cpb.select(root, ["tools/dev/src/z.py"]) == ["tools/dev"]
+
+
+def test_the_cli_prints_the_picked_paths(tmp_path, capsys):
+ root = _workspace(tmp_path, {"tools/a": PYTEST, "tools/b": PYTEST})
+ changed = tmp_path / "changed.txt"
+ changed.write_text("tools/a/src/x.py\ntools/b/tests/test_y.py\n")
+ assert cpb.main(["--root", str(root), "--changed-file", str(changed), "--paths"]) == 0
+ assert capsys.readouterr().out == "tools/a\ntools/b\n"
+
+
def test_up_to_solo_max_members_get_one_job_each_named_after_the_member():
paths = [f"tools/m{i}" for i in range(cpb.SOLO_MAX)]
out = cpb.batches(paths, dict.fromkeys(paths, 1))
diff --git a/uv.lock b/uv.lock
index 491435fa0..e6049d88e 100644
--- a/uv.lock
+++ b/uv.lock
@@ -632,6 +632,15 @@ requires-dist = [{ name = "proxy-py", specifier = ">=2.4,<3" }]
[package.metadata.requires-dev]
dev = [{ name = "magpie-dev", editable = "tools/dev" }]
+[[package]]
+name = "execnet"
+version = "2.1.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/bf/89/780e11f9588d9e7128a3f87788354c7946a9cbb1401ad38a48c4db9a4f07/execnet-2.1.2.tar.gz", hash = "sha256:63d83bfdd9a23e35b9c6a3261412324f964c2ec8dcd8d3c6916ee9373e0befcd", size = 166622, upload-time = "2025-11-12T09:56:37.75Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/ab/84/02fc1827e8cdded4aa65baef11296a9bbe595c474f0d6d758af082d849fd/execnet-2.1.2-py3-none-any.whl", hash = "sha256:67fba928dd5a544b783f6056f449e5e3931a5c378b128bc18501f7ea79e296ec", size = 40708, upload-time = "2025-11-12T09:56:36.333Z" },
+]
+
[[package]]
name = "generate-cve-json"
version = "0.1.0"
@@ -958,6 +967,7 @@ source = { editable = "tools/dev" }
dependencies = [
{ name = "mypy" },
{ name = "pytest" },
+ { name = "pytest-xdist" },
{ name = "ruff" },
]
@@ -965,6 +975,7 @@ dependencies = [
requires-dist = [
{ name = "mypy", specifier = ">=2.3.0" },
{ name = "pytest", specifier = ">=9.1.1" },
+ { name = "pytest-xdist", specifier = ">=3.8.0" },
{ name = "ruff", specifier = ">=0.16.2" },
]
@@ -1537,6 +1548,19 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" },
]
+[[package]]
+name = "pytest-xdist"
+version = "3.8.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "execnet" },
+ { name = "pytest" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/78/b4/439b179d1ff526791eb921115fca8e44e596a13efeda518b9d845a619450/pytest_xdist-3.8.0.tar.gz", hash = "sha256:7e578125ec9bc6050861aa93f2d59f1d8d085595d6551c2c90b6f4fad8d3a9f1", size = 88069, upload-time = "2025-07-01T13:30:59.346Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/ca/31/d4e37e9e550c2b92a9cbc2e4d0b7420a27224968580b5a447f420847c975/pytest_xdist-3.8.0-py3-none-any.whl", hash = "sha256:202ca578cfeb7370784a8c33d6d05bc6e13b4f25b5053c30a152269fd10f0b88", size = 46396, upload-time = "2025-07-01T13:30:56.632Z" },
+]
+
[[package]]
name = "python-multipart"
version = "0.0.32"