You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@magpie.apache.org by po...@apache.org on 2026/07/02 22:32:45 UTC
(magpie) branch main updated: feat(tools): model pull-request review/merge as contract:change-request (#678)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/magpie.git
The following commit(s) were added to refs/heads/main by this push:
new 950efdcd feat(tools): model pull-request review/merge as contract:change-request (#678)
950efdcd is described below
commit 950efdcda9caf47e2b6568fb4e5ce69cb77f6a2e
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Fri Jul 3 00:32:40 2026 +0200
feat(tools): model pull-request review/merge as contract:change-request (#678)
The vendor-neutrality scorer classified every pr-management skill as
"portable" because they invoke `gh pr`, which matched contract:tracker —
and tracker is green (GitHub + Atlassian). But Jira has no pull-request
model: its adapter covers issues, not PR review/merge. So a skill driving
`gh pr` is coupled to GitHub even though the issue side of tracker is
portable.
Introduce contract:change-request — the proposed-change review + merge gate
(pull request / merge request / Gerrit change) — as a distinct vendor-backed
contract with GitHub as the sole backend today. `gh pr` moves from the
tracker usage tokens into change-request; tag tools/github with it.
Result: 8/10 contracts vendor neutral (was 8/9); change-request joins
mail-draft as the second open gap. 23 skills are vendor-coupled (the 19
that drive `gh pr`, plus the 4 mail-draft ones) — each flips to portable
the moment a second forge review/merge backend (GitLab, Gitea) ships.
Regenerate the docs/vendor-neutrality.md block (a pytest keeps it in sync)
and register the contract in the skill-and-tool-validator allow-set.
Raised on apache/magpie-site#17.
Co-authored-by: Tester <t...@example.com>
---
docs/labels-and-capabilities.md | 7 +--
docs/vendor-neutrality.md | 50 ++++++++++++++++------
tools/github/README.md | 2 +-
.../src/skill_and_tool_validator/__init__.py | 1 +
.../src/vendor_neutrality_score/__init__.py | 17 ++++++--
.../tests/test_vendor_neutrality_score.py | 16 ++++++-
6 files changed, 73 insertions(+), 20 deletions(-)
diff --git a/docs/labels-and-capabilities.md b/docs/labels-and-capabilities.md
index db2d4c73..4ca75e77 100644
--- a/docs/labels-and-capabilities.md
+++ b/docs/labels-and-capabilities.md
@@ -97,8 +97,9 @@ framework substrate:
| Label | Kind | Definition |
|---|---|---|
-| `contract:tracker` | contract | Issue / PR / board / label backend. |
+| `contract:tracker` | contract | Issue / board / label backend. |
| `contract:source-control` | contract | Branch / commit / diff / push (VCS). |
+| `contract:change-request` | contract | Proposed-change review + merge gate (pull request / merge request / Gerrit change). |
| `contract:mail-archive` | contract | Mailing-list / forum archive reads. |
| `contract:mail-source` | contract | Inbound-mail ingestion (mbox / IMAP / …). |
| `contract:mail-draft` | contract | Outbound mail composition (draft, never send). |
@@ -238,7 +239,7 @@ it implements multiple contracts (e.g. `tools/gmail` provides both
| [`tools/dev`](../tools/dev/) | `substrate:framework-dev` | Framework dev-loop helpers |
| [`tools/egress-gateway`](../tools/egress-gateway/) | `substrate:sandbox` | Egress-allowlist forward proxy (proxy.py plugin); host-level egress chokepoint — defence-in-depth for RFC-AI-0003 §4.4 |
| [`tools/forwarder-relay`](../tools/forwarder-relay/) | `contract:report-relay` | Adapter contract for inbound-relay backends (ASF Security relay, huntr.com, HackerOne triagers). Pure interface spec; adapters declare detection + credit-extraction + reporter-addressing rules. |
-| [`tools/github`](../tools/github/) | `contract:tracker` + `contract:source-control` | GitHub REST / GraphQL tracker substrate (called by every lifecycle phase) plus the Git source-control binding documented in [`source-control.md`](../tools/github/source-control.md) (runnable backend in [`tools/vcs`](../tools/vcs/)) |
+| [`tools/github`](../tools/github/) | `contract:tracker` + `contract:source-control` + `contract:change-request` | GitHub REST / GraphQL tracker substrate (called by every lifecycle phase) plus the Git source-control binding documented in [`source-control.md`](../tools/github/source-control.md) (runnable backend in [`tools/vcs`](../tools/vcs/)) and the pull-request review/merge gate (`change-request`; the sole backend today — Jira has no PR model) |
| [`tools/github-body-field`](../tools/github-body-field/) | `contract:tracker` | Read or rewrite one `### Field` section of a GitHub issue body without bringing the body into agent context — substrate helper for the security-sync skills |
| [`tools/github-rollup`](../tools/github-rollup/) | `contract:tracker` | Append to (or create) the status-rollup comment on a GitHub issue without bringing the rollup body into agent context — substrate helper for every status-update-emitting skill |
| [`tools/gmail`](../tools/gmail/) | `contract:mail-source` + `contract:mail-draft` + `contract:mail-archive` | Gmail API substrate — inbound report intake (`mail-source`), thread / archive reads (`mail-archive`), plus outbound courtesy-reply drafting (`mail-draft`); read + draft only, never sends |
@@ -285,7 +286,7 @@ backend the adopter wired in. The framework consumes four:
| MCP server | Tool prefix | Wrapped by | Capability provided | Organization |
|---|---|---|---|---|
-| GitHub MCP | `mcp__github__*` | [`tools/github`](../tools/github/) | `contract:tracker` + `contract:source-control` | — |
+| GitHub MCP | `mcp__github__*` | [`tools/github`](../tools/github/) | `contract:tracker` + `contract:source-control` + `contract:change-request` | — |
| Gmail MCP (claude.ai) | `mcp__claude_ai_Gmail__*` | [`tools/gmail`](../tools/gmail/) | `contract:mail-source` + `contract:mail-draft` + `contract:mail-archive` | — |
| PonyMail MCP (`apache/comdev`) | `mcp__ponymail__*` | [`tools/ponymail`](../tools/ponymail/) | `contract:mail-archive` + `contract:mail-source` | ASF |
| apache-projects MCP (`apache/comdev`) | `mcp__apache-projects__*` | [`tools/apache-projects`](../tools/apache-projects/) | `contract:project-metadata` | ASF |
diff --git a/docs/vendor-neutrality.md b/docs/vendor-neutrality.md
index c69c9405..ce40007d 100644
--- a/docs/vendor-neutrality.md
+++ b/docs/vendor-neutrality.md
@@ -533,12 +533,13 @@ alternative, and *vendor-coupled* only if it reaches for the sole
implementation of a capability.
<!-- BEGIN vendor-neutrality-score — generated by `uv run --project tools/vendor-neutrality-score vendor-neutrality-score --markdown`; do not edit by hand -->
-**Overall vendor-neutrality score: 8/9 capability contracts (89%).** Generated by [`tools/vendor-neutrality-score`](../tools/vendor-neutrality-score/); re-run it to refresh this section.
+**Overall vendor-neutrality score: 8/10 capability contracts (80%).** Generated by [`tools/vendor-neutrality-score`](../tools/vendor-neutrality-score/); re-run it to refresh this section.
| Capability contract | Neutral? | Class | Backends today | Basis |
|---|---|---|---|---|
| `contract:tracker` | ✅ | vendor-backed | Atlassian, GitHub | 2 backend vendors: Atlassian, GitHub |
| `contract:source-control` | ✅ | vendor-backed | Git, GitHub, Subversion | 3 backend vendors: Git, GitHub, Subversion |
+| `contract:change-request` | ❌ | vendor-backed | GitHub | only 1 backend vendor (GitHub); needs 1 more |
| `contract:mail-archive` | ✅ | vendor-backed | Google, PonyMail | 2 backend vendors: Google, PonyMail |
| `contract:mail-source` | ✅ | vendor-backed | Google, PonyMail | 2 backend vendors: Google, PonyMail |
| `contract:mail-draft` | ❌ | vendor-backed | Google | only 1 backend vendor (Google); needs 1 more |
@@ -547,34 +548,59 @@ implementation of a capability.
| `contract:scan-format` | ✅ | agnostic | — | vendor-neutral by construction — one spec serves every backend |
| `contract:project-metadata` | ✅ | single-org | ASF | single-organisation capability (ASF); no vendor choice to make |
-**Per-skill assessment: 59/63 skills carry no vendor lock-in.** A skill is *capability-pure* when it names no backend at all, *portable* when every backend it names has an alternative (its contract is green), and *vendor-coupled* only when it reaches for a backend that is the sole implementation of a capability.
+**Per-skill assessment: 40/63 skills carry no vendor lock-in.** A skill is *capability-pure* when it names no backend at all, *portable* when every backend it names has an alternative (its contract is green), and *vendor-coupled* only when it reaches for a backend that is the sole implementation of a capability.
| Skill neutrality | Count |
|---|---|
| capability-pure (names no backend) | 9 |
-| portable (named backends are swappable) | 50 |
-| vendor-coupled (sole-backend dependency) | 4 |
+| portable (named backends are swappable) | 31 |
+| vendor-coupled (sole-backend dependency) | 23 |
Organization scope (declared, orthogonal to vendor): ASF = 14, agnostic = 49.
Vendor-coupled skills (the only lock-ins today):
+- `audit-finding-fix` — `GitHub` → `contract:change-request`
+- `committer-onboarding` — `GitHub` → `contract:change-request`
+- `issue-fix-workflow` — `GitHub` → `contract:change-request`
+- `mentoring-welcome` — `GitHub` → `contract:change-request`
+- `pr-management-code-review` — `GitHub` → `contract:change-request`
+- `pr-management-mentor` — `GitHub` → `contract:change-request`
+- `pr-management-quick-merge` — `GitHub` → `contract:change-request`
+- `pr-management-stats` — `GitHub` → `contract:change-request`
+- `pr-management-triage` — `GitHub` → `contract:change-request`
+- `release-announce-draft` — `GitHub` → `contract:change-request`
+- `release-audit-report` — `GitHub` → `contract:change-request`
+- `release-prepare` — `GitHub` → `contract:change-request`
+- `release-vote-tally` — `GitHub` → `contract:change-request`
+- `reviewer-routing` — `GitHub` → `contract:change-request`
+- `security-issue-fix` — `GitHub` → `contract:change-request`
- `security-issue-import` — `Google` → `contract:mail-draft`
+- `security-issue-import-from-pr` — `GitHub` → `contract:change-request`
+- `security-issue-import-from-scan` — `GitHub` → `contract:change-request`
- `security-issue-import-via-forwarder` — `Google` → `contract:mail-draft`
- `security-issue-invalidate` — `Google` → `contract:mail-draft`
- `security-issue-sync` — `Google` → `contract:mail-draft`
+- `security-issue-triage` — `GitHub` → `contract:change-request`
+- `setup-override-upstream` — `GitHub` → `contract:change-request`
<!-- END vendor-neutrality-score -->
### What the number means
-89% is not "89% done." It reads as: **eight of nine capabilities already
-work across more than one vendor, and the ninth — outbound mail drafting
-— is one adapter away.** The architecture privileges no vendor on any of
-the nine axes; the single red cell is a missing *implementation* (a
-second `mail-draft` backend), tracked in the open, not a design that
-assumes Gmail. Likewise only four skills touch that one lock-in, and each
-does so through the `mail-draft` contract — swap in a second backend and
-all four become portable without a line of skill code changing.
+80% is not "80% done." It reads as: **eight of ten capabilities already
+work across more than one vendor, and the two that don't — the
+pull-request review/merge gate (`change-request`) and outbound mail
+drafting (`mail-draft`) — are each one adapter away.** The architecture
+privileges no vendor on any of the ten axes; the two red cells are
+missing *implementations* (a second forge that reviews/merges proposed
+changes, e.g. GitLab or Gitea; and a second `mail-draft` backend),
+tracked in the open — not a design that assumes GitHub or Gmail. The
+change-request gap is the larger of the two: `contract:tracker` is green
+because Jira also handles issues, but Jira has no pull-request model, so
+every skill that drives `gh pr` (review, merge, triage) is coupled to
+GitHub today. Ship one non-GitHub review/merge backend and all of those
+skills become portable without a line of skill code changing — the seam
+is the contract, not the workflow.
## What "vendor neutral" does and does not claim
diff --git a/tools/github/README.md b/tools/github/README.md
index c2ebc069..6aae4078 100644
--- a/tools/github/README.md
+++ b/tools/github/README.md
@@ -12,7 +12,7 @@
# `tools/github/`
-**Capability:** contract:tracker + contract:source-control
+**Capability:** contract:tracker + contract:source-control + contract:change-request
**Kind:** implementation
diff --git a/tools/skill-and-tool-validator/src/skill_and_tool_validator/__init__.py b/tools/skill-and-tool-validator/src/skill_and_tool_validator/__init__.py
index fce2c731..69ca65bc 100644
--- a/tools/skill-and-tool-validator/src/skill_and_tool_validator/__init__.py
+++ b/tools/skill-and-tool-validator/src/skill_and_tool_validator/__init__.py
@@ -275,6 +275,7 @@ SKILL_CAPABILITIES = {
TOOL_CAPABILITIES = {
"contract:tracker",
"contract:source-control",
+ "contract:change-request",
"contract:mail-archive",
"contract:mail-source",
"contract:mail-draft",
diff --git a/tools/vendor-neutrality-score/src/vendor_neutrality_score/__init__.py b/tools/vendor-neutrality-score/src/vendor_neutrality_score/__init__.py
index 0703741e..db952fdc 100644
--- a/tools/vendor-neutrality-score/src/vendor_neutrality_score/__init__.py
+++ b/tools/vendor-neutrality-score/src/vendor_neutrality_score/__init__.py
@@ -73,8 +73,12 @@ MIN_VENDORS = 2
# contract -> (neutrality class, human-readable capability summary)
CONTRACT_POLICY: dict[str, tuple[str, str]] = {
- "contract:tracker": (VENDOR_BACKED, "Issue / PR / board / label backend"),
+ "contract:tracker": (VENDOR_BACKED, "Issue / board / label backend"),
"contract:source-control": (VENDOR_BACKED, "Branch / commit / diff / push (VCS)"),
+ "contract:change-request": (
+ VENDOR_BACKED,
+ "Proposed-change review + merge gate (PR / MR / Gerrit change)",
+ ),
"contract:mail-archive": (VENDOR_BACKED, "Mailing-list / forum archive reads"),
"contract:mail-source": (VENDOR_BACKED, "Inbound-mail ingestion (mbox / IMAP / …)"),
"contract:mail-draft": (VENDOR_BACKED, "Outbound mail composition (draft, never send)"),
@@ -89,16 +93,23 @@ CONTRACT_POLICY: dict[str, tuple[str, str]] = {
# and canonical hostnames. A skill is coupled only when a contract it
# invokes has no alternative backend — so this maps *usage*, not prose
# mentions (a skill that merely names "GitHub" in a sentence is not caught;
-# one that calls ``gh pr`` or ``mcp__github__*`` is). Contracts that are
+# one that calls ``gh issue`` or ``mcp__github__*`` is). Contracts that are
# agnostic by construction (report-relay, scan-format) and mail-source
# (indistinguishable from archive reads at the token level) are omitted —
# they never change a skill's verdict.
+#
+# ``gh pr`` is deliberately NOT a tracker signal: driving pull requests is the
+# change-request contract (below), which only GitHub implements today. Jira —
+# the second tracker backend — has no pull-request model, so a skill that runs
+# ``gh pr`` is coupled to GitHub even though the *issue* side of tracker is
+# portable.
CONTRACT_USAGE_TOKENS: dict[str, tuple[str, ...]] = {
"contract:tracker": (
r"mcp__github__",
- r"\bgh (?:pr|issue|api|search|run|workflow|release|label)\b",
+ r"\bgh (?:issue|api|search|run|workflow|release|label)\b",
r"\bJIRA\b",
),
+ "contract:change-request": (r"\bgh pr\b",),
"contract:source-control": (
r"\bgit (?:commit|push|checkout|branch|rebase|merge|switch|worktree)\b",
r"\bsvn (?:checkout|commit|update|cat|list|mkdir|import|move|delete|switch|copy)\b",
diff --git a/tools/vendor-neutrality-score/tests/test_vendor_neutrality_score.py b/tools/vendor-neutrality-score/tests/test_vendor_neutrality_score.py
index e61cb715..eb5b4531 100644
--- a/tools/vendor-neutrality-score/tests/test_vendor_neutrality_score.py
+++ b/tools/vendor-neutrality-score/tests/test_vendor_neutrality_score.py
@@ -126,13 +126,27 @@ def test_skill_capability_pure_when_no_backend_named() -> None:
def test_skill_portable_when_named_contract_is_green() -> None:
- skills = [("triage", "agnostic", "Run `gh pr list` then classify each PR.")]
+ skills = [("triage", "agnostic", "Run `gh issue list` then classify each issue.")]
(s,) = vns.score_skills(skills, _contract_results_with_gap())
assert s.verdict == "portable"
assert s.contracts == ["contract:tracker"]
assert s.coupled == []
+def test_skill_vendor_coupled_on_github_only_change_request() -> None:
+ # Driving pull requests (`gh pr`) is the change-request contract, which only
+ # GitHub implements — so a skill that runs it is coupled to GitHub even
+ # though the issue side of tracker is portable.
+ tools = [
+ _tool("github", "contract:change-request", vns.IMPLEMENTATION, "GitHub"),
+ ]
+ contracts = vns.score_contracts(tools)
+ skills = [("merge", "agnostic", "Run `gh pr merge --squash` after review.")]
+ (s,) = vns.score_skills(skills, contracts)
+ assert s.verdict == "vendor-coupled"
+ assert s.coupled == [("GitHub", "contract:change-request")]
+
+
def test_skill_vendor_coupled_on_sole_backend_contract() -> None:
skills = [("reply", "ASF", "Draft a courtesy reply with mcp__claude_ai_Gmail__create_draft.")]
(s,) = vns.score_skills(skills, _contract_results_with_gap())