You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@magpie.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2026/10/13 14:11:37 UTC
[PR] perf(release-promote, release-announce-draft): render the promotion and announcement as code [magpie]
potiuk opened a new pull request, #1618:
URL: https://github.com/apache/magpie/pull/1618
## Summary
Fourth PR of the stack; it builds on the vote PR.
- **`release-promote`:**
- `verdict` fails closed: no `--vote-passed` means blocked.
- `render` draws the per-backend block: svnpubsub `svn mv` + `svn list` + the final signed tag on the RC commit; github-releases, which refuses a draft-tag mismatch until it is confirmed; s3; and self-hosted.
- It also draws the hand-off note, the convenience publish commands, the HOLD notes and the mandatory mirror note.
- A self-hosted backend without a template is now also a Step 0 blocker in release-config.
- **`release-announce-draft`:**
- `verdict` adds the `promoted` check.
- `render --step 2` draws the `[ANNOUNCE]` for all four backends. The `@apache.org` reminder is always present (Golden rule 4), and a `dist.apache.org` Download Page gets a warning (Golden rule 5).
- `render --step 3` draws the site-bump PR and computes scope violations as a set difference.
- The site-bump step moves, byte-identical with its generated adversarial block, to `site-bump.md` at the skill root.
- **Size:** promote goes from 6,982 to 4,886 tokens, announce-draft from 6,809 to 5,662.
## Security notes
Every rendered command quotes its values with the shared shell helpers. That matters most here: the RC commit SHA and the vote-result URL come from the planning issue, which is external content. A test proves both come out as inert words.
## Behaviour changes worth a look
- **A bug fix:** the svnpubsub template's `\ # release_dist_backend=svnpubsub` continuation lines broke the pasted command, so the comments are gone. The step headers keep the backend tag.
- **The default `[ANNOUNCE]` body drops its `<NOTE: do not include direct dist.apache.org links…>` line.** It was a drafting instruction inside the template; the rule is now enforced as a warning.
- **When `announce_cc_lists` is empty, the Cc line is omitted.**
## Type of change
- [x] Skill change — eval fixtures updated below
- [x] Python package
## Test plan
- [x] `prek run --all-files` passes on the stack tip.
- [x] `tools/release-management`: pytest / ruff / mypy, with two real-`load` integration tests. `tools/release-config` has a new self-hosted blocker test.
- [x] Evals: the scripted cases are now pytest cases named after each former case. The kept injection and verdict-smoke cases pass 2/2 per skill, against a baseline of 9/9 each.
---
##### Was generative AI tooling used to co-author this PR?
- [x] Yes — Claude Code (Opus 5). I reviewed the diff and the test results; commits carry `Generated-by:`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01GTPsZ5aEE5bUVYqp47Dpvv
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@magpie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(release-promote, release-announce-draft): render the promotion and announcement as code [magpie]
Posted by "github-advanced-security[bot] (via GitHub)" <gi...@apache.org>.
github-advanced-security[bot] commented on code in PR #1618:
URL: https://github.com/apache/magpie/pull/1618#discussion_r4261704738
##########
tools/release-management/tests/test_announce_draft.py:
##########
@@ -0,0 +1,346 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""release-announce-draft: the Step 0 verdict, the [ANNOUNCE] draft and the site-bump PR.
+
+The scripted cases of the retired model-graded suites `step-0-preflight`
+(cases 1 and 2), `step-2-announce-draft` (cases 1 to 3) and `step-3-site-bump`
+(cases 1 and 2) live here, named after each former case, with the same scenario
+and the same assertions.
+"""
+
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+import pytest
+
+from release_management.announce_draft import render
+from release_management.cli import main
+
+DOWNLOAD = "https://airflow.apache.org/docs/apache-airflow/2.11.0/installation/installing-from-pypi.html"
+CHANGELOG = "https://github.com/apache/airflow/blob/2.11.0/CHANGELOG.md"
+WAIT = (
+ "Promote-wait gate: promote commit was at 2026-06-11T09:45:00Z; the one-hour gate clears at "
+ "2026-06-11T10:45:00Z (in ~30 minutes). Pass --skip-promote-wait <reason> to override."
+)
+SITE_FILES = [
+ "landing-pages/site/content/en/_index.md",
+ "landing-pages/site/content/en/announcements/2.11.0.md",
+]
+
+
+def preflight(
+ *, blockers: list[str] | None = None, clear: str | None = None, wait: bool = False, skip: bool = False
+) -> dict[str, Any]:
+ return {
+ "ok": not blockers,
+ "skill": "announce-draft",
+ "blockers": blockers or [],
+ "warnings": [],
+ "values": {
+ "version": "2.11.0",
+ "skip_promote_wait_override": skip,
+ "non_asf": False,
+ "promote_clear_after_utc": clear,
+ "promote_wait_active": wait,
+ "download_page_url": DOWNLOAD,
+ "release_announce_backend": "announce-list",
+ },
+ }
+
+
+def loaded(
+ *,
+ version: str = "2.11.0",
+ backend: str = "announce-list",
+ subject: str | None = "[ANNOUNCE] Apache Airflow <version> released",
+ keys: str | None = "https://dist.apache.org/repos/dist/release/airflow/KEYS",
+ announce_list: str | None = "announce@apache.org",
+ cc: list[str] | None = None,
+ site_repo: str | None = "apache/airflow-site",
+ site_files: list[str] | None = None,
+) -> dict[str, Any]:
+ return {
+ "metadata": {
+ "version": version,
+ "promote_timestamp": "2026-06-10T08:00:00Z",
+ "keys_url": keys,
+ "announce_list": announce_list,
+ "announce_cc_lists": ["dev@airflow.apache.org", "users@airflow.apache.org"] if cc is None else cc,
+ "subject_template": subject,
+ "site_repo": site_repo,
+ "site_pr_files": SITE_FILES if site_files is None else site_files,
+ "release_announce_backend": backend,
+ "dist_release_url": f"https://dist.apache.org/repos/dist/release/airflow/{version}/",
+ "convenience_artefacts": [],
+ }
+ }
+
+
+def announce(product: str = "Apache Airflow", **kw: Any) -> render.Announce:
+ return render.Announce.from_load(loaded(**kw), product_name=product)
+
+
+# The step-2-announce-draft assertions.json predicates, on `body`.
+HAS = {
+ "apache_org_sender_reminder": r"@apache\.org address",
+ "download_page_link": r"download page[\s\S]{0,120}https?://",
+ "download_link": r"download[\s\S]{0,120}https?://",
+ "changelog_link": r"(?:changelog|release notes)[\s\S]{0,120}https?://",
+ "keys_link": r"https?://\S*KEYS",
+ "skip_promote_wait_note": r"skip-promote-wait|promote-wait gate overridden",
+}
+
+
+def has(key: str, body: str) -> bool:
+ return re.search(HAS[key], body, re.I) is not None
+
+
+# --- step-0-preflight ------------------------------------------------------ #
+
+
+def test_step0_case_1_clean_pass() -> None:
+ out = render.verdict(preflight(), promoted=True)
+ assert {k: out[k] for k in out if k != "read"} == {
+ "verdict": "proceed",
+ "blockers": [],
+ "skip_promote_wait_override": False,
+ "non_asf": False,
+ "promote_clear_after_utc": None,
+ }
+
+
+def test_step0_case_2_not_promoted() -> None:
+ out = render.verdict(
+ preflight(blockers=["promote timestamp unavailable", "Download Page URL unavailable"]), promoted=False
+ )
+ assert out["verdict"] == "blocked"
+ assert out["skip_promote_wait_override"] is False and out["non_asf"] is False
+ assert out["promote_clear_after_utc"] is None
+ assert render.NOT_PROMOTED in out["blockers"]
+
+
+def test_step0_promote_wait_only_blocker_keeps_the_clear_time() -> None:
+ out = render.verdict(preflight(blockers=[WAIT], clear="2026-06-11T10:45:00Z", wait=True), promoted=True)
+ assert out["verdict"] == "blocked"
+ assert out["blockers"] == [WAIT] # verbatim from release-config
+ assert out["promote_clear_after_utc"] == "2026-06-11T10:45:00Z"
+ assert out["read"] == ["outcomes/promote-wait.md"]
+
+
+def test_step0_label_check_also_blocking_nulls_the_clear_time() -> None:
+ out = render.verdict(preflight(blockers=[WAIT], clear="2026-06-11T10:45:00Z", wait=True), promoted=False)
+ assert out["verdict"] == "blocked" and len(out["blockers"]) == 2
+ assert out["promote_clear_after_utc"] is None
+
+
+def test_step0_skip_override_proceeds() -> None:
+ out = render.verdict(preflight(wait=True, skip=True), promoted=True)
+ assert out["verdict"] == "proceed" and out["skip_promote_wait_override"] is True and out["read"] == []
+
+
+def test_step0_no_planning_issue_blocks() -> None:
+ out = render.verdict(preflight(), promoted=True, planning_issue_found=False)
+ assert out["verdict"] == "blocked" and "no planning issue found" in out["blockers"][0]
+
+
+# --- step-2-announce-draft ------------------------------------------------- #
+
+
+def test_step2_case_1_standard_announce() -> None:
+ out = render.step2(announce(), download_page=DOWNLOAD, changelog_url=CHANGELOG)
+ body = out["body"]
+ assert out["subject"] == "[ANNOUNCE] Apache Airflow 2.11.0 released"
+ assert out["backend"] == "announce-list"
+ assert out["skip_promote_wait_logged"] is False
+ assert out["asf_address_reminder_present"] is True
+ for key in ("apache_org_sender_reminder", "download_page_link", "changelog_link", "keys_link"):
+ assert has(key, body), key
+ assert not has("skip_promote_wait_note", body)
+ assert body.startswith(
+ "To: announce@apache.org\nCc: dev@airflow.apache.org, users@airflow.apache.org\n"
+ "Subject: [ANNOUNCE] Apache Airflow 2.11.0 released\n"
+ )
+ assert (
+ "The Apache Airflow community is pleased to announce the release\nof Apache Airflow 2.11.0." in body
+ )
+ assert "The\nannounce@apache.org rejects non-@apache.org senders" in body
+ assert "dist.apache.org/repos/dist/release/airflow/2.11.0" not in body # Download Page, not dist
+ assert out["download_page_is_dist"] is False and out["warnings"] == []
+
+
+def test_step2_case_2_skip_promote_wait_logged() -> None:
+ reason = "Critical security release; mirrors propagated within 30 minutes per infra confirmation."
+ out = render.step2(announce(), download_page=DOWNLOAD, changelog_url=CHANGELOG, skip_reason=reason)
+ body = out["body"]
+ assert out["subject"] == "[ANNOUNCE] Apache Airflow 2.11.0 released"
+ assert out["backend"] == "announce-list"
+ assert out["skip_promote_wait_logged"] is True and out["asf_address_reminder_present"] is True
+ for key in (
+ "apache_org_sender_reminder",
+ "download_page_link",
+ "changelog_link",
+ "keys_link",
+ "skip_promote_wait_note",
+ ):
+ assert has(key, body), key
+ assert reason in body
+
+
+def test_step2_case_3_non_asf_github_releases() -> None:
+ a = render.Announce.from_load(
+ loaded(
+ version="1.5.0",
+ backend="github-release-notes",
+ subject="MyProject <version> released",
+ keys=None,
+ announce_list=None,
+ cc=[],
+ ),
+ product_name="MyProject",
+ )
+ out = render.step2(
+ a,
+ download_page="https://myproject.example.com/download",
+ changelog_url="https://github.com/example-org/myproject/blob/v1.5.0/CHANGELOG.md",
+ )
+ body = out["body"]
+ assert out["subject"] == "MyProject 1.5.0 released"
+ assert out["backend"] == "github-release-notes"
+ assert out["skip_promote_wait_logged"] is False
+ assert out["asf_address_reminder_present"] is False
+ assert not has("apache_org_sender_reminder", body)
+ assert has("download_link", body) and has("changelog_link", body)
+ assert "To:" not in body and "Cc:" not in body
+ assert "## Downloads" in body and "## Changelog" in body
+
+
+def test_step2_site_post_and_discord_shapes() -> None:
+ kw: dict[str, Any] = {"download_page": "https://p.example/dl", "changelog_url": "https://p.example/cl"}
+ post = render.step2(announce("MyProject", backend="site-post", announce_list=None), **kw)
+ assert post["body"].startswith("## MyProject 2.11.0 released")
+ assert "[Download Page](https://p.example/dl)" in post["body"]
+ assert post["asf_address_reminder_present"] is False
+ discord = render.step2(announce("MyProject", backend="discord-channel"), **kw)["body"]
+ assert (
+ discord.count("\n- ") == 2 and "https://p.example/dl" in discord and "https://p.example/cl" in discord
+ )
+
+
+def test_step2_dist_apache_download_page_warns() -> None:
+ out = render.step2(
+ announce(),
+ download_page="https://dist.apache.org/repos/dist/release/airflow/2.11.0/",
+ changelog_url=CHANGELOG,
+ )
+ assert out["download_page_is_dist"] is True
+ assert any("dist.apache.org" in w and "Golden rule 5" in w for w in out["warnings"])
Review Comment:
## CodeQL / Incomplete URL substring sanitization
The string [dist.apache.org](1) may be at an arbitrary position in the sanitized URL.
[Show more details](https://github.com/apache/magpie/security/code-scanning/122)
##########
tools/release-management/src/release_management/promote/render.py:
##########
@@ -0,0 +1,399 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Assemble release-promote's Step 0 verdict and render its Step 2 command set.
+
+The functions are pure: they take what `release-config preflight` / `load
+--skill promote` printed and the facts the agent read (the planning-issue
+labels, the target listing, the verify-rc attestation), and return the step's
+JSON contract. Nothing here runs `svn`, `gh`, `aws` or `git`, and nothing
+applies a label: the release manager runs every command (Golden rule 1).
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any
+from urllib.parse import urlparse
+
+from release_management.shell import dq, w
+
+PROPOSED_LABEL = "promoted"
+DEFAULT_ROSTER = "<project-config>/pmc-roster.md"
+
+MIRROR_NOTE = """\
+Mirror propagation (svnpubsub) / CDN cache (other backends):
+ Allow up to 24 hours for the promoted release to appear on all mirrors.
+ ASF policy requires waiting at least 1 hour after the promote commit
+ before updating the download page or sending the [ANNOUNCE] email.
+ Earliest announce time: <promote_timestamp + 1h> UTC (once the promote commit is confirmed)."""
+
+NOT_VOTE_PASSED = "planning issue does not carry vote-passed label"
+TRUSTED_HARDWARE = (
+ "automated release signing requires every artefact to be rebuilt bit-by-bit identical on trusted hardware "
+ "before publication ([Infra § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing)); "
+ "run `release-verify-rc {tag} --trusted-hardware --post-to {planning_issue}` on your own machine first"
+)
+
+
+class RenderError(ValueError):
+ """The command set cannot be rendered as configured; the skill surfaces this and stops."""
+
+
+# --------------------------------------------------------------------------- Step 0
+
+
+def _short_target(url: str) -> str:
+ """`https://dist.apache.org/repos/dist/release/foo/1.0/` → `dist/release/foo/1.0/`."""
+ index = url.find("dist/release/")
+ return url[index:] if index >= 0 else url
+
+
+def verdict(
+ preflight: dict[str, Any],
+ *,
+ vote_passed: bool,
+ target_populated: bool,
+ attestation: bool = False,
+ planning_issue_found: bool = True,
+ planning_issue: str = "<planning-issue>",
+) -> dict[str, Any]:
+ """Step 0: the tool's blockers plus the checks it cannot see, folded into one verdict.
+
+ Every fact defaults to the blocking side: no `--vote-passed` means the label
+ was not seen, and there is no override for it (Golden rule 3).
+ """
+ values = preflight.get("values") or {}
+ version, rc = values.get("version"), values.get("rc")
+ tag = f"{version}-{rc}" if version and rc else "<version>-rc<N>"
+ blockers = [str(b) for b in preflight.get("blockers") or []]
+ if not planning_issue_found:
+ blockers.append(f"no planning issue found for {version or '<version>'}: pass --planning-issue <url>")
+ elif not vote_passed:
+ blockers.append(NOT_VOTE_PASSED)
+ if target_populated:
+ blockers.append(
+ f"target URL {_short_target(str(values.get('target_url') or '<target_url>'))} already contains content"
+ )
+ if values.get("trusted_hardware_attestation_required") and not attestation:
+ blockers.append(TRUSTED_HARDWARE.format(tag=tag, planning_issue=planning_issue))
+ non_asf = bool(values.get("non_asf"))
+ rm_is_pmc = True if non_asf else values.get("rm_is_pmc") is True
+ if blockers:
+ outcome = "blocked"
+ elif not rm_is_pmc:
+ outcome = "handoff-non-pmc"
+ else:
+ outcome = "proceed"
+ read = []
+ if target_populated:
+ read.append("blockers/target-exists.md")
+ if any(b.startswith("automated release signing requires") for b in blockers):
+ read.append("blockers/trusted-hardware.md")
+ if outcome == "handoff-non-pmc":
+ read.append("outcomes/handoff-non-pmc.md")
+ return {
+ "verdict": outcome,
+ "blockers": blockers,
+ "rm_is_pmc": rm_is_pmc,
+ "non_asf": non_asf,
+ "version": version,
+ "rc": rc,
+ "dist_backend": values.get("dist_backend"),
+ "read": read,
+ }
+
+
+# --------------------------------------------------------------------------- Step 2
+
+
+def apache_name(product: str) -> str:
+ """The `Apache <product-name>` the svn message and tag message spell, without doubling `Apache`."""
+ return product if product.startswith("Apache ") or product.startswith("<") else f"Apache {product}"
+
+
+@dataclass
+class Promote:
+ """The resolved inputs the Step 2 command set is rendered from."""
+
+ version: str
+ rc: str
+ meta: dict[str, Any]
+ upstream: str = "<upstream>"
+ project: str = "<project>"
+ product_name: str = "<product-name>"
+ warnings: list[str] = field(default_factory=list)
+
+ @classmethod
+ def from_load(
+ cls,
+ loaded: dict[str, Any],
+ *,
+ upstream: str | None = None,
+ project: str | None = None,
+ product_name: str | None = None,
+ ) -> Promote:
+ meta = loaded.get("metadata") or {}
+ version, rc = meta.get("version"), meta.get("rc")
+ if not version or not rc:
+ raise RenderError(
+ "release-config load output has no version / rc: run it with --skill promote <version>-rc<N>"
+ )
+ return cls(
+ version=str(version),
+ rc=str(rc),
+ meta=meta,
+ upstream=upstream or "<upstream>",
+ project=project or "<project>",
+ product_name=product_name or "<product-name>",
+ )
+
+ @property
+ def tag(self) -> str:
+ return f"{self.version}-{self.rc}"
+
+ @property
+ def backend(self) -> str:
+ return str(self.meta.get("dist_backend") or "")
+
+ def value(self, given: str | None, placeholder: str, what: str) -> str:
+ """A fact the agent supplies; missing → the placeholder, and a warning naming it."""
+ if given:
+ return given
+ self.warnings.append(f"{what} is unknown: the block carries {placeholder} for the RM to fill in")
+ return placeholder
+
+ def render(self, text: str) -> str:
+ out = text
+ for spelling in ("<version>-<rcN>", "<version>-rc<N>", "<version>-rcN"):
+ out = out.replace(spelling, self.tag)
+ return out.replace("<rcN>", self.rc).replace("rc<N>", self.rc).replace("<version>", self.version)
+
+
+def s3_bucket(template: str) -> str:
+ """The bucket name in `release_dist_url_template`: `s3://<bucket>/…` or an `*.s3.*amazonaws.com` URL."""
+ parsed = urlparse(template)
+ host = parsed.netloc
+ bucket = ""
+ if parsed.scheme == "s3":
+ bucket = host
+ elif host.endswith("amazonaws.com"):
Review Comment:
## CodeQL / Incomplete URL substring sanitization
The string [amazonaws.com](1) may be at an arbitrary position in the sanitized URL.
[Show more details](https://github.com/apache/magpie/security/code-scanning/121)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@magpie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] perf(release-promote, release-announce-draft): render the promotion and announcement as code [magpie]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #1618:
URL: https://github.com/apache/magpie/pull/1618
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@magpie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org