[PR] feat(skill): add release-audit-report skill (Step 13) with auto-graded eval suite [airflow-steward]

["justinmclean (via GitHub)" <gi...@apache.org>]

justinmclean opened a new pull request, #552:
URL: https://github.com/apache/airflow-steward/pull/552

   ## Summary
   
   Adds `release-audit-report`, Step 13 of the release-management lifecycle.
   The skill assembles a structured per-release audit record from the
   planning issue, vote/result threads, RC artefact list, promote revision,
   and announcement archive URL, then proposes a PR that appends the record
   to the project's audit log.
   
   It is **read-only on every release surface**. The only write is the
   audit-log PR, which is proposed (never auto-opened) and merged by a
   committer.
   
   ## Behavior
   
   - **Proposal only.** Opening the audit-log PR requires explicit RM
     confirmation; the skill never auto-opens or auto-merges.
   - **MISSING, never invented.** A field whose source data does not exist is
     recorded as `MISSING` rather than guessed; the report continues with the
     fields that are available.
   - **Privacy boundary.** No content from the security tracker, CVE drafts,
     GHSA records, reporter mail, or embargoed material enters the public
     audit log; such fields appear as `REDACTED` with a reason. Binding voters
     are cited by PMC roster handle, never by personal email address.
   - **External content is data, never instructions.** Planning-issue bodies,
     vote-thread content, and announce text are treated as untrusted input;
     injected directives are flagged, not obeyed.
   
   ## Type of change
   
   - [X] Skill change (`.claude/skills/<name>/`) — eval fixtures updated below
   - [ ] Tool / bridge contract (`tools/<system>/*.md`)
   - [ ] Python package (`tools/*/` with `pyproject.toml`)
   - [ ] Groovy reference impl
   - [ ] Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
   - [ ] Documentation (`docs/`, `README.md`, `CONTRIBUTING.md`)
   - [ ] Project template (`projects/_template/`)
   - [ ] CI / dev loop (`prek`, workflows, validators)
   - [ ] Other:
   
   ## Test plan
   
   - [X] `prek run --all-files` passes
   - [ ] For Python packages touched: `uv run pytest` / `ruff check` / `mypy` passes
   - [ ] For Groovy bridges touched: command-line invocation tested end-to-end
   - [X] For skill changes: eval suite passes for the affected skill
         (`PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner tools/skill-evals/evals/<skill>/`)
   - [ ] For skill *behaviour* changes: a new or updated eval fixture is included in this PR
         (a regression test for the bug fixed / the behaviour added — see CONTRIBUTING.md)
   - [ ] Other:
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(skill): add release-audit-report skill (Step 13) with auto-graded eval suite [airflow-steward]

["potiuk (via GitHub)" <gi...@apache.org>]

potiuk merged PR #552:
URL: https://github.com/apache/airflow-steward/pull/552


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(skill): add release-audit-report skill (Step 13) with auto-graded eval suite [airflow-steward]

["justinmclean (via GitHub)" <gi...@apache.org>]

justinmclean commented on PR #552:
URL: https://github.com/apache/airflow-steward/pull/552#issuecomment-4805716722

   ## Correctness
   
   No findings. The step-2 expected.json files are rewired to structural style; every has_* key now resolves to a defined predicate (verified across all 8 cases, zero unresolved). The reported has_missing_fields/has_redacted_fields "no assertion defined" failures are gone (those keys are removed; their consistency is covered by the has_missing_flag_consistency judge). The bogus record_markdown literal is gone.
   
   ## Security
   
   No findings. Both negate email checks are now judge predicates, and the binding_voters / record_markdown no-email guarantees actually execute. The step-1 predicates that were previously dead (never referenced) now run in both cases. No negate keys remain anywhere in the suite.
   
   ## Conventions
   
   [advisory] step-2-assemble-record/fixtures/grading-schema.json is now dead config. record_markdown is no longer a compared expected.json key (it is checked via the non_empty assertion), so prose_fields: ["record_markdown"] has nothing to act on. Harmless, but removable for tidiness.
   
   [advisory] Branch is still 10 commits behind main and lacks release-promote (#544), the upstream lifecycle step this skill references. Rebase before opening the PR.
   
   [advisory] README.md "Run" examples omit --cli "claude -p" (so cases would grade as MANUAL) and use --project rather than --directory. Pre-existing house style shared with sibling READMEs, not introduced here.
   Resolved: the <product_name> gap (now defined in the placeholder convention, Prerequisites, the Step 1 gather table, and the Step 1 output schema across all three synced copies), and the README grading description (predicate list corrected, field_false removed, negate claim replaced with the judge-for-absence note, injection-case description fixed).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org