Re: [I] Optimize the release-management skill family [magpie]

["potiuk (via GitHub)" <gi...@apache.org>]

potiuk commented on issue #1345:
URL: https://github.com/apache/magpie/issues/1345#issuecomment-5991742253

   The release-management optimisation is complete.
   
   ## Savings
   
   Measured across all 10 release skills, from before the first PR to `main` with every pass merged:
   
   | | Before | Now | Change |
   |---|---|---|---|
   | `SKILL.md` bodies, read on every run (`measured_tokens`) | 78,241 | 65,114 | **−17%** |
   | A `release-prepare` run (`SKILL.md` + the sub-command it executes) | 14,285 | 6,600–9,300 | **−35–55%** |
   | Advertised descriptions, loaded in every session (`docs/setup/marketplace.md`) | ~1.4k | ~0.8k | **−43%** |
   
   The split and the description pass cut most of the cost. The wording pass saved little, because the extraction and the split had already made the skills lean.
   
   ## What was done
   
   - **Deterministic steps moved into code.** The steps the model used to perform by hand now run in code:
     - `tools/release-config` handles config loading and every skill's Step 0 checks (#1510);
     - `tools/release-verify` handles verify-rc's signature, checksum, binary, symlink and version checks and the verdict (#1511);
     - sibling scripts handle vote counting, the audit record, retention, the key check and prepare's version helpers (#1512).
     These save few tokens; their value is that the checks are deterministic and tested. The skills had disagreed on several rules, and each is now settled to one rule, as the maintainer decided:
     - RC numbering starts at `rc1`;
     - convenience artefacts have their own version scheme;
     - there is one approver-roster key;
     - `non_asf` is derived from `organization:`;
     - the automated-signing gate is the organisation manifest's;
     - an expired key blocks;
     - only a voter's latest vote counts, by thread order.
   - **Security review findings fixed:**
     - quoted paste recipes;
     - stale signatures fail;
     - local paths stay out of reports;
     - revoked keys are refused;
     - vote identities can't collide;
     - pre-releases never decide retention;
     - the audit record is escaped (#1511, #1512, #1514).
   - **Split.** Steps a run uses only conditionally load on demand (#1515).
   - **Wording pass** on all ten skills (#1517), with a check that no condition or prohibition was lost.
   - **Descriptions** for the five skills over budget (#1519).
   - **Behaviour fixes:**
     - PR and comment bodies go through a file (#1502);
     - the `[VOTE]` expedited reason keeps CVE IDs out until the advisory ships (#1509);
     - `release-rc-cut` makes its two GitHub calls through vetted operations (#1518).
   - **Tests.** The new tools carry unit tests, and a new pre-commit hook runs the skills' sibling-script tests. Eval cases were added for each settled rule.
   
   Still over the 500-line cap: `release-rc-cut`, `-verify-rc`, `-announce-draft`, `-vote-draft` and `-audit-report`. What remains in them is used on every run, so splitting further would save no tokens.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@magpie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org