[PR] perf(security): fetch once per run instead of once per item [magpie]

["potiuk (via GitHub)" <gi...@apache.org>]

potiuk opened a new pull request, #1443:
URL: https://github.com/apache/magpie/pull/1443

   Seventh PR in the **security** family stack (#1344): the optimize-skill *fetch-upfront* pass.
   
   ## Summary
   
   The security skills re-ran the same reads per candidate, per tracker or per finding. They now fetch each shared set once per run and reuse it. The set of items processed and the per-item decisions are unchanged. Every batched list keeps an explicit `--limit` and says what to do when the cap is hit.
   
   | skill | before → after |
   |---|---|
   | issue-import | mail-thread reads per candidate 4 → 2. Step 7 keeps one fresh read, because a draft or reporter reply can land between scan and apply. Semantic-sweep list, collaborator list and rejections-ledger lookup once per run. Thread-id dedup: 6 ids per search. |
   | issue-import-from-md | labels set at creation: 2 calls per finding → 1 |
   | issue-import-from-pr | two duplicate-guard searches → one |
   | issue-import-from-scan | open-tracker list once for all findings |
   | issue-triage | per-tracker `gh issue view` → one GraphQL query per 20 trackers (body, comments, linked PRs) |
   | issue-invalidate `proposed` | reactions call + issue view per tracker → one GraphQL query per 20. Limit aligned with triage: 1000, stop when capped. |
   | issue-sync | linked PRs in one query; PR authors reused; hand-off comment 3 calls → 1; milestone bullet 4 calls → 0; advisory probe doubles as the fetch |
   | issue-fix | collaborator check: one list per run instead of one call per author |
   | issue-deduplicate / cve-allocate | redundant existence probes and title re-fetch removed |
   | model-verify | `AGENTS.md` + `SECURITY.md` for every repository in one query |
   | workflow-security-audit | zizmor over ≤ 50 repositories per run. A failed batch is re-run per repository, so a failure is never reported as clean. |
   | tracker-stats-dashboard | bodies and linked PRs from the list call: one `gh issue view` per issue → 0. Warns at the list cap. |
   
   **Deliberately kept per item:**
   - import Step 2b's closed-invalid search and from-md's duplicate guard. Both match keywords server-side in issue bodies; a local title-only match would miss duplicates.
   - triage's cross-repo PR search, because an OR'd search cannot attribute hits to a tracker.
   - freshness reads right before a draft is created.
   
   **Along the way:**
   - Several `gh` calls wrapped in `$(…)`, piped into `jq`, or redirected are now plain commands. Under the secure setup a wrapped `gh` stays sandboxed and fails.
   - import Step 2c asked `gh search prs` for a `mergedAt` field it does not have, and computed its date with GNU-only `date -d`.
   
   ## Test plan
   
   - [x] `prek run --all-files`, including lychee; `skill-and-tool-validate` (`--limit` rule); `check-placeholders`
   - [x] `tools/security-tracker-stats-dashboard`: 142 tests
   - [x] Every eval whose extracted section changed, run before (this PR's base) and after, outside the sandbox:
   
   | eval step | before | after |
   |---|---|---|
   | cve-allocate step-2 | 4/4 | 4/4 |
   | issue-deduplicate step-1 | 4/4 | 4/4 |
   | issue-fix step-3 | 5/5 | 5/5 |
   | issue-import step-0 / 2 / 4 / 7 | 3/3, 3/3, 5/5, 3/3 | 3/3, 3/3, 5/5, 3/3 |
   | issue-import step-2a | 6/6 | 6/6 |
   | issue-import step-2b | 2/4 | 2/4 (same two cases) |
   | issue-import step-3 | 9/9 | 9/9 |
   
   `step-2a/case-3-same-reporter` flipped to failing on the first after-run. Its output spec said a reporter-identity hit needs the *address* to match, while the skill matches on the local-part. The fixture's addresses share a local-part across different domains, so small wording changes flipped the verdict. The spec now states the skill's local-part rule, and the case passes 3/3.
   
   `step-3/case-2` flipped once and passed 3/3 on re-run: noise.
   
   Part of #1344.
   
   Generated-by: Claude Code (Opus 5.5)
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@magpie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] perf(security): fetch once per run instead of once per item [magpie]

["potiuk (via GitHub)" <gi...@apache.org>]

potiuk merged PR #1443:
URL: https://github.com/apache/magpie/pull/1443


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@magpie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org