You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@magpie.apache.org by "shahar1 (via GitHub)" <gi...@apache.org> on 2026/06/30 06:48:06 UTC

[PR] feat(pr-management-code-review): detect undisclosed AI-authored PRs [magpie]

shahar1 opened a new pull request, #655:
URL: https://github.com/apache/magpie/pull/655

   ## Summary
   
   - `pr-management-code-review` now flags PRs that are clearly **AI-authored**
     (structured `## Summary` / `## Changes` / `## Test plan` with task-list
     checkboxes, an echoed `**Title:**` line, a dropped PR template) but **omit
     the generative-AI disclosure the project's own contribution guidelines
     require** — raised as a `minor` contribution-guidelines finding in the
     normal review.
   - The check is a **framework-level default that self-calibrates**: whether
     disclosure is required is *project policy*, so it reads the repository's own
     PR template / contributing docs and raises nothing where no such requirement
     exists. It is deliberately **not** a `slop-detection` signal and never gates
     a merge on its own.
   - Motivated by a real miss — an AI-authored docs PR with no disclosure passed a
     full review pass without the gap being surfaced. The existing Step 3 body
     checks only caught *empty* / *boilerplate* / *prompt-injection* bodies, not
     the inverse (a template **replaced** by AI-style prose).
   
   ## Type of change
   
   - [x] Skill change (`skills/pr-management-code-review/`) — eval fixtures added
   - [x] CI / dev loop (new eval suite)
   
   ## Test plan
   
   - [x] `markdownlint-cli2` clean on all changed/new markdown (criteria.md,
         review-flow.md, README, fixtures)
   - [x] New eval suite `step-3-ai-authorship-disclosure` (5 cases) is discovered
         and loads via the runner; verdicts are internally consistent
         (`finding == requires_disclosure ∧ ai_authored ∧ ¬disclosure_affirmed`)
         and mirror the `step-3-security-disclosure-scan` fixture shape
   - [x] Cross-file anchors between criteria.md and review-flow.md resolve both
         directions (lychee-style fragment check)
   - [ ] CI: `prek run --all-files`, lychee, eval matrix
   
   ---
   
   Generated-by: Claude Code (Opus 4.8)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@magpie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(pr-management-code-review): detect undisclosed AI-authored PRs [magpie]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #655:
URL: https://github.com/apache/magpie/pull/655


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@magpie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org