You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "justinmclean (via GitHub)" <gi...@apache.org> on 2026/05/25 02:30:11 UTC

[PR] feat(pairing): add multi-agent review pipeline skill and eval suite [airflow-steward]

justinmclean opened a new pull request, #269:
URL: https://github.com/apache/airflow-steward/pull/269

   Implements work item 5 from the spec-loop plan: a new pairing-multi-agent-review skill that fans a local diff through three independent, axis-isolated review passes (correctness, security, conventions) and merges their findings into one structured report.
   
   Key design points:
   - Each sub-agent receives only its own axis scope to prevent one axis from anchoring or suppressing findings on another.
   - Sub-agents run in parallel (single Agent tool call message).
   - Deduplication annotates cross-axis findings with also_flagged_by rather than silently dropping them.
   - Injection-guard callout (Pattern 4) is present; injection attempts detected in diff content are flagged as blocking findings in the Security section.
   - Report format is identical to pairing-self-review for a consistent developer experience across the Pairing skill family.
   
   Includes a 15-case eval suite across 6 step-suites covering diff collection, per-axis sub-agent passes, merge/deduplication, and report composition — including an adversarial injection-resistance case per axis.
   
   Updates docs/modes.md to mark Pairing as experimental with 1 skill.
   
   Generated-by: Claude (Opus 4.7)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(pairing): add multi-agent review pipeline skill and eval suite [airflow-steward]

Posted by "justinmclean (via GitHub)" <gi...@apache.org>.
justinmclean commented on PR #269:
URL: https://github.com/apache/airflow-steward/pull/269#issuecomment-4539857712

   Pre-flight self-review — PR #269 (pairing-multi-agent-review)
     
     https://github.com/apache/airflow-steward/pull/269 · draft · author: 
     justinmclean
   
     Base: main · Files changed: 52 (all added) · Diff size: +1038 / −3
   
     A new Pairing-mode skill that fans the diff through three independent axis
     passes (correctness, security, conventions) and merges findings. SKILL.md
     (~400 lines) + a 6-step eval suite (step-1-collect-diff, step-2a/b/c axis
     passes, step-3-merge-findings, step-4-compose-report) + docs/modes.md table
     update.
   
     Correctness
   
     No findings. Eval-spec ↔ expected.json keys match exactly across all 6 step
     suites (step-1:
     resolved_base/files_changed/lines_added/lines_removed/diff_empty/stop_reason;
     step-2a/b/c: axis/findings/injection_attempts; step-3:
     merged_findings/blocking_count/advisory_count/aggregated_injection_attempts;
     step-4:
     sections_present/overall_signal/blocking_count/advisory_count/footer_present).
      docs/modes.md table is internally consistent against current origin/main (no
     Pairing skill exists there yet, so "Pairing: 1 skill" is accurate at merge
     time).
   
     Security
   
     No findings. Standard injection-guard callout present in SKILL.md. The
     three-axis fan-out design itself is a security feature (one axis can't
     suppress another's findings). Adversarial coverage in the eval suite: step-2a 
     case-3-injection-blocked, step-3 case-3-injection-aggregation. Read-only skill
      — no posting, no shell, no subprocess.
   
     Conventions
   
     No findings. skill-validate --strict clean for this skill (the one violation
     in the run is the pre-existing security-tracker-stats-dashboard
     action-inventory, unrelated). markdownlint-cli2 reports 0 errors across all 31
      changed .md files. SPDX header in place; frontmatter well-formed; description
      comma count under the action-inventory threshold.
   
     Summary
   
     Ready — no blocking or advisory findings.
   
     Blocking: 0  Advisory: 0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(pairing): add multi-agent review pipeline skill and eval suite [airflow-steward]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #269:
URL: https://github.com/apache/airflow-steward/pull/269


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(pairing): add multi-agent review pipeline skill and eval suite [airflow-steward]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #269:
URL: https://github.com/apache/airflow-steward/pull/269#issuecomment-4602035198

   I will rebase/solve conflicts @justinmclean -> so that we can rename things - we can continue working on it later :) 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(pairing): add multi-agent review pipeline skill and eval suite [airflow-steward]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #269:
URL: https://github.com/apache/airflow-steward/pull/269#issuecomment-4558371657

   Hi @justinmclean — same situation as #228 / #229. The `pairing-multi-agent-review` skill isn't in `main` yet (so the content is still unique), but the branch is 357 files / ~12k-line behind current `main` and a maintainer-side rebase isn't mechanical at this staleness.
   
   Whenever you get a chance, a fresh rebase from your end would help.
   
   No rush.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org