You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@magpie.apache.org by "justinmclean (via GitHub)" <gi...@apache.org> on 2026/07/05 08:11:05 UTC
[PR] feat(contributor-sentiment): add methodology doc, skill, and eval suite [magpie]
justinmclean opened a new pull request, #744:
URL: https://github.com/apache/magpie/pull/744
## Summary
Implements MISSION v1 Initial Goal: settle on a contributor-sentiment evaluation methodology covering both ASF and non-ASF cohorts.
- docs/contributor-sentiment.md — four signal dimensions (thread tone, time-to-first-reply, first-PR retention, reviewer load), data-gathering approach (GitHub API, no new telemetry per PRINCIPLE 10), ASF/non-ASF cohort handling, and the conjunctive promotion rule for experimental→stable advancement.
- skills/contributor-sentiment/SKILL.md — read-only skill (Steps 0–3) that queries GitHub, scores each signal against its threshold, and produces a structured JSON gate report.
- tools/skill-evals/evals/contributor-sentiment/ — 10-case eval suite across 3 steps: input resolution, signal scoring (healthy/failing/ tone-only/injection cases), and report generation (pass/fail/no-baseline).
- Cross-links RFC-AI-0004 (gate clause for Agentic Autonomous) and docs/pilot-report-template.md to the new methodology doc.
- docs/modes.md and docs/labels-and-capabilities.md updated for the new skill; skill-and-tool-validator passes with no violations.
Generated-by: Claude (claude-sonnet-4-6)
## Type of change
- [X] Skill change (`.claude/skills/<name>/`) — eval fixtures updated below
- [ ] Tool / bridge contract (`tools/<system>/*.md`)
- [ ] Python package (`tools/*/` with `pyproject.toml`)
- [ ] Groovy reference impl
- [ ] Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
- [ ] Documentation (`docs/`, `README.md`, `CONTRIBUTING.md`)
- [ ] Project template (`projects/_template/`)
- [ ] CI / dev loop (`prek`, workflows, validators)
- [ ] Other:
## Test plan
- [X] `prek run --all-files` passes
- [ ] For Python packages touched: `uv run pytest` / `ruff check` / `mypy` passes
- [ ] For Groovy bridges touched: command-line invocation tested end-to-end
- [X] For skill changes: eval suite passes for the affected skill
(`PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner tools/skill-evals/evals/<skill>/`)
- [ ] For skill *behaviour* changes: a new or updated eval fixture is included in this PR
(a regression test for the bug fixed / the behaviour added — see CONTRIBUTING.md)
- [X] Other: validated skill
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@magpie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(contributor-sentiment): add methodology doc, skill, and eval suite [magpie]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #744:
URL: https://github.com/apache/magpie/pull/744
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@magpie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org