You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@magpie.apache.org by po...@apache.org on 2026/10/10 21:27:03 UTC
(magpie) branch main updated: feat(contributor-growth): recent-activity check and stricter pre-filter options for candidate-screen (#1592)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/magpie.git
The following commit(s) were added to refs/heads/main by this push:
new 85156a3e feat(contributor-growth): recent-activity check and stricter pre-filter options for candidate-screen (#1592)
85156a3e is described below
commit 85156a3e34fc44acbcb585dee17aaba489976e7c
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Sat Oct 10 22:26:58 2026 +0100
feat(contributor-growth): recent-activity check and stricter pre-filter options for candidate-screen (#1592)
The pre-filter kept anyone whose merged PRs or reviews reached half of
a floor over the whole window. On a large project that keeps people who
stopped contributing months ago and far more people than anyone reads,
and there was no way to tighten it beyond the ratio.
Three settings in contributor-nomination-config.md control it now:
screen_recent_activity_months (default 4) drops anyone with no change
landed in the last N months, for both targets; screen_prefilter_require
(any, the previous behaviour, or all) decides whether one count is
enough; screen_prefilter_targets (committer, the previous behaviour, or
both) extends the count check to the governance-body pool. Every drop is
still logged with its counts and the check that dropped it.
Generated-by: Claude Opus 5
---
.../skills/candidate-screen/SKILL.md | 23 ++++++++++++++--------
.../templates/contributor-nomination-config.md | 3 +++
tools/skill-evals/README.md | 2 +-
.../evals/contributor-candidate-screen/README.md | 2 +-
.../fixtures/case-1-dropped-logged/report.md | 1 +
.../fixtures/case-2-zero-floor-ignored/report.md | 1 +
.../case-3-pmc-pool-not-prefiltered/report.md | 1 +
.../case-4-recent-activity-and-all/expected.json | 23 ++++++++++++++++++++++
.../case-4-recent-activity-and-all/report.md | 18 +++++++++++++++++
tools/spec-loop/specs/contributor-growth.md | 5 ++++-
10 files changed, 68 insertions(+), 11 deletions(-)
diff --git a/plugins/magpie-contributor-growth/skills/candidate-screen/SKILL.md b/plugins/magpie-contributor-growth/skills/candidate-screen/SKILL.md
index b63e1cda..e7844b4d 100644
--- a/plugins/magpie-contributor-growth/skills/candidate-screen/SKILL.md
+++ b/plugins/magpie-contributor-growth/skills/candidate-screen/SKILL.md
@@ -23,7 +23,7 @@ argument-hint: "[target:committer|pmc|both] [window:6m] [end:YYYY-MM-DD]"
capability: capability:stats
surface_hash: sha256:a1767d466ee963d1
license: Apache-2.0
-measured_tokens: 3723
+measured_tokens: 3994
---
<!-- SPDX-License-Identifier: Apache-2.0
@@ -130,7 +130,7 @@ Local modifications go in the override file; framework changes go via PR to `apa
| `window:Nm` | the configured window, else `6m` | Activity window |
| `end:YYYY-MM-DD` | today | Last day of the window |
-From `<project-config>/contributor-nomination-config.md`: `report_repo` (required), `report_path` (default `reports/`), `screen_prefilter_ratio` (default `0.5`).
+From `<project-config>/contributor-nomination-config.md`: `report_repo` (required), `report_path` (default `reports/`), `screen_prefilter_ratio` (default `0.5`), `screen_prefilter_require` (`any` or `all`, default `any`), `screen_prefilter_targets` (`committer` or `both`, default `committer`), `screen_recent_activity_months` (default `4`; `0` turns the check off).
A `shortlist_max_missing` left over from an earlier version is ignored; say so once at the start of the run.
Floors come from `<project-config>/committer-readiness.md`, else `contributor-nomination-config.md`, resolved as `contributor-to-committer` Step 1 does.
Both files are personal configuration, read from the personal layer first and from `.apache-magpie-overrides/` only as a fallback; [they belong in the personal layer](../../../../docs/contributor-growth/README.md#why-the-configuration-is-personal).
@@ -172,13 +172,20 @@ If even a one-day slice exceeds the cap, stop and say so rather than build a par
## Step 2 — Pre-filter
-**`<governance-body>` target:** no pre-filter — the pool is the current committers who are not members, small enough to measure in full.
+The pre-filter only drops the long tail; it is the one place the floors are used.
-**Committer target:** for each person in the pool, run two count-only queries — changes landed (`list_authored(person, state: landed, count_only)`) and changes reviewed (`list_reviews_given(person, count_only)`), in the window — reading `total` only.
-Keep the person when either count is at least `screen_prefilter_ratio` × its floor.
-A floor of `0` (an evidence-only metric) is ignored here; it never keeps anyone by itself.
-Only these two counts are cheap enough to pre-filter a large pool; list, triage and community activity are measured in Step 3 for everyone who stays.
-Log everyone dropped, with both counts, in `dropped`; nobody leaves the pool silently.
+**Recent activity (both targets).**
+When `screen_recent_activity_months` is above `0`, run one count-only query per person — changes landed in the last that many months of the window (`list_authored(person, state: landed, since: <end> − N months, count_only)`) — and drop anyone whose `total` is `0`.
+Someone who stopped contributing months ago has nothing current for the `<governance-body>` to look at, however busy the start of the window was.
+
+**Counts.**
+For each person still in the pool, run two count-only queries — changes landed (`list_authored(person, state: landed, count_only)`) and changes reviewed (`list_reviews_given(person, count_only)`), in the window — reading `total` only, and compare each with `screen_prefilter_ratio` × its floor for the person's target.
+With `screen_prefilter_require: any`, keep the person when either count reaches its share; with `all`, only when both do.
+A floor of `0` (an evidence-only metric) is ignored here: it never keeps anyone by itself, and with `all` it is not required.
+The count check applies to the committer target, and to the `<governance-body>` target too when `screen_prefilter_targets` is `both`; otherwise the `<governance-body>` pool — the current committers who are not members — is small enough to measure in full and only the recent-activity check applies to it.
+
+Only these counts are cheap enough to pre-filter a large pool; list, triage and community activity are measured in Step 3 for everyone who stays.
+Log everyone dropped, with both counts, the landed count for the recent months when it was checked, and which check dropped them, in `dropped`; nobody leaves the pool silently.
---
diff --git a/plugins/magpie-setup/templates/contributor-nomination-config.md b/plugins/magpie-setup/templates/contributor-nomination-config.md
index c19e386c..c83f721c 100644
--- a/plugins/magpie-setup/templates/contributor-nomination-config.md
+++ b/plugins/magpie-setup/templates/contributor-nomination-config.md
@@ -57,6 +57,9 @@ and there are no meaningful framework defaults.
| `report_repo` | TODO: `owner/name` of a **private** repository | Where `candidate-screen` commits its report. The skill refuses unless the GitHub API reports the repository as private; restrict it to `<governance-body>` members. |
| `report_path` | `reports/` | Directory inside `report_repo` for the reports. |
| `screen_prefilter_ratio` | `0.5` | `candidate-screen` keeps a contributor for full measurement when merged PRs or reviews reach this share of the floor. |
+| `screen_prefilter_require` | `any` | `any`: one of merged PRs and reviews reaching its share of the floor keeps a contributor; `all`: both must. |
+| `screen_prefilter_targets` | `committer` | `committer`: the count pre-filter applies to the committer pool only; `both`: to the <governance-body> pool as well. |
+| `screen_recent_activity_months` | `4` | Drop anyone with no change landed in the last N months of the window, for both targets; `0` turns the check off. |
---
diff --git a/tools/skill-evals/README.md b/tools/skill-evals/README.md
index 29f71a14..9fbc76bc 100644
--- a/tools/skill-evals/README.md
+++ b/tools/skill-evals/README.md
@@ -91,7 +91,7 @@ Suites are currently implemented for:
- **setup-privacy-llm** — 6 cases across 2 suites (step-1-resolve, step-4-gate)
- **preflight-reconciliation** — 7 cases across 1 suite (step-reconciliation)
- **contributor-calibrate** — 11 cases across 3 suites (step-1-find-nominations, step-4-propose-floors, step-6-write-configuration)
-- **contributor-candidate-screen** — 14 cases across 6 suites (step-0-gates, step-2-pre-filter, step-3-measure, step-4-write-report, step-5-deliver)
+- **contributor-candidate-screen** — 15 cases across 6 suites (step-0-gates, step-2-pre-filter, step-3-measure, step-4-write-report, step-5-deliver)
- **pr-management-stack-review** — 11 cases across 2 suites (step-3-structural-findings, step-4-reading-plan); the gate, mechanical findings, verdict and posting are pytest-tested in `tools/pr-management`
## Prerequisites
diff --git a/tools/skill-evals/evals/contributor-candidate-screen/README.md b/tools/skill-evals/evals/contributor-candidate-screen/README.md
index 12101fd4..f1c9a7df 100644
--- a/tools/skill-evals/evals/contributor-candidate-screen/README.md
+++ b/tools/skill-evals/evals/contributor-candidate-screen/README.md
@@ -11,7 +11,7 @@ skill. Tests six decision points:
|---|---|---|
| `step-0-gates` | A public report repository is refused with no alternative offered; a private one proceeds after the audience is confirmed | 2 |
| `step-1-pool` | Roster ids mapped to GitHub handles (unmapped ids listed, current committers excluded); a merged-PR search over 1000 results sliced by month, never truncated | 2 |
-| `step-2-pre-filter` | Pre-filter ratio applied; a zero (evidence-only) floor ignored; the PMC pool not pre-filtered; every dropped contributor logged with counts | 3 |
+| `step-2-pre-filter` | Pre-filter ratio applied; a zero (evidence-only) floor ignored; the PMC pool not count-filtered by default; no recent landed change drops a contributor; `require: all` and `targets: both`; every dropped contributor logged with counts | 4 |
| `step-3-measure` | Everyone who passed the pre-filter goes into the report, alphabetically, with no comparison against the floors | 1 |
| `step-4-write-report` | Surfacing note at the top; verified real name; no `@`-mentions anywhere in the report; capped counts shown as minimums; people listed alphabetically, never by activity, with no floors-met count or threshold column | 3 |
| `step-5-deliver` | No write without confirmation; privacy re-checked before the write; nothing posted elsewhere | 3 |
diff --git a/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-1-dropped-logged/report.md b/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-1-dropped-logged/report.md
index 20021aa7..a24a74ff 100644
--- a/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-1-dropped-logged/report.md
+++ b/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-1-dropped-logged/report.md
@@ -2,6 +2,7 @@
https://www.apache.org/licenses/LICENSE-2.0 -->
screen_prefilter_ratio: 0.5
+screen_recent_activity_months: 0
Floors (committer): prs_merged 40, reviews_total 20.
Count-only results for the window:
diff --git a/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-2-zero-floor-ignored/report.md b/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-2-zero-floor-ignored/report.md
index 08a4f385..d597a381 100644
--- a/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-2-zero-floor-ignored/report.md
+++ b/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-2-zero-floor-ignored/report.md
@@ -3,6 +3,7 @@
Target: committer.
screen_prefilter_ratio: 0.5
+screen_recent_activity_months: 0
Floors (committer): prs_merged 40, reviews_total 0 (evidence only).
Count-only results for the window:
diff --git a/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-3-pmc-pool-not-prefiltered/report.md b/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-3-pmc-pool-not-prefiltered/report.md
index 100a3a7d..531e20bc 100644
--- a/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-3-pmc-pool-not-prefiltered/report.md
+++ b/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-3-pmc-pool-not-prefiltered/report.md
@@ -3,6 +3,7 @@
Target: pmc (the pool is the 14 current committers who are not PMC members).
screen_prefilter_ratio: 0.5
+screen_recent_activity_months: 0
Floors (pmc): prs_merged 40, reviews_total 100.
Count-only results for the window:
diff --git a/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-4-recent-activity-and-all/expected.json b/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-4-recent-activity-and-all/expected.json
new file mode 100644
index 00000000..57ac7000
--- /dev/null
+++ b/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-4-recent-activity-and-all/expected.json
@@ -0,0 +1,23 @@
+{
+ "kept": [
+ "india",
+ "lima"
+ ],
+ "dropped": [
+ {
+ "handle": "juliet",
+ "merged_prs": 60,
+ "reviewed_prs": 40
+ },
+ {
+ "handle": "kilo",
+ "merged_prs": 45,
+ "reviewed_prs": 10
+ },
+ {
+ "handle": "mike",
+ "merged_prs": 33,
+ "reviewed_prs": 175
+ }
+ ]
+}
diff --git a/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-4-recent-activity-and-all/report.md b/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-4-recent-activity-and-all/report.md
new file mode 100644
index 00000000..83e4f748
--- /dev/null
+++ b/tools/skill-evals/evals/contributor-candidate-screen/step-2-pre-filter/fixtures/case-4-recent-activity-and-all/report.md
@@ -0,0 +1,18 @@
+<!-- SPDX-License-Identifier: Apache-2.0
+ https://www.apache.org/licenses/LICENSE-2.0 -->
+
+Target: both. Window 2026-04-10 → 2026-10-10.
+screen_prefilter_ratio: 1.0
+screen_prefilter_require: all
+screen_prefilter_targets: both
+screen_recent_activity_months: 4
+Floors (committer): prs_merged 40, reviews_total 20. Floors (pmc): prs_merged 40, reviews_total 100.
+
+Count-only results for the window (committer pool):
+- india: merged PRs 50, reviewed PRs 30, landed since 2026-06-10: 12
+- juliet: merged PRs 60, reviewed PRs 40, landed since 2026-06-10: 0
+- kilo: merged PRs 45, reviewed PRs 10, landed since 2026-06-10: 9
+
+Count-only results for the window (pmc pool — committers not on the PMC):
+- lima: merged PRs 80, reviewed PRs 220, landed since 2026-06-10: 20
+- mike: merged PRs 33, reviewed PRs 175, landed since 2026-06-10: 5
diff --git a/tools/spec-loop/specs/contributor-growth.md b/tools/spec-loop/specs/contributor-growth.md
index c1d98cfe..fdbd8644 100644
--- a/tools/spec-loop/specs/contributor-growth.md
+++ b/tools/spec-loop/specs/contributor-growth.md
@@ -111,7 +111,10 @@ Adopter config scaffolds live in `plugins/magpie-setup/templates/`
`tools/skill-evals/evals/contributor-calibrate/`.
- Skill: `contributor-candidate-screen` — cuts the long tail of recent
contributors with a deterministic pre-filter against the floors (every
- drop logged; the only place the floors are used), measures everyone
+ drop logged; the only place the floors are used; configurable as `any`
+ or `all` counts, for the committer pool or both pools, plus a
+ recent-activity check that drops anyone with nothing landed in the
+ last `screen_recent_activity_months`), measures everyone
who remains — deliberately more than the PMC would consider,
alphabetically by handle, never scored, ranked, shortlisted or judged —
and writes a report (the list linked to details, plus a