You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@magpie.apache.org by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/10/05 11:51:17 UTC
[PR] fix(validator): check skill files reached through skills/ symlinks [magpie]
dpol1 opened a new pull request, #1522:
URL: https://github.com/apache/magpie/pull/1522
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
## Summary
- Every `skills/<name>` entry is a symlink into `plugins/magpie-<family>/skills/<alias>`.
`collect_files_to_check()` used `Path.rglob()`, which does not descend into symlinked directories before Python 3.13, so it returned only `skills/.pytest_cache/README.md` and every per-file check in `run_validation()` skipped every skill.
It now walks `skills/` with `glob`'s `**`, which follows the symlinks and skips dot-entries.
Paths stay under `skills/<name>/` and each real file is returned once.
- `tools/dev/check-placeholders.sh` had the same gap (`grep -r` skips symlinks met while recursing) and now scans with `grep -R`.
It reaches 136 skill `.md` files (0 before) and finds no hardcoded references.
- Checking the skills again surfaced two HARD violations, fixed here:
`pr-triage/backport-check.md` linked an inline `<a id="backports">` anchor the validator does not recognise, and now targets the `## Workflow choices` section that holds the `backport_branches` row (the anchor had no other reference and is removed);
`security-tracker-stats-dashboard/SKILL.md` reads tracker issue titles and bodies and now carries the injection-guard callout.
- Second commit: `validate_pre_pr_review_block()` iterated resolved plugin directories (`model-prepare`), so the `PRE_PR_REVIEW_DELEGATED` entry keyed `security-model-prepare` never matched.
The check now iterates the `skills/<name>` entries.
## Type of change
- [x] Skill change (`.claude/skills/<name>/`) — eval fixtures updated below
- [ ] Tool / bridge contract (`tools/<system>/*.md`)
- [x] Python package (`tools/*/` with `pyproject.toml`)
- [ ] Groovy reference impl
- [ ] Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
- [ ] Documentation (`docs/`, `README.md`, `CONTRIBUTING.md`)
- [x] Project template (`projects/_template/`)
- [x] CI / dev loop (`prek`, workflows, validators)
- [ ] Other:
## Test plan
- [x] `prek run --all-files` passes
- [x] For Python packages touched: `uv run pytest` / `ruff check` / `mypy` passes
- `tools/skill-and-tool-validator`: 487 passed, including `test_real_repo_passes` (0 HARD on the real tree); `ruff check`, `ruff format --check` and `mypy` clean.
- `tools/dev`: 224 passed; `ruff check`, `ruff format --check` and `mypy` clean.
- [ ] For Groovy bridges touched: command-line invocation tested end-to-end
- [ ] For skill changes: eval suite passes for the affected skill
(`PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner tools/skill-evals/evals/<skill>/`)
- [ ] For skill *behaviour* changes: a new or updated eval fixture is included in this PR
(a regression test for the bug fixed / the behaviour added — see CONTRIBUTING.md)
- [x] Other:
- New tests, each run red before the fix:
`TestCollectFilesToCheck::test_follows_symlinked_skill_dirs` (symlinked skill, twin symlink de-duplicated, `.pytest_cache` skipped),
`TestRunValidation::test_checks_files_of_symlinked_skills` (a per-file violation inside a symlinked skill is reported),
`TestRunValidation::test_delegated_pr_opener_is_matched_by_its_skills_name` (second commit),
and `tools/dev/tests/test_check_placeholders.py` (runs the script as a CLI against a tmp tree; expects exit 1 for a forbidden string in a symlinked skill).
- `skill-and-tool-validate` on the real tree with `SKILL_VALIDATOR_BASE_REF=main`: 0 HARD.
SOFT warnings go from 23 to 58; the 35 newly surfaced advisories are left for follow-up.
- `measured_tokens` restamped with `skill-token-count --write`; `skill-surface-hash.py` reports all 78 skills in sync.
- `tools/skill-evals/evals/security-tracker-stats-dashboard/` exists; it was not run (the added callout is prose).
## RFC-AI-0004 compliance
- [x] **Vendor neutrality** — placeholders (`<PROJECT>`, `<tracker>`, `<upstream>`, `<security-list>`) used in all skill / tool prose (the `check-placeholders` prek hook is the mechanical gate)
## Linked issues
Closes #1490
## Notes for reviewers (optional)
- `collect_files_to_check()` returns unresolved `skills/<name>/…` paths on purpose: the placeholder allowlist and other checks match on the `skills/` prefix.
- The `#backports` link could also be fixed by teaching `extract_headings()` about inline `<a id>` anchors (lychee already accepts them).
This PR repoints the link instead and leaves the validator's anchor rules unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@magpie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(validator): check skill files reached through skills/ symlinks [magpie]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #1522:
URL: https://github.com/apache/magpie/pull/1522
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@magpie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org