You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@magpie.apache.org by "dpol1 (via GitHub)" <gi...@apache.org> on 2026/10/02 14:48:09 UTC
[I] fix(validator): per-file checks and check-placeholders.sh skip every skill since skills/ became symlinks [magpie]
dpol1 opened a new issue, #1490:
URL: https://github.com/apache/magpie/issues/1490
### What
Since #1208 every `skills/<name>` entry is a symlink into `plugins/magpie-<family>/skills/<alias>`. Two linters walk `skills/` without following those symlinks, so neither of them sees a single skill file:
| Linter | How it walks `skills/` | Files it finds there |
|---|---|---|
| `skill-and-tool-validator`, `collect_files_to_check()` (`tools/skill-and-tool-validator/src/skill_and_tool_validator/__init__.py:1744-1749`) | `base.rglob("*.md")` | 1 (`skills/.pytest_cache/README.md`) |
| `tools/dev/check-placeholders.sh`, `SCAN_PATHS=("skills" "tools")` (line 118), greps at lines 171 and 176 | `grep -r` | 0 |
`Path.rglob()` does not descend into symlinked directories, and `grep -r` only follows symlinks named on the command line, not the ones it meets while recursing. `grep -R` finds all 78 `SKILL.md`.
So everything `run_validation()` does inside its `for path in files:` loop is skipped for every skill: `validate_frontmatter`, `validate_name_convention`, `validate_injection_guard`, `validate_principle_compliance`, `validate_privacy_patterns`, `validate_trigger_preservation`, `validate_asf_coupling`, `validate_skill_line_limit`, and for all skill files `validate_links`, `validate_placeholders`, `validate_security_patterns`, `validate_gh_list_limit`, `validate_lowercase_f_field`, `validate_branch_name_confidentiality`. The repo-wide checks that go through `collect_skill_dirs()` (`iterdir()` + `is_dir()`, which does follow symlinks) still run.
### Why it matters
CI and the pre-commit hook stay green while the per-file checks look at no skill at all; only the repo-wide checks still see them. `test_real_repo_passes` (`tests/test_validator.py:879`) says it "exercises every SKILL.md, every supporting file, and every internal link"; it passes because the collector hands it no skill file. Both copies of the placeholder check are blind at the same time, so the hardcoded-value class #1139 fixed can come back without any linter noticing.
With a collector that follows the symlinks, the per-file loop goes from 1 file to 251 and finds 77 violations on top of the 23 (all SOFT) the suite reports today. Three are HARD:
- `skills/release-prepare/SKILL.md`: description + when_to_use is 1599 chars; Claude Code truncates past 1536.
- `skills/security-tracker-stats-dashboard/SKILL.md`: missing injection-guard callout (Pattern 4), although the skill reads `gh issue view`.
- `skills/pr-management-triage/backport-check.md:15`: anchor `#backports` not found in `plugins/magpie-setup/templates/pr-management-config.md`.
The other 74 are SOFT (39 trigger-preservation, 13 skill-line-limit, 11 principle-compliance, 8 ASF-coupling, 2 gh-list-limit, 1 privacy).
### Suggested fix
1. Make `collect_files_to_check()` follow the `skills/<name>` symlinks: iterate the skill directories, `rglob("*.md")` inside each resolved one, and de-duplicate on the resolved path. (`rglob(..., recurse_symlinks=True)` would be shorter, but it needs Python 3.13 and the tool declares `>=3.11`.)
2. Switch the two greps in `check-placeholders.sh` from `-r` to `-R`. The reported paths stay `skills/<name>/...`, so the existing `ALLOWLIST_PATHS` entries keep matching.
3. Fix the three HARD findings above in the same PR, otherwise `test_real_repo_passes` goes red as soon as the collector works.
4. Add a test whose fixture lays skills out the way the repo does, real directories under `plugins/*/skills/` and symlinks under `skills/`, so the collector cannot drop to zero files again without a failing test.
### Found via
An architecture pass over the validator: `collect_files_to_check()` returned one file on a clean checkout of `main` (`37670575`).
### Related
- #1208 introduced the symlink layout.
- #1139 / #1144: the hardcoded-value class the placeholder check exists to catch.
- `PRE_PR_REVIEW_DELEGATED` (`__init__.py:3752-3754`) is keyed `security-model-prepare`, but `validate_pre_pr_review_block()` compares it with resolved directory names (`model-prepare`), so the entry never matches. No effect today, because `model-prepare` carries the block, but the delegation rule is dead. Same root cause: a skill has a flat name and an alias name, and the validator mixes the two.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@magpie.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] fix(validator): per-file checks and check-placeholders.sh skip every skill since skills/ became symlinks [magpie]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #1490: fix(validator): per-file checks and check-placeholders.sh skip every skill since skills/ became symlinks
URL: https://github.com/apache/magpie/issues/1490
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@magpie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org