You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@magpie.apache.org by po...@apache.org on 2026/06/28 12:11:50 UTC
(magpie) branch main updated: test(skill-evals): make the local-smoke tag model-neutral (qwen3.5:9b + llama3.1:8b) (#597)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/magpie.git
The following commit(s) were added to refs/heads/main by this push:
new d7505bbb test(skill-evals): make the local-smoke tag model-neutral (qwen3.5:9b + llama3.1:8b) (#597)
d7505bbb is described below
commit d7505bbbf8b8d32a59ca914dd8a999534bdc6a32
Author: Justin Mclean <ju...@classsoftware.com>
AuthorDate: Sun Jun 28 22:11:45 2026 +1000
test(skill-evals): make the local-smoke tag model-neutral (qwen3.5:9b + llama3.1:8b) (#597)
Rename the local smoke-signal tag llama -> local-smoke (and smoke stays)
so it names the purpose, not a model, and document two interchangeable
local targets: qwen3.5:9b (Apache-2.0, preferred for ASF licensing) and
llama3.1:8b (Llama Community License). Pure rename of the existing 61
tagged cases plus README; no change to which cases are tested.
---
tools/skill-evals/README.md | 44 ++++++++++++++++++----
.../fixtures/case-1-untriaged/case-meta.json | 2 +-
.../fixtures/case-2-triaged/case-meta.json | 2 +-
.../fixtures/case-3-in-progress/case-meta.json | 2 +-
.../fixtures/case-4-stale-candidate/case-meta.json | 2 +-
.../fixtures/case-5-skip-security/case-meta.json | 2 +-
.../case-6-prompt-injection/case-meta.json | 2 +-
.../fixtures/case-1-mixed-pool/case-meta.json | 2 +-
.../fixtures/case-2-stale-heavy/case-meta.json | 2 +-
.../fixtures/case-3-empty-pool/case-meta.json | 2 +-
.../fixtures/case-1-high-untriaged/case-meta.json | 2 +-
.../fixtures/case-2-high-stale/case-meta.json | 2 +-
.../fixtures/case-3-healthy/case-meta.json | 2 +-
.../fixtures/case-1-all-sections/case-meta.json | 2 +-
.../fixtures/case-2-no-area-labels/case-meta.json | 2 +-
.../fixtures/case-3-markdown-flag/case-meta.json | 2 +-
.../fixtures/case-1-clean-commit/case-meta.json | 2 +-
.../fixtures/case-2-all-fixed/case-meta.json | 2 +-
.../fixtures/case-2-no-baselines/case-meta.json | 2 +-
.../case-1-default-selector/case-meta.json | 2 +-
.../case-2-component-filter/case-meta.json | 2 +-
.../case-3-invalid-thresholds/case-meta.json | 2 +-
.../case-4-explicit-numbers/case-meta.json | 2 +-
.../fixtures/case-1-request-update/case-meta.json | 2 +-
.../case-2-close-stale-nudged/case-meta.json | 2 +-
.../case-3-close-stale-hard/case-meta.json | 2 +-
.../fixtures/case-4-skip-security/case-meta.json | 2 +-
.../case-5-skip-no-timestamps/case-meta.json | 2 +-
.../case-6-prompt-injection/case-meta.json | 2 +-
.../case-1-request-update-clean/case-meta.json | 2 +-
.../case-2-close-stale-clean/case-meta.json | 2 +-
.../fixtures/case-3-bare-issue-ref/case-meta.json | 2 +-
.../fixtures/case-1-post-all/case-meta.json | 2 +-
.../fixtures/case-2-skip-one/case-meta.json | 2 +-
.../fixtures/case-3-cancel/case-meta.json | 2 +-
.../fixtures/case-1-mixed-results/case-meta.json | 2 +-
.../case-2-all-request-update/case-meta.json | 2 +-
.../case-3-security-flagged/case-meta.json | 2 +-
.../fixtures/case-1-explicit-key/case-meta.json | 2 +-
.../fixtures/case-2-component/case-meta.json | 2 +-
.../fixtures/case-1-post-all/case-meta.json | 2 +-
.../fixtures/case-4-cancel/case-meta.json | 2 +-
.../fixtures/case-2-verbose-request/case-meta.json | 2 +-
.../fixtures/case-6-empty-diff/case-meta.json | 2 +-
.../fixtures/case-3-no-images/case-meta.json | 2 +-
.../fixtures/case-1-untriaged/case-meta.json | 2 +-
.../case-meta.json | 2 +-
.../case-4-triaged-responded-commit/case-meta.json | 2 +-
.../fixtures/case-5-stale-marker/case-meta.json | 2 +-
.../fixtures/case-1-clean-pass/case-meta.json | 2 +-
.../fixtures/case-1-apply-all/case-meta.json | 2 +-
.../fixtures/case-3-cancel/case-meta.json | 2 +-
.../fixtures/case-1-clean-pass/case-meta.json | 2 +-
.../fixtures/case-3-cancel/case-meta.json | 2 +-
.../fixtures/case-3-cancel/case-meta.json | 2 +-
.../fixtures/case-1-apply-all/case-meta.json | 2 +-
.../fixtures/case-3-cancel/case-meta.json | 2 +-
.../fixtures/case-4-cancel/case-meta.json | 2 +-
.../case-1-override-yaml-found/case-meta.json | 2 +-
.../case-2-no-yaml-fallback/case-meta.json | 2 +-
.../fixtures/case-3-quarterly-arg/case-meta.json | 2 +-
.../fixtures/case-4-output-path-arg/case-meta.json | 2 +-
62 files changed, 97 insertions(+), 69 deletions(-)
diff --git a/tools/skill-evals/README.md b/tools/skill-evals/README.md
index 86988218..6a39be31 100644
--- a/tools/skill-evals/README.md
+++ b/tools/skill-evals/README.md
@@ -81,10 +81,12 @@ PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner --cli "llm -m gpt
PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner --cli "claude -p" --verbose \
tools/skill-evals/evals/issue-triage/step-3-classify/fixtures/case-1-clear-bug
-# Run only cases tagged as useful smoke tests for local llama3.1:8b.
-PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner --tag llama \
- --cli "ollama run llama3.1:8b --nowordwrap --format json" \
+# Run only cases tagged as useful smoke tests for a local model.
+# Either supported local target works; swap the --cli model as you like.
+PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner --tag local-smoke \
+ --cli "ollama run qwen3.5:9b --nowordwrap --format json" \
tools/skill-evals/evals/
+# (or --cli "ollama run llama3.1:8b --nowordwrap --format json")
```
**JSON extraction** tries three strategies in order: parse the whole
@@ -193,14 +195,40 @@ Cases can opt into runner filters with a `case-meta.json` file next to
```json
{
- "tags": ["llama", "smoke"]
+ "tags": ["local-smoke", "smoke"]
}
```
Use `--tag <name>` to run only matching cases. Tags are intentionally
-conservative: for example, `llama` means the case is known to be useful
-as a local `llama3.1:8b` smoke signal, not that the whole suite is
-expected to pass on that model.
+conservative: for example, `local-smoke` means the case is known to be
+useful as a **local-model** smoke signal, not that the whole suite is
+expected to pass on a small local model. The tag is deliberately
+model-neutral: it names the purpose, not a specific model, so swapping
+the local target later is a one-line `--cli` change, not a repo-wide
+re-tag.
+
+#### Supported local models
+
+Two local targets are supported for the `local-smoke` set; either can run
+it, and you pick the `--cli` model per run:
+
+| Model | Ollama tag | License | Notes |
+|---|---|---|---|
+| Qwen3.5 9B | `qwen3.5:9b` | **Apache-2.0** | Recommended for ASF adopters: the license matches the framework's open / vendor-neutral posture. ~6.6 GB (Q4_K_M). |
+| Llama 3.1 8B | `llama3.1:8b` | Llama Community License (not an OSI/open license) | The historical reference target the current `local-smoke` set was first baselined against. ~4.7 GB. |
+
+`local-smoke` marks cases that pass on a local model. `qwen3.5:9b` and
+`llama3.1:8b` are interchangeable targets; pick one per run with `--cli`.
+Pin temperature for reproducibility, the runner sets none, so model
+defaults (llama ~0.8, qwen ~1.0) make single runs stochastic. Run the set
+against whichever you have installed:
+
+```bash
+PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner --tag local-smoke \
+ --cli "ollama run qwen3.5:9b --nowordwrap --format json" \
+ tools/skill-evals/evals/
+# or: --cli "ollama run llama3.1:8b --nowordwrap --format json"
+```
## Structure
@@ -217,7 +245,7 @@ evals/
case-N-<name>/
report.md # mock tool call outputs for this case
expected.json # ground-truth JSON the model should produce
- case-meta.json # optional runner tags, e.g. {"tags":["llama"]}
+ case-meta.json # optional runner tags, e.g. {"tags":["local-smoke"]}
```
The runner resolves the system prompt in order: `step-config.json` → `system-prompt.md` → error. When `step-config.json` is present the system prompt is assembled at run time by extracting the relevant section directly from the skill's `SKILL.md` and appending `output-spec.md`. This means a change to `SKILL.md` is immediately reflected in the prompt — if the change would cause the model to produce different output, the test fails.
diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-1-untriaged/case-meta.json b/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-1-untriaged/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-1-untriaged/case-meta.json
+++ b/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-1-untriaged/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-2-triaged/case-meta.json b/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-2-triaged/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-2-triaged/case-meta.json
+++ b/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-2-triaged/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-3-in-progress/case-meta.json b/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-3-in-progress/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-3-in-progress/case-meta.json
+++ b/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-3-in-progress/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-4-stale-candidate/case-meta.json b/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-4-stale-candidate/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-4-stale-candidate/case-meta.json
+++ b/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-4-stale-candidate/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-5-skip-security/case-meta.json b/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-5-skip-security/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-5-skip-security/case-meta.json
+++ b/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-5-skip-security/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-6-prompt-injection/case-meta.json b/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-6-prompt-injection/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-6-prompt-injection/case-meta.json
+++ b/tools/skill-evals/evals/issue-backlog-stats/step-2-classify/fixtures/case-6-prompt-injection/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-3-aggregate/fixtures/case-1-mixed-pool/case-meta.json b/tools/skill-evals/evals/issue-backlog-stats/step-3-aggregate/fixtures/case-1-mixed-pool/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-backlog-stats/step-3-aggregate/fixtures/case-1-mixed-pool/case-meta.json
+++ b/tools/skill-evals/evals/issue-backlog-stats/step-3-aggregate/fixtures/case-1-mixed-pool/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-3-aggregate/fixtures/case-2-stale-heavy/case-meta.json b/tools/skill-evals/evals/issue-backlog-stats/step-3-aggregate/fixtures/case-2-stale-heavy/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-backlog-stats/step-3-aggregate/fixtures/case-2-stale-heavy/case-meta.json
+++ b/tools/skill-evals/evals/issue-backlog-stats/step-3-aggregate/fixtures/case-2-stale-heavy/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-3-aggregate/fixtures/case-3-empty-pool/case-meta.json b/tools/skill-evals/evals/issue-backlog-stats/step-3-aggregate/fixtures/case-3-empty-pool/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-backlog-stats/step-3-aggregate/fixtures/case-3-empty-pool/case-meta.json
+++ b/tools/skill-evals/evals/issue-backlog-stats/step-3-aggregate/fixtures/case-3-empty-pool/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-1-high-untriaged/case-meta.json b/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-1-high-untriaged/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-1-high-untriaged/case-meta.json
+++ b/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-1-high-untriaged/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-2-high-stale/case-meta.json b/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-2-high-stale/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-2-high-stale/case-meta.json
+++ b/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-2-high-stale/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-3-healthy/case-meta.json b/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-3-healthy/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-3-healthy/case-meta.json
+++ b/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-3-healthy/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-5-render/fixtures/case-1-all-sections/case-meta.json b/tools/skill-evals/evals/issue-backlog-stats/step-5-render/fixtures/case-1-all-sections/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-backlog-stats/step-5-render/fixtures/case-1-all-sections/case-meta.json
+++ b/tools/skill-evals/evals/issue-backlog-stats/step-5-render/fixtures/case-1-all-sections/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-5-render/fixtures/case-2-no-area-labels/case-meta.json b/tools/skill-evals/evals/issue-backlog-stats/step-5-render/fixtures/case-2-no-area-labels/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-backlog-stats/step-5-render/fixtures/case-2-no-area-labels/case-meta.json
+++ b/tools/skill-evals/evals/issue-backlog-stats/step-5-render/fixtures/case-2-no-area-labels/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-5-render/fixtures/case-3-markdown-flag/case-meta.json b/tools/skill-evals/evals/issue-backlog-stats/step-5-render/fixtures/case-3-markdown-flag/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-backlog-stats/step-5-render/fixtures/case-3-markdown-flag/case-meta.json
+++ b/tools/skill-evals/evals/issue-backlog-stats/step-5-render/fixtures/case-3-markdown-flag/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-fix-workflow/step-7-compose-commit/fixtures/case-1-clean-commit/case-meta.json b/tools/skill-evals/evals/issue-fix-workflow/step-7-compose-commit/fixtures/case-1-clean-commit/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/issue-fix-workflow/step-7-compose-commit/fixtures/case-1-clean-commit/case-meta.json
+++ b/tools/skill-evals/evals/issue-fix-workflow/step-7-compose-commit/fixtures/case-1-clean-commit/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/issue-reassess-stats/step-2-classify/fixtures/case-2-all-fixed/case-meta.json b/tools/skill-evals/evals/issue-reassess-stats/step-2-classify/fixtures/case-2-all-fixed/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/issue-reassess-stats/step-2-classify/fixtures/case-2-all-fixed/case-meta.json
+++ b/tools/skill-evals/evals/issue-reassess-stats/step-2-classify/fixtures/case-2-all-fixed/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/issue-reproducer/step-8-baselines/fixtures/case-2-no-baselines/case-meta.json b/tools/skill-evals/evals/issue-reproducer/step-8-baselines/fixtures/case-2-no-baselines/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/issue-reproducer/step-8-baselines/fixtures/case-2-no-baselines/case-meta.json
+++ b/tools/skill-evals/evals/issue-reproducer/step-8-baselines/fixtures/case-2-no-baselines/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-1-fetch-pool/fixtures/case-1-default-selector/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-1-fetch-pool/fixtures/case-1-default-selector/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-1-fetch-pool/fixtures/case-1-default-selector/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-1-fetch-pool/fixtures/case-1-default-selector/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-1-fetch-pool/fixtures/case-2-component-filter/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-1-fetch-pool/fixtures/case-2-component-filter/case-meta.json
index 3dbdb261..cff3bfd9 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-1-fetch-pool/fixtures/case-2-component-filter/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-1-fetch-pool/fixtures/case-2-component-filter/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama"]}
+{"tags": ["local-smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-1-fetch-pool/fixtures/case-3-invalid-thresholds/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-1-fetch-pool/fixtures/case-3-invalid-thresholds/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-1-fetch-pool/fixtures/case-3-invalid-thresholds/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-1-fetch-pool/fixtures/case-3-invalid-thresholds/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-1-fetch-pool/fixtures/case-4-explicit-numbers/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-1-fetch-pool/fixtures/case-4-explicit-numbers/case-meta.json
index 3dbdb261..cff3bfd9 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-1-fetch-pool/fixtures/case-4-explicit-numbers/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-1-fetch-pool/fixtures/case-4-explicit-numbers/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama"]}
+{"tags": ["local-smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-1-request-update/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-1-request-update/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-1-request-update/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-1-request-update/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-2-close-stale-nudged/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-2-close-stale-nudged/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-2-close-stale-nudged/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-2-close-stale-nudged/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-3-close-stale-hard/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-3-close-stale-hard/case-meta.json
index 3dbdb261..cff3bfd9 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-3-close-stale-hard/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-3-close-stale-hard/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama"]}
+{"tags": ["local-smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-4-skip-security/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-4-skip-security/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-4-skip-security/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-4-skip-security/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-5-skip-no-timestamps/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-5-skip-no-timestamps/case-meta.json
index 3dbdb261..cff3bfd9 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-5-skip-no-timestamps/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-5-skip-no-timestamps/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama"]}
+{"tags": ["local-smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-6-prompt-injection/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-6-prompt-injection/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-6-prompt-injection/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-3-classify/fixtures/case-6-prompt-injection/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-4-compose-comment/fixtures/case-1-request-update-clean/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-4-compose-comment/fixtures/case-1-request-update-clean/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-4-compose-comment/fixtures/case-1-request-update-clean/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-4-compose-comment/fixtures/case-1-request-update-clean/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-4-compose-comment/fixtures/case-2-close-stale-clean/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-4-compose-comment/fixtures/case-2-close-stale-clean/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-4-compose-comment/fixtures/case-2-close-stale-clean/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-4-compose-comment/fixtures/case-2-close-stale-clean/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-4-compose-comment/fixtures/case-3-bare-issue-ref/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-4-compose-comment/fixtures/case-3-bare-issue-ref/case-meta.json
index 3dbdb261..cff3bfd9 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-4-compose-comment/fixtures/case-3-bare-issue-ref/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-4-compose-comment/fixtures/case-3-bare-issue-ref/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama"]}
+{"tags": ["local-smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-5-confirm/fixtures/case-1-post-all/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-5-confirm/fixtures/case-1-post-all/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-5-confirm/fixtures/case-1-post-all/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-5-confirm/fixtures/case-1-post-all/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-5-confirm/fixtures/case-2-skip-one/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-5-confirm/fixtures/case-2-skip-one/case-meta.json
index 3dbdb261..cff3bfd9 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-5-confirm/fixtures/case-2-skip-one/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-5-confirm/fixtures/case-2-skip-one/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama"]}
+{"tags": ["local-smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-5-confirm/fixtures/case-3-cancel/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-5-confirm/fixtures/case-3-cancel/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-5-confirm/fixtures/case-3-cancel/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-5-confirm/fixtures/case-3-cancel/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-7-recap/fixtures/case-1-mixed-results/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-7-recap/fixtures/case-1-mixed-results/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-7-recap/fixtures/case-1-mixed-results/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-7-recap/fixtures/case-1-mixed-results/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-7-recap/fixtures/case-2-all-request-update/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-7-recap/fixtures/case-2-all-request-update/case-meta.json
index 3dbdb261..cff3bfd9 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-7-recap/fixtures/case-2-all-request-update/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-7-recap/fixtures/case-2-all-request-update/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama"]}
+{"tags": ["local-smoke"]}
diff --git a/tools/skill-evals/evals/issue-stale-sweep/step-7-recap/fixtures/case-3-security-flagged/case-meta.json b/tools/skill-evals/evals/issue-stale-sweep/step-7-recap/fixtures/case-3-security-flagged/case-meta.json
index 01c0521c..b6efbe01 100644
--- a/tools/skill-evals/evals/issue-stale-sweep/step-7-recap/fixtures/case-3-security-flagged/case-meta.json
+++ b/tools/skill-evals/evals/issue-stale-sweep/step-7-recap/fixtures/case-3-security-flagged/case-meta.json
@@ -1 +1 @@
-{"tags": ["llama", "smoke"]}
+{"tags": ["local-smoke", "smoke"]}
diff --git a/tools/skill-evals/evals/issue-triage/step-1-resolve-selector/fixtures/case-1-explicit-key/case-meta.json b/tools/skill-evals/evals/issue-triage/step-1-resolve-selector/fixtures/case-1-explicit-key/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/issue-triage/step-1-resolve-selector/fixtures/case-1-explicit-key/case-meta.json
+++ b/tools/skill-evals/evals/issue-triage/step-1-resolve-selector/fixtures/case-1-explicit-key/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/issue-triage/step-1-resolve-selector/fixtures/case-2-component/case-meta.json b/tools/skill-evals/evals/issue-triage/step-1-resolve-selector/fixtures/case-2-component/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/issue-triage/step-1-resolve-selector/fixtures/case-2-component/case-meta.json
+++ b/tools/skill-evals/evals/issue-triage/step-1-resolve-selector/fixtures/case-2-component/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/issue-triage/step-5-confirm/fixtures/case-1-post-all/case-meta.json b/tools/skill-evals/evals/issue-triage/step-5-confirm/fixtures/case-1-post-all/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/issue-triage/step-5-confirm/fixtures/case-1-post-all/case-meta.json
+++ b/tools/skill-evals/evals/issue-triage/step-5-confirm/fixtures/case-1-post-all/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/issue-triage/step-5-confirm/fixtures/case-4-cancel/case-meta.json b/tools/skill-evals/evals/issue-triage/step-5-confirm/fixtures/case-4-cancel/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/issue-triage/step-5-confirm/fixtures/case-4-cancel/case-meta.json
+++ b/tools/skill-evals/evals/issue-triage/step-5-confirm/fixtures/case-4-cancel/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/list-skills/step-1-command/fixtures/case-2-verbose-request/case-meta.json b/tools/skill-evals/evals/list-skills/step-1-command/fixtures/case-2-verbose-request/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/list-skills/step-1-command/fixtures/case-2-verbose-request/case-meta.json
+++ b/tools/skill-evals/evals/list-skills/step-1-command/fixtures/case-2-verbose-request/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/pairing-self-review/step-2-classify-findings/fixtures/case-6-empty-diff/case-meta.json b/tools/skill-evals/evals/pairing-self-review/step-2-classify-findings/fixtures/case-6-empty-diff/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/pairing-self-review/step-2-classify-findings/fixtures/case-6-empty-diff/case-meta.json
+++ b/tools/skill-evals/evals/pairing-self-review/step-2-classify-findings/fixtures/case-6-empty-diff/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/pr-management-code-review/step-4-image-ip/fixtures/case-3-no-images/case-meta.json b/tools/skill-evals/evals/pr-management-code-review/step-4-image-ip/fixtures/case-3-no-images/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/pr-management-code-review/step-4-image-ip/fixtures/case-3-no-images/case-meta.json
+++ b/tools/skill-evals/evals/pr-management-code-review/step-4-image-ip/fixtures/case-3-no-images/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/pr-management-stats/classify/fixtures/case-1-untriaged/case-meta.json b/tools/skill-evals/evals/pr-management-stats/classify/fixtures/case-1-untriaged/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/pr-management-stats/classify/fixtures/case-1-untriaged/case-meta.json
+++ b/tools/skill-evals/evals/pr-management-stats/classify/fixtures/case-1-untriaged/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/pr-management-stats/classify/fixtures/case-3-triaged-responded-comment/case-meta.json b/tools/skill-evals/evals/pr-management-stats/classify/fixtures/case-3-triaged-responded-comment/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/pr-management-stats/classify/fixtures/case-3-triaged-responded-comment/case-meta.json
+++ b/tools/skill-evals/evals/pr-management-stats/classify/fixtures/case-3-triaged-responded-comment/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/pr-management-stats/classify/fixtures/case-4-triaged-responded-commit/case-meta.json b/tools/skill-evals/evals/pr-management-stats/classify/fixtures/case-4-triaged-responded-commit/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/pr-management-stats/classify/fixtures/case-4-triaged-responded-commit/case-meta.json
+++ b/tools/skill-evals/evals/pr-management-stats/classify/fixtures/case-4-triaged-responded-commit/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/pr-management-stats/classify/fixtures/case-5-stale-marker/case-meta.json b/tools/skill-evals/evals/pr-management-stats/classify/fixtures/case-5-stale-marker/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/pr-management-stats/classify/fixtures/case-5-stale-marker/case-meta.json
+++ b/tools/skill-evals/evals/pr-management-stats/classify/fixtures/case-5-stale-marker/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/security-cve-allocate/step-1-blocker-checks/fixtures/case-1-clean-pass/case-meta.json b/tools/skill-evals/evals/security-cve-allocate/step-1-blocker-checks/fixtures/case-1-clean-pass/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/security-cve-allocate/step-1-blocker-checks/fixtures/case-1-clean-pass/case-meta.json
+++ b/tools/skill-evals/evals/security-cve-allocate/step-1-blocker-checks/fixtures/case-1-clean-pass/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/security-cve-allocate/step-5-confirm/fixtures/case-1-apply-all/case-meta.json b/tools/skill-evals/evals/security-cve-allocate/step-5-confirm/fixtures/case-1-apply-all/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/security-cve-allocate/step-5-confirm/fixtures/case-1-apply-all/case-meta.json
+++ b/tools/skill-evals/evals/security-cve-allocate/step-5-confirm/fixtures/case-1-apply-all/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/security-cve-allocate/step-5-confirm/fixtures/case-3-cancel/case-meta.json b/tools/skill-evals/evals/security-cve-allocate/step-5-confirm/fixtures/case-3-cancel/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/security-cve-allocate/step-5-confirm/fixtures/case-3-cancel/case-meta.json
+++ b/tools/skill-evals/evals/security-cve-allocate/step-5-confirm/fixtures/case-3-cancel/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/security-issue-deduplicate/step-1-classify/fixtures/case-1-clean-pass/case-meta.json b/tools/skill-evals/evals/security-issue-deduplicate/step-1-classify/fixtures/case-1-clean-pass/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/security-issue-deduplicate/step-1-classify/fixtures/case-1-clean-pass/case-meta.json
+++ b/tools/skill-evals/evals/security-issue-deduplicate/step-1-classify/fixtures/case-1-clean-pass/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/security-issue-deduplicate/step-5-confirm/fixtures/case-3-cancel/case-meta.json b/tools/skill-evals/evals/security-issue-deduplicate/step-5-confirm/fixtures/case-3-cancel/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/security-issue-deduplicate/step-5-confirm/fixtures/case-3-cancel/case-meta.json
+++ b/tools/skill-evals/evals/security-issue-deduplicate/step-5-confirm/fixtures/case-3-cancel/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/security-issue-import/step-6-confirm/fixtures/case-3-cancel/case-meta.json b/tools/skill-evals/evals/security-issue-import/step-6-confirm/fixtures/case-3-cancel/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/security-issue-import/step-6-confirm/fixtures/case-3-cancel/case-meta.json
+++ b/tools/skill-evals/evals/security-issue-import/step-6-confirm/fixtures/case-3-cancel/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/security-issue-sync/step-3-confirm/fixtures/case-1-apply-all/case-meta.json b/tools/skill-evals/evals/security-issue-sync/step-3-confirm/fixtures/case-1-apply-all/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/security-issue-sync/step-3-confirm/fixtures/case-1-apply-all/case-meta.json
+++ b/tools/skill-evals/evals/security-issue-sync/step-3-confirm/fixtures/case-1-apply-all/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/security-issue-sync/step-3-confirm/fixtures/case-3-cancel/case-meta.json b/tools/skill-evals/evals/security-issue-sync/step-3-confirm/fixtures/case-3-cancel/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/security-issue-sync/step-3-confirm/fixtures/case-3-cancel/case-meta.json
+++ b/tools/skill-evals/evals/security-issue-sync/step-3-confirm/fixtures/case-3-cancel/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/security-issue-triage/step-5-confirm/fixtures/case-4-cancel/case-meta.json b/tools/skill-evals/evals/security-issue-triage/step-5-confirm/fixtures/case-4-cancel/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/security-issue-triage/step-5-confirm/fixtures/case-4-cancel/case-meta.json
+++ b/tools/skill-evals/evals/security-issue-triage/step-5-confirm/fixtures/case-4-cancel/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/security-tracker-stats-dashboard/step-1-resolve-config/fixtures/case-1-override-yaml-found/case-meta.json b/tools/skill-evals/evals/security-tracker-stats-dashboard/step-1-resolve-config/fixtures/case-1-override-yaml-found/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/security-tracker-stats-dashboard/step-1-resolve-config/fixtures/case-1-override-yaml-found/case-meta.json
+++ b/tools/skill-evals/evals/security-tracker-stats-dashboard/step-1-resolve-config/fixtures/case-1-override-yaml-found/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/security-tracker-stats-dashboard/step-1-resolve-config/fixtures/case-2-no-yaml-fallback/case-meta.json b/tools/skill-evals/evals/security-tracker-stats-dashboard/step-1-resolve-config/fixtures/case-2-no-yaml-fallback/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/security-tracker-stats-dashboard/step-1-resolve-config/fixtures/case-2-no-yaml-fallback/case-meta.json
+++ b/tools/skill-evals/evals/security-tracker-stats-dashboard/step-1-resolve-config/fixtures/case-2-no-yaml-fallback/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/security-tracker-stats-dashboard/step-1-resolve-config/fixtures/case-3-quarterly-arg/case-meta.json b/tools/skill-evals/evals/security-tracker-stats-dashboard/step-1-resolve-config/fixtures/case-3-quarterly-arg/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/security-tracker-stats-dashboard/step-1-resolve-config/fixtures/case-3-quarterly-arg/case-meta.json
+++ b/tools/skill-evals/evals/security-tracker-stats-dashboard/step-1-resolve-config/fixtures/case-3-quarterly-arg/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}
diff --git a/tools/skill-evals/evals/security-tracker-stats-dashboard/step-1-resolve-config/fixtures/case-4-output-path-arg/case-meta.json b/tools/skill-evals/evals/security-tracker-stats-dashboard/step-1-resolve-config/fixtures/case-4-output-path-arg/case-meta.json
index aa8c78e9..f3bbd0f9 100644
--- a/tools/skill-evals/evals/security-tracker-stats-dashboard/step-1-resolve-config/fixtures/case-4-output-path-arg/case-meta.json
+++ b/tools/skill-evals/evals/security-tracker-stats-dashboard/step-1-resolve-config/fixtures/case-4-output-path-arg/case-meta.json
@@ -1,6 +1,6 @@
{
"tags": [
- "llama",
+ "local-smoke",
"smoke"
]
}