You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@sourcelume.apache.org by an...@apache.org on 2026/10/20 21:23:45 UTC
(sourcelume-sample) 01/01: Update spec pin, CI validation, and sample naming
This is an automated email from the ASF dual-hosted git repository.
ANierbeck pushed a commit to branch update-spec-pin-and-naming
in repository https://gitbox.apache.org/repos/asf/sourcelume-sample.git
commit 5958152bee1ad2a817e8042e33685bce2ae19a06
Author: anierbeck <ac...@codecentric.de>
AuthorDate: Tue Oct 20 23:06:28 2026 +0200
Update spec pin, CI validation, and sample naming
- validate-records.py: fail fast without SOURCELUME_SPEC instead of
defaulting to a stale local worktree path; PR #10 is merged into
spec main, so any current checkout works
- ci.yml: check out apache/sourcelume-spec and validate both records
in CI (previously skipped); spec main is followed unpinned until a
spec release is published
- README: record-type status rows now reference merged spec main and
the registry follow-up PR (sourcelume-registry#15)
- train.py and data/provenance-record.json: use the sample naming
consistently after the training-demo rename
Apache-ai: Yes
Generated-by: pi (claude-sonnet-5-5)
Reviewed-by: anierbeck
---
.github/workflows/ci.yml | 18 +++++++++---------
README.md | 4 ++--
data/provenance-record.json | 2 +-
scripts/validate-records.py | 13 ++++++++++---
src/train.py | 6 +++---
5 files changed, 25 insertions(+), 18 deletions(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 2c4a308..d9b1aa2 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -27,6 +27,11 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
+ - name: Check out the spec for record validation
+ uses: actions/checkout@v4
+ with:
+ repository: apache/sourcelume-spec
+ path: sourcelume-spec
- uses: actions/setup-python@v5
with:
python-version: '3.12'
@@ -38,12 +43,7 @@ jobs:
run: uv run python src/train.py --epochs 1 --chars 20000
- name: Validate records against the spec
env:
- # TODO: pin to the spec release once PR #10 (TrainingProvenanceRecord)
- # is merged and published; until then CI validates offline only.
- SOURCELUME_SPEC: ''
- run: |
- if [ -n "$SOURCELUME_SPEC" ]; then
- uv run python scripts/validate-records.py
- else
- echo "record validation skipped: no spec checkout available in CI yet"
- fi
+ # Unpinned main for now: the spec has no published release yet, and
+ # validating against the current spec is the point of this check.
+ SOURCELUME_SPEC: ${{ github.workspace }}/sourcelume-spec
+ run: uv run python scripts/validate-records.py
diff --git a/README.md b/README.md
index fa8cb32..172b248 100644
--- a/README.md
+++ b/README.md
@@ -64,9 +64,9 @@ make demo
|---|---|
| Dataset record: spec + registry validation | ✅ works today |
| Dataset record: publish to Atlas | ✅ works with the Atlas stack up |
-| Training record: spec validation | ✅ works — **requires a spec checkout at the `model-usage-record` (PR #10) state or later**; set `SOURCELUME_SPEC` accordingly |
+| Training record: spec validation | ✅ works — validate against a `sourcelume-spec` checkout at `main` (the `TrainingProvenanceRecord` type was merged with PR #10); set `SOURCELUME_SPEC` accordingly |
| Training record: plugin validate/publish | ❌ **expected to fail** — the Python plugin itself only models `ProvenanceRecord` (its Pydantic model rejects the record client-side). The plugin needs the same type-dispatch the spec validator has (tooling follow-up; see `make registry-validate`). |
-| Training record: registry validate/publish | ❌ **expected to fail** until the registry learns the `TrainingProvenanceRecord` type (registry follow-up to PR #10). Direct `curl -X POST .../records/validate` shows the registry's own rejection (HTTP 422, `must be the constant value 'ProvenanceRecord'`) — the demo's exhibit for that follow-up. |
+| Training record: registry validate/publish | ❌ **expected to fail** until the registry learns the `TrainingProvenanceRecord` type (registry follow-up to PR #10, currently in review as sourcelume-registry#15). Direct `curl -X POST .../records/validate` shows the registry's own rejection (HTTP 422, `must be the constant value 'ProvenanceRecord'`) — the demo's exhibit for that follow-up. |
This demo **builds on PR #10 as given**: the training record type, the
type-dispatching validator and the record examples come from that branch.
diff --git a/data/provenance-record.json b/data/provenance-record.json
index a01c50b..90210af 100644
--- a/data/provenance-record.json
+++ b/data/provenance-record.json
@@ -28,7 +28,7 @@
"created": "2026-10-15T00:00:00Z",
"added": "2026-10-15T00:00:00Z",
"contentCreated": "1865-11-26T00:00:00Z",
- "origin": "Plain text of Lewis Carroll's 'Alice's Adventures in Wonderland' (first published 26 November 1865), fetched verbatim from Project Gutenberg (ebook #11, https://www.gutenberg.org/ebooks/11) in October 2026 and bundled unchanged at data/corpus/alice-in-wonderland-1865.txt as demo training data. The work is in the public domain worldwide (published 1865, author died 1898); the Project Gutenberg framing text included in the file is distributed under the Project Gutenberg Licens [...]
+ "origin": "Plain text of Lewis Carroll's 'Alice's Adventures in Wonderland' (first published 26 November 1865), fetched verbatim from Project Gutenberg (ebook #11, https://www.gutenberg.org/ebooks/11) in October 2026 and bundled unchanged at data/corpus/alice-in-wonderland-1865.txt as demo training data. The work is in the public domain worldwide (published 1865, author died 1898); the Project Gutenberg framing text included in the file is distributed under the Project Gutenberg Licens [...]
"custodyChain": [
{
"agent": "https://www.gutenberg.org/",
diff --git a/scripts/validate-records.py b/scripts/validate-records.py
index aa8401f..5b2e17e 100644
--- a/scripts/validate-records.py
+++ b/scripts/validate-records.py
@@ -18,8 +18,8 @@
SHACL), reusing the spec's own validator.
SOURCELUME_SPEC must point at a sourcelume-spec checkout that includes the
-TrainingProvenanceRecord type — i.e. the model-usage-record branch (PR #10)
-until it is merged into main. Defaults to the local worktree of that branch.
+TrainingProvenanceRecord type (spec main since PR #10). There is no default:
+point it at your local checkout, e.g. SOURCELUME_SPEC=../sourcelume-spec.
"""
import json
@@ -28,7 +28,14 @@ import sys
from pathlib import Path
BASE = Path(__file__).resolve().parent.parent
-SPEC = Path(os.environ.get("SOURCELUME_SPEC", "/private/tmp/wt-model")).resolve()
+spec_env = os.environ.get("SOURCELUME_SPEC", "")
+if not spec_env:
+ print(
+ "SOURCELUME_SPEC must point at a sourcelume-spec checkout (e.g. ../sourcelume-spec)",
+ file=sys.stderr,
+ )
+ sys.exit(2)
+SPEC = Path(spec_env).resolve()
if not (SPEC / "tools" / "validate.py").is_file():
print(f"SOURCELUME_SPEC is not a spec checkout: {SPEC}", file=sys.stderr)
sys.exit(2)
diff --git a/src/train.py b/src/train.py
index fd93ccd..4a7d4b7 100644
--- a/src/train.py
+++ b/src/train.py
@@ -140,12 +140,12 @@ def main() -> int:
"id": MODEL_ID,
"type": "TrainingProvenanceRecord",
"identifier": "tiny-lm",
- "name": "Tiny char-LM (Apache Sourcelume training demo)",
+ "name": "Tiny char-LM (Apache Sourcelume sample)",
"version": "v1",
"creator": [
{
"type": "schema:Organization",
- "name": "Apache Sourcelume training demo",
+ "name": "Apache Sourcelume sample",
"role": "originator"
}
],
@@ -167,7 +167,7 @@ def main() -> int:
f"{tokens_per_epoch}-character slice of the referenced corpus "
f"({len(vocab)}-character vocabulary, {params} parameters, "
f"final loss {history[-1]:.4f}, see model/training-run.json). "
- f"The referenced dataset record is published by this demo itself "
+ f"The referenced dataset record is published by this sample itself "
f"(data/provenance-record.json). Unsigned producer claim — see README."
),
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@sourcelume.apache.org
For additional commands, e-mail: commits-help@sourcelume.apache.org