You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@sourcelume.apache.org by an...@apache.org on 2026/10/20 21:23:44 UTC
(sourcelume-sample) branch update-spec-pin-and-naming created (now 5958152)
This is an automated email from the ASF dual-hosted git repository.
ANierbeck pushed a change to branch update-spec-pin-and-naming
in repository https://gitbox.apache.org/repos/asf/sourcelume-sample.git
at 5958152 Update spec pin, CI validation, and sample naming
This branch includes the following new commits:
new 5958152 Update spec pin, CI validation, and sample naming
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@sourcelume.apache.org
For additional commands, e-mail: commits-help@sourcelume.apache.org
(sourcelume-sample) 01/01: Update spec pin, CI validation, and sample naming
Posted by an...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ANierbeck pushed a commit to branch update-spec-pin-and-naming
in repository https://gitbox.apache.org/repos/asf/sourcelume-sample.git
commit 5958152bee1ad2a817e8042e33685bce2ae19a06
Author: anierbeck <ac...@codecentric.de>
AuthorDate: Tue Oct 20 23:06:28 2026 +0200
Update spec pin, CI validation, and sample naming
- validate-records.py: fail fast without SOURCELUME_SPEC instead of
defaulting to a stale local worktree path; PR #10 is merged into
spec main, so any current checkout works
- ci.yml: check out apache/sourcelume-spec and validate both records
in CI (previously skipped); spec main is followed unpinned until a
spec release is published
- README: record-type status rows now reference merged spec main and
the registry follow-up PR (sourcelume-registry#15)
- train.py and data/provenance-record.json: use the sample naming
consistently after the training-demo rename
Apache-ai: Yes
Generated-by: pi (claude-sonnet-5-5)
Reviewed-by: anierbeck
---
.github/workflows/ci.yml | 18 +++++++++---------
README.md | 4 ++--
data/provenance-record.json | 2 +-
scripts/validate-records.py | 13 ++++++++++---
src/train.py | 6 +++---
5 files changed, 25 insertions(+), 18 deletions(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 2c4a308..d9b1aa2 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -27,6 +27,11 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
+ - name: Check out the spec for record validation
+ uses: actions/checkout@v4
+ with:
+ repository: apache/sourcelume-spec
+ path: sourcelume-spec
- uses: actions/setup-python@v5
with:
python-version: '3.12'
@@ -38,12 +43,7 @@ jobs:
run: uv run python src/train.py --epochs 1 --chars 20000
- name: Validate records against the spec
env:
- # TODO: pin to the spec release once PR #10 (TrainingProvenanceRecord)
- # is merged and published; until then CI validates offline only.
- SOURCELUME_SPEC: ''
- run: |
- if [ -n "$SOURCELUME_SPEC" ]; then
- uv run python scripts/validate-records.py
- else
- echo "record validation skipped: no spec checkout available in CI yet"
- fi
+ # Unpinned main for now: the spec has no published release yet, and
+ # validating against the current spec is the point of this check.
+ SOURCELUME_SPEC: ${{ github.workspace }}/sourcelume-spec
+ run: uv run python scripts/validate-records.py
diff --git a/README.md b/README.md
index fa8cb32..172b248 100644
--- a/README.md
+++ b/README.md
@@ -64,9 +64,9 @@ make demo
|---|---|
| Dataset record: spec + registry validation | ✅ works today |
| Dataset record: publish to Atlas | ✅ works with the Atlas stack up |
-| Training record: spec validation | ✅ works — **requires a spec checkout at the `model-usage-record` (PR #10) state or later**; set `SOURCELUME_SPEC` accordingly |
+| Training record: spec validation | ✅ works — validate against a `sourcelume-spec` checkout at `main` (the `TrainingProvenanceRecord` type was merged with PR #10); set `SOURCELUME_SPEC` accordingly |
| Training record: plugin validate/publish | ❌ **expected to fail** — the Python plugin itself only models `ProvenanceRecord` (its Pydantic model rejects the record client-side). The plugin needs the same type-dispatch the spec validator has (tooling follow-up; see `make registry-validate`). |
-| Training record: registry validate/publish | ❌ **expected to fail** until the registry learns the `TrainingProvenanceRecord` type (registry follow-up to PR #10). Direct `curl -X POST .../records/validate` shows the registry's own rejection (HTTP 422, `must be the constant value 'ProvenanceRecord'`) — the demo's exhibit for that follow-up. |
+| Training record: registry validate/publish | ❌ **expected to fail** until the registry learns the `TrainingProvenanceRecord` type (registry follow-up to PR #10, currently in review as sourcelume-registry#15). Direct `curl -X POST .../records/validate` shows the registry's own rejection (HTTP 422, `must be the constant value 'ProvenanceRecord'`) — the demo's exhibit for that follow-up. |
This demo **builds on PR #10 as given**: the training record type, the
type-dispatching validator and the record examples come from that branch.
diff --git a/data/provenance-record.json b/data/provenance-record.json
index a01c50b..90210af 100644
--- a/data/provenance-record.json
+++ b/data/provenance-record.json
@@ -28,7 +28,7 @@
"created": "2026-10-15T00:00:00Z",
"added": "2026-10-15T00:00:00Z",
"contentCreated": "1865-11-26T00:00:00Z",
- "origin": "Plain text of Lewis Carroll's 'Alice's Adventures in Wonderland' (first published 26 November 1865), fetched verbatim from Project Gutenberg (ebook #11, https://www.gutenberg.org/ebooks/11) in October 2026 and bundled unchanged at data/corpus/alice-in-wonderland-1865.txt as demo training data. The work is in the public domain worldwide (published 1865, author died 1898); the Project Gutenberg framing text included in the file is distributed under the Project Gutenberg Licens [...]
+ "origin": "Plain text of Lewis Carroll's 'Alice's Adventures in Wonderland' (first published 26 November 1865), fetched verbatim from Project Gutenberg (ebook #11, https://www.gutenberg.org/ebooks/11) in October 2026 and bundled unchanged at data/corpus/alice-in-wonderland-1865.txt as demo training data. The work is in the public domain worldwide (published 1865, author died 1898); the Project Gutenberg framing text included in the file is distributed under the Project Gutenberg Licens [...]
"custodyChain": [
{
"agent": "https://www.gutenberg.org/",
diff --git a/scripts/validate-records.py b/scripts/validate-records.py
index aa8401f..5b2e17e 100644
--- a/scripts/validate-records.py
+++ b/scripts/validate-records.py
@@ -18,8 +18,8 @@
SHACL), reusing the spec's own validator.
SOURCELUME_SPEC must point at a sourcelume-spec checkout that includes the
-TrainingProvenanceRecord type — i.e. the model-usage-record branch (PR #10)
-until it is merged into main. Defaults to the local worktree of that branch.
+TrainingProvenanceRecord type (spec main since PR #10). There is no default:
+point it at your local checkout, e.g. SOURCELUME_SPEC=../sourcelume-spec.
"""
import json
@@ -28,7 +28,14 @@ import sys
from pathlib import Path
BASE = Path(__file__).resolve().parent.parent
-SPEC = Path(os.environ.get("SOURCELUME_SPEC", "/private/tmp/wt-model")).resolve()
+spec_env = os.environ.get("SOURCELUME_SPEC", "")
+if not spec_env:
+ print(
+ "SOURCELUME_SPEC must point at a sourcelume-spec checkout (e.g. ../sourcelume-spec)",
+ file=sys.stderr,
+ )
+ sys.exit(2)
+SPEC = Path(spec_env).resolve()
if not (SPEC / "tools" / "validate.py").is_file():
print(f"SOURCELUME_SPEC is not a spec checkout: {SPEC}", file=sys.stderr)
sys.exit(2)
diff --git a/src/train.py b/src/train.py
index fd93ccd..4a7d4b7 100644
--- a/src/train.py
+++ b/src/train.py
@@ -140,12 +140,12 @@ def main() -> int:
"id": MODEL_ID,
"type": "TrainingProvenanceRecord",
"identifier": "tiny-lm",
- "name": "Tiny char-LM (Apache Sourcelume training demo)",
+ "name": "Tiny char-LM (Apache Sourcelume sample)",
"version": "v1",
"creator": [
{
"type": "schema:Organization",
- "name": "Apache Sourcelume training demo",
+ "name": "Apache Sourcelume sample",
"role": "originator"
}
],
@@ -167,7 +167,7 @@ def main() -> int:
f"{tokens_per_epoch}-character slice of the referenced corpus "
f"({len(vocab)}-character vocabulary, {params} parameters, "
f"final loss {history[-1]:.4f}, see model/training-run.json). "
- f"The referenced dataset record is published by this demo itself "
+ f"The referenced dataset record is published by this sample itself "
f"(data/provenance-record.json). Unsigned producer claim — see README."
),
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@sourcelume.apache.org
For additional commands, e-mail: commits-help@sourcelume.apache.org