You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@sourcelume.apache.org by jg...@apache.org on 2026/10/11 16:45:11 UTC
(sourcelume-registry) branch coc-2026 created (now e195fc2)
This is an automated email from the ASF dual-hosted git repository.
jgoodyear pushed a change to branch coc-2026
in repository https://gitbox.apache.org/repos/asf/sourcelume-registry.git
at e195fc2 Updates to demo
This branch includes the following new commits:
new 87919b4 Community over Code Glasgow 2026 demo.
new e195fc2 Updates to demo
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@sourcelume.apache.org
For additional commands, e-mail: commits-help@sourcelume.apache.org
(sourcelume-registry) 01/02: Community over Code Glasgow 2026 demo.
Posted by jg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jgoodyear pushed a commit to branch coc-2026
in repository https://gitbox.apache.org/repos/asf/sourcelume-registry.git
commit 87919b4ee84e9c17e70596583a166acbeeabca85
Author: jgoodyear <jg...@apache.org>
AuthorDate: Sun Oct 11 13:59:47 2026 -0230
Community over Code Glasgow 2026 demo.
Apache-ai: Yes
Generated-by: Junie (Gemini 3.7 Flash)
Reviewed-by: jgoodyear
---
README.md | 4 +
examples/simple-demo/README.md | 198 +++++++++++++++++++++
examples/simple-demo/run-demo.sh | 174 ++++++++++++++++++
examples/simple-demo/sample-dataset.json | 7 +
.../simple-demo/sample-provenance-record.jsonld | 37 ++++
5 files changed, 420 insertions(+)
diff --git a/README.md b/README.md
index 4b748d7..a9f859c 100644
--- a/README.md
+++ b/README.md
@@ -102,6 +102,10 @@ curl -s http://localhost:8082/api/v1/types/typedefs | jq .
curl -s http://localhost:8082/api/v1/types/dataset | jq .
```
+## Examples
+
+- **Simple Demo (`examples/simple-demo`)**: A self-contained, automated walkthrough (referencing the Community Over Code Glasgow 2026 Hackathon) demonstrating registry health verification, Atlas typedef inspection, dataset registration, and query resolution. See [examples/simple-demo/README.md](examples/simple-demo/README.md).
+
## Get involved
- Mailing list: dev@sourcelume.apache.org ([archives](https://lists.apache.org/list.html?dev@sourcelume.apache.org))
diff --git a/examples/simple-demo/README.md b/examples/simple-demo/README.md
new file mode 100644
index 0000000..8b1f786
--- /dev/null
+++ b/examples/simple-demo/README.md
@@ -0,0 +1,198 @@
+# Apache Sourcelume Simple Demo
+**Community Over Code Glasgow 2026 Hackathon**
+
+This demo provides a hands-on walkthrough of **Apache Sourcelume Spec** and **Apache Sourcelume Registry** working together with **Apache Atlas** to persist, index, and query dataset metadata and provenance records.
+
+---
+
+## What This Demo Demonstrates
+
+### The Problem
+AI models are often trained on large-scale datasets with murky origins, untracked licensing constraints, and missing audit trails. Without standardized provenance, organizations risk intellectual property infringement, non-compliance with open source licenses (e.g. SPDX), and untraceable data contamination.
+
+### The Sourcelume Solution
+1. **Spec (`sourcelume-spec`)**: A standardized JSON-LD / SHACL provenance specification defining data structures for dataset identification, custody chains, actor attestations, and SPDX licensing history.
+2. **Registry (`sourcelume-registry`)**: A cloud-native metadata registry service that validates dataset records against spec schemas, automatically bootstraps domain type definitions into **Apache Atlas**, and provides REST APIs for dataset registration and query resolution.
+
+### The Story You Present
+Within seconds, an operator or CI pipeline can:
+1. Verify system readiness and Atlas backend connectivity.
+2. Observe zero-configuration bootstrapping of Sourcelume's custom type system (`sourcelume_dataset`) into Apache Atlas.
+3. Ingest dataset provenance metadata (curated during the Glasgow Hackathon) with automated validation.
+4. Query the dataset by both canonical domain URI (`qualifiedName`) and internal graph GUID.
+5. Visually explore the dataset entity, attributes, and audit history in the Apache Atlas Web UI.
+
+---
+
+## Directory Contents
+
+| File | Description |
+|---|---|
+| `run-demo.sh` | Automated end-to-end demo script with clear narration, formatted console output, and step-by-step verification. |
+| `sample-dataset.json` | Sample dataset payload mapped to `SourcelumeDatasetDto` for registry ingestion. |
+| `sample-provenance-record.jsonld` | Spec-compliant JSON-LD provenance card showing full metadata, custody chain, and SPDX licensing. |
+| `README.md` | This guided walkthrough and reference documentation. |
+
+---
+
+## Prerequisites
+
+Before running the demo, ensure you have:
+
+- **Docker Engine** (≥ 20.10) & **Docker Compose** (≥ 1.28)
+- **Java 21** (LTS) & **Maven 3.9+**
+- `curl` and `jq` CLI utilities installed
+
+---
+
+## Quickstart (Automated Execution)
+
+### 1. Start the Environment
+
+Make sure Apache Atlas and the Sourcelume Registry runtime are running:
+
+```bash
+# Terminal 1: Start Apache Atlas backend (from project root)
+cd dev-support
+./setup.sh
+# Ensure Atlas container is running (see dev-support/README.md for build steps)
+docker compose -f docker-compose.atlas.yml up -d
+
+# Terminal 2: Start Quarkus Registry runtime (from project root)
+mvn -pl sourcelume-registry-runtime-quarkus quarkus:dev
+```
+
+### 2. Run the Demo Script
+
+From the `examples/simple-demo` directory:
+
+```bash
+cd examples/simple-demo
+./run-demo.sh
+```
+
+---
+
+## What `run-demo.sh` Executes (Step-by-Step Breakdown)
+
+The automated script walks through 5 verification stages:
+
+### Step 1: System Readiness & Health Probe (`/q/health/ready`)
+- **What it does**: Issues a SmallRye Health check against the Quarkus registry runtime.
+- **What you are showing**: Sourcelume actively monitors its connection to the Apache Atlas backend. If Atlas is reachable, the registry reports `UP` and is ready for traffic.
+```bash
+curl -s http://localhost:8082/q/health/ready | jq .
+```
+
+### Step 2: Automatic Atlas Type Bootstrapping (`/api/v1/types/typedefs`)
+- **What it does**: Queries the registry to inspect active entity type definitions in Apache Atlas.
+- **What you are showing**: Zero manual Atlas configuration required. On startup, Sourcelume automatically registered the `sourcelume_dataset` type (derived from `sourcelume_model.json`), extending Atlas's native `DataSet` type hierarchy with attributes like `sourceUri` and `licenseId`.
+```bash
+# Inspect all registered typedefs
+curl -s http://localhost:8082/api/v1/types/typedefs | jq .
+
+# Inspect the sourcelume_dataset entity type definition
+curl -s http://localhost:8082/api/v1/types/dataset | jq .
+```
+
+### Step 3: Dataset Registration & Ingestion (`POST /api/v1/datasets`)
+- **What it does**: Ingests `sample-dataset.json` (representing the Glasgow Hackathon dataset) into the registry.
+- **What you are showing**:
+ - The registry validates input fields against Jakarta Validation rules and spec definitions (`qualifiedName`, `licenseId`, `sourceUri`).
+ - It creates the entity inside Apache Atlas and returns a newly minted Atlas internal `guid` alongside its domain URI.
+```bash
+curl -s -X POST http://localhost:8082/api/v1/datasets \
+ -H "Content-Type: application/json" \
+ -d @sample-dataset.json | jq .
+```
+
+### Step 4: Domain-Key Query (`GET /api/v1/datasets?qualifiedName=...`)
+- **What it does**: Queries the dataset using its canonical URI identifier (`sourcelume://datasets/glasgow-hackathon-corpus-v1`).
+- **What you are showing**: Upstream ML pipelines, curators, and compliance tools can retrieve dataset metadata using meaningful domain URIs without needing to know internal database identifiers.
+```bash
+curl -s "http://localhost:8082/api/v1/datasets?qualifiedName=sourcelume://datasets/glasgow-hackathon-corpus-v1" | jq .
+```
+
+### Step 5: Backend-Key Query (`GET /api/v1/datasets/guid/...`)
+- **What it does**: Queries the dataset using its internal Atlas GUID (`ASSIGNED_GUID`).
+- **What you are showing**: Downstream governance tools and Atlas-native services have direct access to entities via their persistent graph GUIDs.
+```bash
+curl -s "http://localhost:8082/api/v1/datasets/guid/<ASSIGNED_GUID>" | jq .
+```
+
+---
+
+## Exploring in the Apache Atlas Web UI
+
+To conclude the demo with visual verification:
+
+1. Open your browser to [http://localhost:21000](http://localhost:21000).
+2. Log in with:
+ - **Username**: `admin`
+ - **Password**: `atlasR0cks!`
+3. In the search box, select **Type**: `sourcelume_dataset`.
+4. Click **Search** to view the `Glasgow Hackathon Instruction Corpus v1` entity, its indexed attributes (`sourceUri`, `licenseId`), and audit history.
+
+---
+
+## Companion Artifact: The Spec Provenance Record
+
+The `sample-provenance-record.jsonld` file illustrates the broader **Sourcelume Spec (v0.0.1)** model and linked-data format:
+
+- **Custody Chain (`custodyChain`)**: Tracking who touched the dataset (`agent`), what operation was performed (`action`), and when (`startTime`).
+- **License Governance (`licenseHistory`)**: Tracking SPDX license attribution (`Apache-2.0`) and compatibility verification.
+- **JSON-LD Interoperability**: Demonstrating compliance with standard W3C linked data formats.
+
+```json
+{
+ "@context": "https://sourcelume.apache.org/context/0.0.1/sourcelume.jsonld",
+ "id": "https://sourcelume.apache.org/records/rec-glasgow-hackathon-001",
+ "type": "ProvenanceRecord",
+ "identifier": "sourcelume://datasets/glasgow-hackathon-corpus-v1",
+ "name": "Glasgow Hackathon Instruction Corpus v1",
+ "version": "1.0.0",
+ "license": "https://spdx.org/licenses/Apache-2.0.html",
+ "licenseScope": "dataset",
+ "licenseCategory": "permissive",
+ "licenseNote": "SPDX compliant Apache-2.0 dataset metadata verification",
+ "licenseHistory": [
+ {
+ "license": "https://spdx.org/licenses/Apache-2.0.html",
+ "event": "initial_curation",
+ "date": "2026-10-11T09:00:00Z"
+ }
+ ],
+ "creator": [
+ {
+ "type": "schema:Organization",
+ "name": "Community Over Code Glasgow 2026 Hackathon Team",
+ "role": "originator"
+ }
+ ],
+ "created": "2026-10-11T09:00:00Z",
+ "added": "2026-10-11T10:00:00Z",
+ "contentCreated": "2026-10-11T08:30:00Z",
+ "origin": "Curated multi-turn instruction data assembled during Community Over Code Glasgow 2026 Hackathon.",
+ "custodyChain": [
+ {
+ "agent": "https://sourcelume.apache.org/agents/glasgow-curator-01",
+ "action": "dataset_ingestion_and_validation",
+ "startTime": "2026-10-11T10:00:00Z"
+ }
+ ]
+}
+```
+
+---
+
+## Teardown
+
+To shut down local services after completing the demo:
+
+```bash
+# Stop Quarkus (Ctrl+C in terminal)
+
+# Stop Atlas backend
+cd dev-support
+docker compose -f docker-compose.atlas.yml down
+```
diff --git a/examples/simple-demo/run-demo.sh b/examples/simple-demo/run-demo.sh
new file mode 100755
index 0000000..0a10331
--- /dev/null
+++ b/examples/simple-demo/run-demo.sh
@@ -0,0 +1,174 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# ==============================================================================
+# Apache Sourcelume Simple Demo
+# Community Over Code Glasgow 2026 Hackathon
+# ==============================================================================
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+DATASET_FILE="${SCRIPT_DIR}/sample-dataset.json"
+PROVENANCE_FILE="${SCRIPT_DIR}/sample-provenance-record.jsonld"
+
+REGISTRY_URL="${SOURCELUME_REGISTRY_URL:-http://localhost:8082}"
+ATLAS_URL="${SOURCELUME_ATLAS_URL:-http://localhost:21000}"
+
+# ANSI formatting
+if [ -t 1 ]; then
+ BOLD="\033[1m"
+ GREEN="\033[0;32m"
+ BLUE="\033[0;34m"
+ YELLOW="\033[1;33m"
+ RED="\033[0;31m"
+ CYAN="\033[0;36m"
+ MAGENTA="\033[0;35m"
+ RESET="\033[0m"
+else
+ BOLD=""
+ GREEN=""
+ BLUE=""
+ YELLOW=""
+ RED=""
+ CYAN=""
+ MAGENTA=""
+ RESET=""
+fi
+
+# Sourcelume ASCII Logo Banner
+echo -e "${CYAN}"
+cat << "EOF"
+ |
+ _____ __ --+--
+ / ___/____ __ __________________ / /_ ______ ___ ___ |
+ \__ \/ __ \/ / / / ___/ ___/ _ \/ / / / / / / __ `__ \/ _ \
+ ___/ / /_/ / /_/ / / / /__/ __/ / / / /_/ / / / / / / __/
+/____/\____/\__,_/_/ \___/\___/_/ /_/\__,_/_/ /_/ /_/\___/
+EOF
+echo -e "${RESET}"
+
+echo -e "${BOLD}${CYAN}================================================================================${RESET}"
+echo -e "${BOLD}${CYAN} Apache Sourcelume Registry — End-to-End Demo ${RESET}"
+echo -e "${CYAN} Community Over Code Glasgow 2026 Hackathon ${RESET}"
+echo -e "${BOLD}${CYAN}================================================================================${RESET}"
+echo -e "${MAGENTA}Demo Story & Objectives:${RESET}"
+echo -e " 1. ${BOLD}The Problem:${RESET} AI models often train on data with unverified provenance/licensing."
+echo -e " 2. ${BOLD}The Solution:${RESET} Sourcelume Spec (provenance schemas) + Sourcelume Registry (Atlas)."
+echo -e " 3. ${BOLD}This Demo:${RESET} Bootstrap typedefs, ingest dataset, validate, and query entities."
+echo -e "${BOLD}${CYAN}--------------------------------------------------------------------------------${RESET}\n"
+
+# Verify required command line tools
+command -v curl >/dev/null 2>&1 || { echo -e "${RED}Error: curl is required but not installed.${RESET}" >&2; exit 1; }
+command -v jq >/dev/null 2>&1 || { echo -e "${RED}Error: jq is required but not installed.${RESET}" >&2; exit 1; }
+
+if [ ! -f "${DATASET_FILE}" ]; then
+ echo -e "${RED}Error: Dataset payload file not found at ${DATASET_FILE}${RESET}" >&2
+ exit 1
+fi
+
+# Step 1: Health Check
+echo -e "${BOLD}${BLUE}[Step 1/5] System Readiness & Health Probe (${REGISTRY_URL}/q/health/ready)${RESET}"
+echo -e " ${YELLOW}Showing:${RESET} Active health probe confirming registry connectivity to Apache Atlas."
+HEALTH_JSON=$(curl -s -f "${REGISTRY_URL}/q/health/ready" 2>/dev/null || true)
+
+if [ -z "${HEALTH_JSON}" ]; then
+ echo -e "\n${RED}✗ Failed to connect to Sourcelume Registry at ${REGISTRY_URL}.${RESET}"
+ echo -e "${YELLOW}Please ensure Apache Atlas and Sourcelume Registry are running:${RESET}"
+ echo -e " 1. Start Atlas: cd dev-support && docker compose -f docker-compose.atlas.yml up -d"
+ echo -e " 2. Start Registry: mvn -pl sourcelume-registry-runtime-quarkus quarkus:dev"
+ exit 1
+fi
+
+HEALTH_STATUS=$(echo "${HEALTH_JSON}" | jq -r '.status // "DOWN"')
+if [ "${HEALTH_STATUS}" != "UP" ]; then
+ echo -e "\n${RED}✗ Registry is running but reporting health status: ${HEALTH_STATUS}${RESET}"
+ echo "${HEALTH_JSON}" | jq .
+ exit 1
+fi
+echo -e "${GREEN}✓ Registry status is UP (Connected to Apache Atlas backend).${RESET}\n"
+
+# Step 2: Inspect Atlas Typedefs
+echo -e "${BOLD}${BLUE}[Step 2/5] Automatic Atlas Type Bootstrapping (/api/v1/types/typedefs)${RESET}"
+echo -e " ${YELLOW}Showing:${RESET} Zero-configuration initialization — Sourcelume bootstraps custom entity"
+echo -e " types ('sourcelume_dataset') into Atlas extending core DataSet attributes."
+TYPEDEFS=$(curl -s "${REGISTRY_URL}/api/v1/types/typedefs")
+ENTITY_DEFS_COUNT=$(echo "${TYPEDEFS}" | jq '.entityDefs | length')
+echo -e " • Total Entity Definitions loaded: ${BOLD}${ENTITY_DEFS_COUNT}${RESET}"
+
+DATASET_TYPE=$(curl -s "${REGISTRY_URL}/api/v1/types/dataset")
+DATASET_TYPENAME=$(echo "${DATASET_TYPE}" | jq -r '.name // empty')
+SUPER_TYPES=$(echo "${DATASET_TYPE}" | jq -c -r '.superTypes // []')
+echo -e " • Verified Entity Type: ${GREEN}${DATASET_TYPENAME}${RESET} (SuperTypes: ${SUPER_TYPES})"
+echo -e "${GREEN}✓ Type system successfully bootstrapped in Apache Atlas.${RESET}\n"
+
+# Step 3: Register Dataset
+echo -e "${BOLD}${BLUE}[Step 3/5] Dataset Registration & Validation (POST /api/v1/datasets)${RESET}"
+echo -e " ${YELLOW}Showing:${RESET} Ingesting dataset metadata with Jakarta Validation and Atlas persistence."
+echo -e "${CYAN}Ingestion Payload (${DATASET_FILE}):${RESET}"
+cat "${DATASET_FILE}" | jq .
+
+CREATE_RESP=$(curl -s -X POST "${REGISTRY_URL}/api/v1/datasets" \
+ -H "Content-Type: application/json" \
+ -d @"${DATASET_FILE}")
+
+ASSIGNED_GUID=$(echo "${CREATE_RESP}" | jq -r '.guid // empty')
+QUALIFIED_NAME=$(echo "${CREATE_RESP}" | jq -r '.qualifiedName // empty')
+
+if [ -z "${QUALIFIED_NAME}" ]; then
+ echo -e "\n${RED}✗ Failed to register dataset. Response:${RESET}"
+ echo "${CREATE_RESP}" | jq .
+ exit 1
+fi
+
+echo -e "\n${GREEN}✓ Dataset registered and indexed in Apache Atlas!${RESET}"
+echo -e " • Canonical Qualified Name: ${BOLD}${QUALIFIED_NAME}${RESET}"
+echo -e " • Assigned Atlas Graph GUID: ${BOLD}${ASSIGNED_GUID}${RESET}\n"
+
+# Step 4: Query Dataset by qualifiedName
+echo -e "${BOLD}${BLUE}[Step 4/5] Domain-Key Query (/api/v1/datasets?qualifiedName=...)${RESET}"
+echo -e " ${YELLOW}Showing:${RESET} Upstream pipelines can retrieve metadata using canonical dataset URIs."
+FETCH_BY_QN=$(curl -s "${REGISTRY_URL}/api/v1/datasets?qualifiedName=${QUALIFIED_NAME}")
+echo "${FETCH_BY_QN}" | jq .
+echo -e "${GREEN}✓ Domain-key resolution verified successfully.${RESET}\n"
+
+# Step 5: Query Dataset by GUID
+if [ -n "${ASSIGNED_GUID}" ]; then
+ echo -e "${BOLD}${BLUE}[Step 5/5] Backend-Key Query (/api/v1/datasets/guid/...)${RESET}"
+ echo -e " ${YELLOW}Showing:${RESET} Downstream governance tools access entities directly via Atlas GUIDs."
+ FETCH_BY_GUID=$(curl -s "${REGISTRY_URL}/api/v1/datasets/guid/${ASSIGNED_GUID}")
+ echo "${FETCH_BY_GUID}" | jq .
+ echo -e "${GREEN}✓ Internal GUID resolution verified successfully.${RESET}\n"
+fi
+
+# Summary
+echo -e "${BOLD}${CYAN}================================================================================${RESET}"
+echo -e "${BOLD}${GREEN} Demo Execution Complete! ${RESET}"
+echo -e "${BOLD}${CYAN}================================================================================${RESET}"
+echo -e "${BOLD}Next Steps for Visual Exploration:${RESET}"
+echo -e " 1. Open the Apache Atlas Web UI: ${BOLD}${ATLAS_URL}${RESET}"
+echo -e " 2. Log in with credentials: ${BOLD}admin${RESET} / ${BOLD}atlasR0cks!${RESET}"
+echo -e " 3. Search for Type: ${BOLD}sourcelume_dataset${RESET}"
+echo -e " 4. Inspect the entity attributes (sourceUri, licenseId) and audit history."
+
+if [ -f "${PROVENANCE_FILE}" ]; then
+ echo -e "\n${BOLD}Companion Spec Artifact:${RESET}"
+ echo -e " Review ${BOLD}${PROVENANCE_FILE}${RESET} to see the complete JSON-LD"
+ echo -e " provenance model including custodyChain and SPDX licenseHistory."
+fi
+echo -e "\n${GREEN}Thank you for exploring Apache Sourcelume!${RESET}"
diff --git a/examples/simple-demo/sample-dataset.json b/examples/simple-demo/sample-dataset.json
new file mode 100644
index 0000000..d13e715
--- /dev/null
+++ b/examples/simple-demo/sample-dataset.json
@@ -0,0 +1,7 @@
+{
+ "qualifiedName": "sourcelume://datasets/glasgow-hackathon-corpus-v1",
+ "name": "Glasgow Hackathon Instruction Corpus v1",
+ "description": "Curated instruction tuning dataset created during Community Over Code Glasgow 2026 Hackathon with verified SPDX license provenance.",
+ "sourceUri": "https://huggingface.co/datasets/apache/glasgow-hackathon-corpus-v1",
+ "licenseId": "Apache-2.0"
+}
diff --git a/examples/simple-demo/sample-provenance-record.jsonld b/examples/simple-demo/sample-provenance-record.jsonld
new file mode 100644
index 0000000..fbc2142
--- /dev/null
+++ b/examples/simple-demo/sample-provenance-record.jsonld
@@ -0,0 +1,37 @@
+{
+ "@context": "https://sourcelume.apache.org/context/0.0.1/sourcelume.jsonld",
+ "id": "https://sourcelume.apache.org/records/rec-glasgow-hackathon-001",
+ "type": "ProvenanceRecord",
+ "identifier": "sourcelume://datasets/glasgow-hackathon-corpus-v1",
+ "name": "Glasgow Hackathon Instruction Corpus v1",
+ "version": "1.0.0",
+ "license": "https://spdx.org/licenses/Apache-2.0.html",
+ "licenseScope": "dataset",
+ "licenseCategory": "permissive",
+ "licenseNote": "SPDX compliant Apache-2.0 dataset metadata verification",
+ "licenseHistory": [
+ {
+ "license": "https://spdx.org/licenses/Apache-2.0.html",
+ "event": "initial_curation",
+ "date": "2026-10-11T09:00:00Z"
+ }
+ ],
+ "creator": [
+ {
+ "type": "schema:Organization",
+ "name": "Community Over Code Glasgow 2026 Hackathon Team",
+ "role": "originator"
+ }
+ ],
+ "created": "2026-10-11T09:00:00Z",
+ "added": "2026-10-11T10:00:00Z",
+ "contentCreated": "2026-10-11T08:30:00Z",
+ "origin": "Curated multi-turn instruction data assembled during Community Over Code Glasgow 2026 Hackathon.",
+ "custodyChain": [
+ {
+ "agent": "https://sourcelume.apache.org/agents/glasgow-curator-01",
+ "action": "dataset_ingestion_and_validation",
+ "startTime": "2026-10-11T10:00:00Z"
+ }
+ ]
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@sourcelume.apache.org
For additional commands, e-mail: commits-help@sourcelume.apache.org
(sourcelume-registry) 02/02: Updates to demo
Posted by jg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jgoodyear pushed a commit to branch coc-2026
in repository https://gitbox.apache.org/repos/asf/sourcelume-registry.git
commit e195fc2a8315a17082b321f3f318745922e3534b
Author: jgoodyear <jg...@apache.org>
AuthorDate: Sun Oct 11 14:14:59 2026 -0230
Updates to demo
---
examples/simple-demo/README.md | 100 +++++++++++++++++++++++++++++++++++++
examples/simple-demo/run-demo.sh | 105 +++++++++++++++++++++++++++++++++++++--
2 files changed, 200 insertions(+), 5 deletions(-)
diff --git a/examples/simple-demo/README.md b/examples/simple-demo/README.md
index 8b1f786..b7f9560 100644
--- a/examples/simple-demo/README.md
+++ b/examples/simple-demo/README.md
@@ -80,6 +80,24 @@ The automated script walks through 5 verification stages:
### Step 1: System Readiness & Health Probe (`/q/health/ready`)
- **What it does**: Issues a SmallRye Health check against the Quarkus registry runtime.
- **What you are showing**: Sourcelume actively monitors its connection to the Apache Atlas backend. If Atlas is reachable, the registry reports `UP` and is ready for traffic.
+
+```text
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | Client / CI | | Sourcelume Registry (Quarkus) | | Apache Atlas |
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | | |
+ | 1. GET /q/health/ready | |
+ | ---------------------------------------> | |
+ | | 2. Probe Atlas connectivity / status |
+ | | -----------------------------------------> |
+ | | |
+ | | 3. Connection OK (HTTP 200 / UP) |
+ | | <----------------------------------------- |
+ | 4. Health check JSON (status: UP) | |
+ | <--------------------------------------- | |
+ | | |
+```
+
```bash
curl -s http://localhost:8082/q/health/ready | jq .
```
@@ -87,6 +105,30 @@ curl -s http://localhost:8082/q/health/ready | jq .
### Step 2: Automatic Atlas Type Bootstrapping (`/api/v1/types/typedefs`)
- **What it does**: Queries the registry to inspect active entity type definitions in Apache Atlas.
- **What you are showing**: Zero manual Atlas configuration required. On startup, Sourcelume automatically registered the `sourcelume_dataset` type (derived from `sourcelume_model.json`), extending Atlas's native `DataSet` type hierarchy with attributes like `sourceUri` and `licenseId`.
+
+```text
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | Client / CI | | Sourcelume Registry (Quarkus) | | Apache Atlas |
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | | |
+ | | [On Startup: Auto-Bootstrap Typedefs] |
+ | | 1. Read model (sourcelume_model.json) |
+ | | 2. POST /api/atlas/v2/types/typedefs |
+ | | -----------------------------------------> |
+ | | Register sourcelume_dataset type |
+ | | |
+ | 3. GET /api/v1/types/typedefs | |
+ | ---------------------------------------> | |
+ | | 4. Fetch registered type definitions |
+ | | -----------------------------------------> |
+ | | |
+ | | 5. Return Atlas entity & struct defs |
+ | | <----------------------------------------- |
+ | 6. Return active Typedef JSON | |
+ | <--------------------------------------- | |
+ | | |
+```
+
```bash
# Inspect all registered typedefs
curl -s http://localhost:8082/api/v1/types/typedefs | jq .
@@ -100,6 +142,27 @@ curl -s http://localhost:8082/api/v1/types/dataset | jq .
- **What you are showing**:
- The registry validates input fields against Jakarta Validation rules and spec definitions (`qualifiedName`, `licenseId`, `sourceUri`).
- It creates the entity inside Apache Atlas and returns a newly minted Atlas internal `guid` alongside its domain URI.
+
+```text
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | Client / CI | | Sourcelume Registry (Quarkus) | | Apache Atlas |
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | | |
+ | 1. POST /api/v1/datasets | |
+ | (Payload: qualifiedName, etc.) | |
+ | ---------------------------------------> | |
+ | | 2. Validate DTO (Jakarta Validation) |
+ | | 3. Map DTO -> sourcelume_dataset Entity |
+ | | -----------------------------------------> |
+ | | Persist entity into Atlas graph |
+ | | |
+ | | 4. Assigned Graph GUID & Indexed |
+ | | <----------------------------------------- |
+ | 5. Return DTO with assigned GUID | |
+ | <--------------------------------------- | |
+ | | |
+```
+
```bash
curl -s -X POST http://localhost:8082/api/v1/datasets \
-H "Content-Type: application/json" \
@@ -109,6 +172,25 @@ curl -s -X POST http://localhost:8082/api/v1/datasets \
### Step 4: Domain-Key Query (`GET /api/v1/datasets?qualifiedName=...`)
- **What it does**: Queries the dataset using its canonical URI identifier (`sourcelume://datasets/glasgow-hackathon-corpus-v1`).
- **What you are showing**: Upstream ML pipelines, curators, and compliance tools can retrieve dataset metadata using meaningful domain URIs without needing to know internal database identifiers.
+
+```text
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | ML Pipeline / Tool | | Sourcelume Registry (Quarkus) | | Apache Atlas |
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | | |
+ | 1. GET /api/v1/datasets | |
+ | ?qualifiedName=sourcelume://... | |
+ | ---------------------------------------> | |
+ | | 2. Query Atlas by unique qualifiedName |
+ | | -----------------------------------------> |
+ | | |
+ | | 3. Return matched Entity data |
+ | | <----------------------------------------- |
+ | 4. Return JSON dataset attributes | |
+ | <--------------------------------------- | |
+ | | |
+```
+
```bash
curl -s "http://localhost:8082/api/v1/datasets?qualifiedName=sourcelume://datasets/glasgow-hackathon-corpus-v1" | jq .
```
@@ -116,6 +198,24 @@ curl -s "http://localhost:8082/api/v1/datasets?qualifiedName=sourcelume://datase
### Step 5: Backend-Key Query (`GET /api/v1/datasets/guid/...`)
- **What it does**: Queries the dataset using its internal Atlas GUID (`ASSIGNED_GUID`).
- **What you are showing**: Downstream governance tools and Atlas-native services have direct access to entities via their persistent graph GUIDs.
+
+```text
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | Governance / Audit | | Sourcelume Registry (Quarkus) | | Apache Atlas |
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | | |
+ | 1. GET /api/v1/datasets/guid/{guid} | |
+ | ---------------------------------------> | |
+ | | 2. Direct Graph node lookup by Atlas GUID |
+ | | -----------------------------------------> |
+ | | |
+ | | 3. Return Entity graph node attributes |
+ | | <----------------------------------------- |
+ | 4. Return JSON dataset attributes | |
+ | <--------------------------------------- | |
+ | | |
+```
+
```bash
curl -s "http://localhost:8082/api/v1/datasets/guid/<ASSIGNED_GUID>" | jq .
```
diff --git a/examples/simple-demo/run-demo.sh b/examples/simple-demo/run-demo.sh
index 0a10331..c1be384 100755
--- a/examples/simple-demo/run-demo.sh
+++ b/examples/simple-demo/run-demo.sh
@@ -84,7 +84,24 @@ fi
# Step 1: Health Check
echo -e "${BOLD}${BLUE}[Step 1/5] System Readiness & Health Probe (${REGISTRY_URL}/q/health/ready)${RESET}"
-echo -e " ${YELLOW}Showing:${RESET} Active health probe confirming registry connectivity to Apache Atlas."
+echo -e " ${YELLOW}Showing:${RESET} Active health probe confirming registry connectivity to Apache Atlas.\n"
+cat << "EOF"
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | Client / CI | | Sourcelume Registry (Quarkus) | | Apache Atlas |
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | | |
+ | 1. GET /q/health/ready | |
+ | ---------------------------------------> | |
+ | | 2. Probe Atlas connectivity / status |
+ | | -----------------------------------------> |
+ | | |
+ | | 3. Connection OK (HTTP 200 / UP) |
+ | | <----------------------------------------- |
+ | 4. Health check JSON (status: UP) | |
+ | <--------------------------------------- | |
+ | | |
+EOF
+echo ""
HEALTH_JSON=$(curl -s -f "${REGISTRY_URL}/q/health/ready" 2>/dev/null || true)
if [ -z "${HEALTH_JSON}" ]; then
@@ -106,7 +123,30 @@ echo -e "${GREEN}✓ Registry status is UP (Connected to Apache Atlas backend).$
# Step 2: Inspect Atlas Typedefs
echo -e "${BOLD}${BLUE}[Step 2/5] Automatic Atlas Type Bootstrapping (/api/v1/types/typedefs)${RESET}"
echo -e " ${YELLOW}Showing:${RESET} Zero-configuration initialization — Sourcelume bootstraps custom entity"
-echo -e " types ('sourcelume_dataset') into Atlas extending core DataSet attributes."
+echo -e " types ('sourcelume_dataset') into Atlas extending core DataSet attributes.\n"
+cat << "EOF"
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | Client / CI | | Sourcelume Registry (Quarkus) | | Apache Atlas |
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | | |
+ | | [On Startup: Auto-Bootstrap Typedefs] |
+ | | 1. Read model (sourcelume_model.json) |
+ | | 2. POST /api/atlas/v2/types/typedefs |
+ | | -----------------------------------------> |
+ | | Register sourcelume_dataset type |
+ | | |
+ | 3. GET /api/v1/types/typedefs | |
+ | ---------------------------------------> | |
+ | | 4. Fetch registered type definitions |
+ | | -----------------------------------------> |
+ | | |
+ | | 5. Return Atlas entity & struct defs |
+ | | <----------------------------------------- |
+ | 6. Return active Typedef JSON | |
+ | <--------------------------------------- | |
+ | | |
+EOF
+echo ""
TYPEDEFS=$(curl -s "${REGISTRY_URL}/api/v1/types/typedefs")
ENTITY_DEFS_COUNT=$(echo "${TYPEDEFS}" | jq '.entityDefs | length')
echo -e " • Total Entity Definitions loaded: ${BOLD}${ENTITY_DEFS_COUNT}${RESET}"
@@ -119,7 +159,27 @@ echo -e "${GREEN}✓ Type system successfully bootstrapped in Apache Atlas.${RES
# Step 3: Register Dataset
echo -e "${BOLD}${BLUE}[Step 3/5] Dataset Registration & Validation (POST /api/v1/datasets)${RESET}"
-echo -e " ${YELLOW}Showing:${RESET} Ingesting dataset metadata with Jakarta Validation and Atlas persistence."
+echo -e " ${YELLOW}Showing:${RESET} Ingesting dataset metadata with Jakarta Validation and Atlas persistence.\n"
+cat << "EOF"
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | Client / CI | | Sourcelume Registry (Quarkus) | | Apache Atlas |
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | | |
+ | 1. POST /api/v1/datasets | |
+ | (Payload: qualifiedName, etc.) | |
+ | ---------------------------------------> | |
+ | | 2. Validate DTO (Jakarta Validation) |
+ | | 3. Map DTO -> sourcelume_dataset Entity |
+ | | -----------------------------------------> |
+ | | Persist entity into Atlas graph |
+ | | |
+ | | 4. Assigned Graph GUID & Indexed |
+ | | <----------------------------------------- |
+ | 5. Return DTO with assigned GUID | |
+ | <--------------------------------------- | |
+ | | |
+EOF
+echo ""
echo -e "${CYAN}Ingestion Payload (${DATASET_FILE}):${RESET}"
cat "${DATASET_FILE}" | jq .
@@ -142,7 +202,25 @@ echo -e " • Assigned Atlas Graph GUID: ${BOLD}${ASSIGNED_GUID}${RESET}\n"
# Step 4: Query Dataset by qualifiedName
echo -e "${BOLD}${BLUE}[Step 4/5] Domain-Key Query (/api/v1/datasets?qualifiedName=...)${RESET}"
-echo -e " ${YELLOW}Showing:${RESET} Upstream pipelines can retrieve metadata using canonical dataset URIs."
+echo -e " ${YELLOW}Showing:${RESET} Upstream pipelines can retrieve metadata using canonical dataset URIs.\n"
+cat << "EOF"
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | ML Pipeline / Tool | | Sourcelume Registry (Quarkus) | | Apache Atlas |
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | | |
+ | 1. GET /api/v1/datasets | |
+ | ?qualifiedName=sourcelume://... | |
+ | ---------------------------------------> | |
+ | | 2. Query Atlas by unique qualifiedName |
+ | | -----------------------------------------> |
+ | | |
+ | | 3. Return matched Entity data |
+ | | <----------------------------------------- |
+ | 4. Return JSON dataset attributes | |
+ | <--------------------------------------- | |
+ | | |
+EOF
+echo ""
FETCH_BY_QN=$(curl -s "${REGISTRY_URL}/api/v1/datasets?qualifiedName=${QUALIFIED_NAME}")
echo "${FETCH_BY_QN}" | jq .
echo -e "${GREEN}✓ Domain-key resolution verified successfully.${RESET}\n"
@@ -150,7 +228,24 @@ echo -e "${GREEN}✓ Domain-key resolution verified successfully.${RESET}\n"
# Step 5: Query Dataset by GUID
if [ -n "${ASSIGNED_GUID}" ]; then
echo -e "${BOLD}${BLUE}[Step 5/5] Backend-Key Query (/api/v1/datasets/guid/...)${RESET}"
- echo -e " ${YELLOW}Showing:${RESET} Downstream governance tools access entities directly via Atlas GUIDs."
+ echo -e " ${YELLOW}Showing:${RESET} Downstream governance tools access entities directly via Atlas GUIDs.\n"
+ cat << "EOF"
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | Governance / Audit | | Sourcelume Registry (Quarkus) | | Apache Atlas |
+ +----------------------+ +----------------------------------+ +-------------------------+
+ | | |
+ | 1. GET /api/v1/datasets/guid/{guid} | |
+ | ---------------------------------------> | |
+ | | 2. Direct Graph node lookup by Atlas GUID |
+ | | -----------------------------------------> |
+ | | |
+ | | 3. Return Entity graph node attributes |
+ | | <----------------------------------------- |
+ | 4. Return JSON dataset attributes | |
+ | <--------------------------------------- | |
+ | | |
+EOF
+ echo ""
FETCH_BY_GUID=$(curl -s "${REGISTRY_URL}/api/v1/datasets/guid/${ASSIGNED_GUID}")
echo "${FETCH_BY_GUID}" | jq .
echo -e "${GREEN}✓ Internal GUID resolution verified successfully.${RESET}\n"
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@sourcelume.apache.org
For additional commands, e-mail: commits-help@sourcelume.apache.org