You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2026/09/11 06:11:51 UTC
(skywalking-ai-sessionizer) branch main updated: Retire the design notes into the docs, and drop the pre-alpha status (#5)
This is an automated email from the ASF dual-hosted git repository.
wu-sheng pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/skywalking-ai-sessionizer.git
The following commit(s) were added to refs/heads/main by this push:
new 526ae60 Retire the design notes into the docs, and drop the pre-alpha status (#5)
526ae60 is described below
commit 526ae60497484fe7e71f51f3bab2b0f2c9e640c8
Author: 吴晟 Wu Sheng <wu...@foxmail.com>
AuthorDate: Fri Sep 11 14:11:46 2026 +0800
Retire the design notes into the docs, and drop the pre-alpha status (#5)
* Retire the design notes into the docs, and drop the pre-alpha status
The three design notes were written while collection, assembly and the
read side were being designed. They had become older than the code, so
they were a second source of truth that disagreed with it.
Each note was read section by section against the code at this commit.
What has landed is now in docs/en, with each measurement and the sample
it was taken on. Where the code differs from a note, the docs describe
the code. Plans, open questions and history stay in the git history.
The Claude Code glossary page had drifted from what asz glossary prints.
It is rebuilt from that output. Its declarations keep the runtime's name
for the two step kinds and the relation asz does not read yet, with a
note saying so, and drop three duplicate entries that NewGlossary hid.
The README and the overview no longer call the project pre-alpha, ahead
of 0.3.0. CLAUDE.md and the contributing guide say that a feature is
documented in docs/en when it lands, and that no notes folder is kept.
* Drop the statements that no design-notes folder exists
A page that says a folder is absent tells a reader nothing they need.
CLAUDE.md loses the paragraph, and the contributing guide keeps only its
sentence about docs/.
---
.dockerignore | 1 -
CLAUDE.md | 3 +-
README.md | 10 +-
design-notes/01-collection-and-raw-storage.md | 1324 --------------------
design-notes/02-parse-and-assembly.md | 623 ---------
design-notes/03-read-and-export.md | 626 ---------
design-notes/README.md | 10 -
docs/en/adapters/claude-code-glossary.md | 36 +-
docs/en/adapters/claude-code.md | 734 +++++++++--
docs/en/changes/changes-0.3.0.md | 6 +
.../concepts-and-designs/conversation-assembly.md | 233 +++-
docs/en/concepts-and-designs/glossary.md | 11 +-
docs/en/concepts-and-designs/overview.md | 24 +-
.../unified-conversation-model.md | 32 +
docs/en/formats/asz-view.md | 49 +-
docs/en/formats/session-data.md | 162 ++-
docs/en/formats/session-flow.md | 142 ++-
docs/en/formats/storage-root.md | 193 ++-
docs/en/guides/contributing.md | 3 +-
docs/en/setup/claude-code-plugin.md | 75 +-
docs/en/setup/command-line.md | 143 ++-
docs/en/setup/configuration.md | 41 +-
docs/en/setup/export-otlp.md | 33 +-
internal/adapters/claudecode/glossary.go | 40 +-
24 files changed, 1662 insertions(+), 2892 deletions(-)
diff --git a/.dockerignore b/.dockerignore
index 64455f1..c0d3587 100644
--- a/.dockerignore
+++ b/.dockerignore
@@ -7,6 +7,5 @@ data
.local
*.wal
docs
-design-notes
tests
*.md
diff --git a/CLAUDE.md b/CLAUDE.md
index 3b9e358..cc1543a 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -21,7 +21,6 @@ Apache-2.0. A subproject of Apache SkyWalking: `github.com/apache/skywalking-ai-
| `docs/en/formats/` | The three shapes: Session Data (`.sd`), Session Flow (`.sf`), and `asz.view` (one conversation as one document, never a file). asz defines and owns all three; a server that mirrors them, such as the OAP, follows these pages. |
| `docs/en/adapters/claude-code-glossary.md` | What Claude Code calls each of them. Generated by `asz glossary`. |
| `docs/en/adapters/claude-code.md` | How Claude Code maps onto the model, and what it cannot supply. |
-| `design-notes/` | **Working notes, not documentation.** Measurements, corrections, open questions. Unpolished by intent. |
Every claim in the docs is backed by a measurement against a real corpus. When changing them,
keep that standard: state what was measured, on what sample, and say `unavailable` rather than
@@ -98,7 +97,7 @@ network, a packaging change rather than a refactor.
**Phase 1 (implemented):** pull collection from local Claude Code files, plus a derived index built
while landing. No configuration of Claude Code required, and it works on history that already exists.
**Phase 2 (implemented):** assembly into a conversation, published as an append-only chain of
-immutable rounds — see `design-notes/02` and the assembly doc.
+immutable rounds — see the assembly doc.
**Phase 3 (in progress):** read and export. `asz collect` is the pipeline: every period it lands
what is new, parses what moved, and sends what `export.otlp` asks for, in that order. `asz view`
serves the conversations as a page and only reads: a list at `/`, and one conversation at `/c/{id}`
diff --git a/README.md b/README.md
index 0b21a34..a66a9f2 100644
--- a/README.md
+++ b/README.md
@@ -6,10 +6,6 @@
**Conversation-level observability, measurement, and export for long-lived AI agents.**
-> **Status:** pre-alpha. The conversation model and the Claude Code data mapping are defined and
-> evidence-backed. Collection and assembly are implemented. The local page that serves the
-> assembled conversations is in progress; static export and OTLP push are not started.
-
SkyWalking AI Sessionizer assembles fragmented agent telemetry into one durable conversation structure. It
preserves sessions as source provenance, keeps parent and child-agent execution lineages separate,
measures model-message continuity, and projects the same committed snapshot into storage, export and
@@ -60,7 +56,7 @@ adapter reports it as `unavailable` rather than approximating it.
| Runtime | Status | Collection |
| --- | --- | --- |
-| [Claude Code](docs/en/adapters/claude-code.md) | collection implemented | local files — no configuration required, and it works on history that already exists. The [plugin](docs/en/setup/claude-code-plugin.md) adds which files each shell command changed. |
+| [Claude Code](docs/en/adapters/claude-code.md) | collection implemented | local files — no configuration required, and it works on history that already exists. The [plugin](docs/en/setup/claude-code-plugin.md) adds which files each shell command changed. The [receiver](docs/en/setup/configuration.md#the-receiver-adapter), `claude-code-otlp`, is off by default. When Claude Code's own OpenTelemetry exporter is pointed at it, it lands the exporter's metrics requests. It drops the exporter [...]
| Codex | planned | — |
| LangChain / LangGraph | planned | — |
@@ -93,10 +89,6 @@ Official documentation lives in [`docs/`](docs/) and is indexed by
and the changelog. It is published at
[skywalking.apache.org/docs/skywalking-ai-sessionizer](https://skywalking.apache.org/docs/skywalking-ai-sessionizer/next/readme/).
-[`design-notes/`](design-notes/) holds working engineering notes — measurements, corrections and open
-questions produced while designing against real runtime data. They are deliberately unpolished and
-are **not** part of the published documentation.
-
## Contributing
Early contributions should focus on schemas, privacy-safe fixtures, deterministic assembly,
diff --git a/design-notes/01-collection-and-raw-storage.md b/design-notes/01-collection-and-raw-storage.md
deleted file mode 100644
index 0e1466c..0000000
--- a/design-notes/01-collection-and-raw-storage.md
+++ /dev/null
@@ -1,1324 +0,0 @@
-# Design Plan 01 — Claude Code Collection and Raw Storage
-
-**Status:** draft for discussion. Not implemented.
-**Scope:** how raw source data is acquired and stored, per session. The per-session processing
-pipeline is Plan 02; conversation output (bundle/export/preview) is Plan 03. Neither is settled
-here, and nothing in this document should be read as fixing them.
-
-**Collection phasing (decided):**
-
-| | Channel | Posture | Delivers |
-| --- | --- | --- | --- |
-| **Phase 1** | local logs — `~/.claude/projects/**` | pull (file tailing) | the whole conversation: structure, content, lineage, epochs. No configuration, and it works on history that already exists. |
-| **Phase 2** | OTLP | push (receiver) | measurement only: per-call `duration_ms` and `cost_usd`, tool durations, permission decisions. Requires user configuration and captures only from the moment it is enabled. |
-
-Phase 1 is self-sufficient: it is the product. Phase 2 is additive and changes nothing in the Phase 1
-layout — `log-` and `trace-` are already reserved prefixes (§2.2). Raw provider bodies
-(`OTEL_LOG_RAW_API_BODIES`) are a separate opt-in, orthogonal to both phases, and are the only route
-to a verified `input_manifest` (§1.8, §1.13).
-
-**Relationship to the earlier design draft (v0.8):** that draft was written when the project was
-still called Agent Sessionizer, and it is not part of this repository. v0.8 is the direction, not a
-specification.
-Where this plan contradicts it, the observed evidence wins and the contradiction is deliberate.
-
-**Evidence basis:** a live corpus of ~2,920 JSONL files (~1.2 GB) across 38–43 project directories —
-52–61 main transcripts, ~107 direct subagent transcripts, ~2,580 workflow-agent transcripts, 176
-workflow journals — plus the 2.1.245 binary, plus live runs. Every measured claim was produced by
-one agent and then re-measured by an independent adversarial verifier on a larger sample; where they
-disagreed, the verifier's number is used and the disagreement is noted.
-
-**Version coverage caveat (applies to everything below):** the corpus spans **only Claude Code
-2.1.220 – 2.1.251** (14 versions). No 2.0.x transcript exists on this machine. Every claim is
-therefore *untested* below 2.1.220 rather than verified stable.
-
-**Corpus liveness caveat:** the corpus grew during measurement (main transcripts went from 60 to 61
-inside one 20-minute run). Absolute counts are snapshots. A collector must tolerate files appearing
-and growing mid-scan.
-
----
-
-## 1. Evidence base — what Claude Code actually exposes
-
-### 1.1 File layout
-
-```
-~/.claude/projects/<slugified-cwd>/<session-id>.jsonl main stream
-~/.claude/projects/<slugified-cwd>/<session-id>/subagents/agent-<agent-id>.jsonl
-~/.claude/projects/<slugified-cwd>/<session-id>/subagents/agent-<agent-id>.meta.json
-~/.claude/projects/<slugified-cwd>/<session-id>/subagents/workflows/<wf-id>/agent-<agent-id>.jsonl
-~/.claude/projects/<slugified-cwd>/<session-id>/subagents/workflows/<wf-id>/journal.jsonl
-~/.claude/projects/<slugified-cwd>/<session-id>/workflows/wf_<run-id>.json run manifests
-~/.claude/projects/<slugified-cwd>/<session-id>/workflows/scripts/
-```
-
-Note workflow agents live under `subagents/workflows/`, **not** under `<session-id>/workflows/` —
-that sibling directory holds run manifests and scripts only.
-
-`agent-*.jsonl` ↔ `agent-*.meta.json` is a perfect 1:1 across all ~2,686 pairs, zero orphans in
-either direction.
-
-### 1.2 Record model — four corrections to the obvious reading
-
-**A line is not always one content block.** 156 records corpus-wide carry a multi-element
-`message.content` array on one line — 11 assistant and **145 user**, including 37 of shape
-`[image, text]` where `content[0]` is a base64 PNG. Reading `.message.content[0]` renders a base64
-image blob as the user's prompt, and silently drops `tool_use` blocks on the assistant side —
-breaking the tool join. **Always iterate the full array.** Not version-specific; observed in
-2.1.220, .221, .223, .227, .237.
-
-**Fragments of one response are not always consecutive.** A `user`/`tool_result` for an earlier
-parallel `tool_use` is written in between, and the `parentUuid` chain runs *through* it. Assembling
-by walking consecutive lines, or assuming `parentUuid` adjacency implies same-message, mis-assembles
-roughly 1,000 manifests.
-
-**`uuid` is not unique within a file.** 1,530 duplicated `(file, uuid)` pairs, confined to 10 main
-transcripts, **multiplicity exactly 2 — never 3**. Copies differ in `slug`, `promptId`, and
-sometimes `cwd`/`gitBranch`/`parentUuid`. The ceiling of 2 is the useful engineering fact: drop duplicates by
-`(file, uuid)` terminates.
-
-> These blocks were initially read as resume replays. They are not — see **§1.12b**, where they turn
-> out to be the corpus's strongest chain-external evidence. Duplicate removal still applies; the *interpretation*
-> changes.
-
-> **Duplicate rule:** keep the **first** occurrence for position, merging in fields the later copy adds
-> (e.g. `slug`). Keeping the last relocates up to 415 records from their true chronological position
-> to the resume point and scrambles line-order reconstruction. Timestamps are identical between
-> copies, so they cannot arbitrate.
->
-> *Contested:* one pass measured "later copy never richer", the verifier measured "later copy larger
-> in a third to half of pairs." Unresolved — see §1.9. The keep-first-and-merge rule is safe under
-> either reading.
-
-**`cwd` and `gitBranch` are per-record write-time stamps, not session constants.** One session
-showed 52 distinct `cwd` and 14 distinct `gitBranch` values. Only `sessionId`, `entrypoint` and
-`userType` are reliable per-file constants (`slug` varies across replay copies).
-
-### 1.3 Provider calls
-
-**Group by `(file, message.id)` — not by `requestId`.** `message.id` is present on 115,162/115,162
-assistant records; `requestId` is absent on 92 including genuine provider calls, and 2 `requestId`s
-map to two `message.id`s because a `<synthetic>` refusal companion reuses the real call's
-`requestId`. Grouping by `requestId` puts a locally fabricated block inside a real provider call.
-Neither key ever appears in more than one file.
-
-**Synthetic filter:** `.message.model == "<synthetic>"`, equivalently `.message.id` not matching
-`^msg_`. Both agree 97/97. Do **not** use `requestId == null` — 24 of 97 synthetics carry a non-null
-`requestId`, and 2 genuine calls have no `requestId` key at all.
-
-**Ordering is by line, never by timestamp.** Timestamps are monotone within a group, but 372 groups
-have tied timestamps, and at file level 2.76% of timestamped records are out of order even after
-duplicate records.
-
-**Usage: take the last fragment. Never sum, and do not assume fragments agree.**
-
-| | behaviour |
-| --- | --- |
-| main transcripts | identical final usage stamped on every fragment (14,167/14,171 groups) |
-| subagent transcripts | **streaming partials** — `output_tokens: 1, 2, 3 …` on all but the last |
-
-`output_tokens` is monotone non-decreasing in line order with last == max in **56,524/56,524**
-groups, zero counterexamples. Summing inflates by a mean of 2.14× (max 10×). This is the single most
-likely source of silently wrong numbers.
-
-**Terminal-fragment test:** the terminal fragment is identified by the presence of
-`{iterations, server_tool_use, speed}`. Prefer `speed` — `iterations` is JSON `null` on the 97
-synthetic records, so a `has("iterations")` test returns true for them.
-
-**7.14% of calls have no terminal record at all** (main 0.02%, subagent 13.70%) — usage and
-`stop_reason` are permanently unavailable for those. The rate tracks subagent workload composition,
-not version.
-
-**Fragment count distribution:** 1:18,318 · 2:19,918 · 3:16,666 · 4:1,386 · 5–12:236. Mean 2.04,
-max 12.
-
-**Useful streaming invariant:** after duplicate uuids are removed, no two provider calls' line ranges overlap
-(0 of 56,524). A group's fragments occupy a contiguous line span interrupted only by its own
-`tool_result` records. A streaming collector can close a ProviderCall the moment a line with a
-different `message.id` arrives.
-
-**Retry attempts are partially derivable.** `type:"system", subtype:"api_error"` records carry
-`retryAttempt`/`retryInMs`/`maxRetries`. 48 of 556 also carry `.error.requestId` — a real `req_…`
-id, present exactly on failures that got an HTTP response (e.g. 529), absent on transport failures.
-Those 48 are disjoint from all successful requestIds, so each failed attempt has its own id.
-`.error.status` is present on 50 of 556.
-
-**Content is complete in transcripts — with one exception.** User prompt text, assistant prose, tool
-inputs and tool results are all stored verbatim and untruncated. This is the reason transcripts, not
-OTLP events, are the v1 channel: events redact `user_prompt` / `assistant_response` by default,
-truncate content at ~60 KB, and omit tool inputs unless `OTEL_LOG_TOOL_DETAILS` is set.
-
-**The exception: thinking text is stripped.** The signature is retained, the reasoning is not:
-
-```json
-{"type":"thinking","thinking":"","signature":"CAISkwMKpgEIERgCKkBzgtqcryyQ3cbpBYwiBCwKlBijnqr7oxML…"}
-```
-
-Measured corpus-wide: **2,603 of 2,604 files containing thinking blocks have empty thinking text**
-(54 main + 2,549 subagent), across every version 2.1.220–2.1.251. Exactly one subagent file retains
-real text (31 blocks, 61–1,742 chars) — unexplained, and worth chasing.
-
-**Cause identified, and it is not a persistence choice.** A controlled comparison inside one session
-(`221ac729…`, all agents at 2.1.227, same agent type) isolates the model as the only variable:
-`claude-haiku-4-5-20251001` retained 31 thinking blocks with real text (61–1,742 chars); eleven
-sibling `claude-opus-5` agents retained none. A live capture then showed the mechanism — the request
-body carries
-
-```json
-"thinking": {"type":"adaptive","display":"omitted"}
-```
-
-Claude Code *asks the API not to return thinking text*. The signature comes back, the reasoning does
-not.
-
-**Verdict: thinking text is unrecoverable by any local mechanism.** Two independent redactions stack:
-the request suppresses it at the API, and the raw-body export redacts it again — a live capture on
-2.1.245 shows the response's thinking block as literally `"<REDACTED>"` (10 chars) beside a 596-char
-signature. OTLP events redact it unconditionally too. Treat reasoning content as `unavailable` in
-the coverage model and do not design around recovering it.
-
-Consequence: a Talk tree can show *that* the model reasoned, with duration and token cost, but not
-*what* it reasoned.
-
-**Injected context is captured.** The `attachment` record type — the third-largest in the corpus at
-~21,000 records, 32 distinct `attachment.type` values — carries what the harness pushed into the
-conversation alongside the human turns:
-
-```
-skill_listing deferred_tools_delta agent_listing_delta nested_memory
-mcp_instructions_delta command_permissions directory date_change
-task_reminder read_truncation_notice selected_lines_in_ide plan_mode_exit
-```
-
-So a reconstruction is not limited to human/assistant turns: skill catalogues, tool-availability
-deltas, memory injections, MCP instructions, IDE state and permission changes are all on disk in
-chain order.
-
-**But the transcript is provably not a superset of the model's context.** Three things are absent
-from every transcript — the **system prompt**, **tool schemas** (only names are recorded; 0 files
-contain `"tools":[`), and **thinking text** (§1.3, which matters extra because thinking blocks are
-re-sent in later requests and were therefore genuinely in context). And there is direct proof rather
-than inference: **`compactMetadata.preservedMessages.allUuids` contains 5 uuids that exist nowhere
-on disk** (§1.6) — records the runtime held in context and never wrote.
-
-**Consequence for `input_manifest.state`: `truncated`, never `full`, and never `unavailable`.**
-`message_refs[]` can be built by walking the chain, so a manifest exists; `system_block_refs[]` and
-`tool_definition_refs[]` cannot. Critically, the message membership so derived is **inferred, not
-observed** — nothing in the transcript proves those messages were the ones serialized into the
-request. Verifying the §2.2 continuity invariant requires raw bodies (§1.8); no amount of file
-reading substitutes.
-
-### 1.4 Tool joins — effectively perfect, once duplicates are removed
-
-Across 2,924 files / 356,583 records / 207,984 tool blocks:
-
-- Every `(file, tool_use_id)` maps to exactly **one** `tool_result` record once duplicates are removed and exactly
- **one** assistant `tool_use` record. 103,503 keys, all cardinality 1. Zero fan-out, zero orphans,
- zero cross-file leakage.
-- `sourceToolAssistantUUID` is present on **103,503/103,503** tool_result records, always equals
- `parentUuid`, and always resolves to the assistant fragment carrying the matching `tool_use`.
- Zero mismatches. It is fully redundant with `parentUuid`, so it adds no join power — but it is a
- free consistency check.
-- **Before duplicates are removed** the join is *exact_ambiguous* for 441 ids (882 of 64,722 tool_results, 1.4%),
- purely from replay copies. Removing duplicates is a precondition, not an optimisation.
-- `tool_use_id` is not only `toolu_` — 35 values are `srvtoolu_` (server-side tools such as web
- search). A `^toolu_` regex silently drops them. 0 cross-file collisions on 103,940 distinct ids.
-
-**Parallel tool batches have two on-disk layouts.** Layout A (80.5%) is strictly linear and
-interleaved — `tool_use, tool_result, tool_use, tool_result` — producing no fork. Layout B forks the
-`parentUuid` chain. Only the per-pair invariant generalises; do not assume a fork to group a batch.
-
-**`toolUseResult` is a main-transcript enrichment.** Present on 100.0% of unique main-transcript
-tool_result records; on workflow subagents it is **exclusively** the error-string form (zero
-object-typed). Structured tool telemetry (`structuredPatch`, `stdout`/`stderr` split, agent usage)
-exists only for the parent stream. The key is *absent*, not null, on child records — a
-`toolUseResult === null` test never fires.
-
-**`tool_result.content` is not always a string.** Over 20,000 records: string 19,707, list-of-image
-124, list-of-text 104, list-of-**`tool_reference`** 92 — an undocumented block type of shape
-`{"type":"tool_reference","tool_name":"Monitor"}`, 194 corpus-wide.
-
-**Long output spills to a file, but the field is unreliable.** Only 45 of 516 spill records carry
-`.toolUseResult.persistedOutputPath`; the path is recoverable in 506 of 516 by parsing
-`Full output saved to: <path>` out of the preview text. Also, the content block is truncated
-relative to `toolUseResult.stdout` by ~28 KB in the truncation cases — `toolUseResult.stdout` is the
-only authoritative copy of long output.
-
-**Interrupts do not break the tool join** — the `[Request interrupted by user]` record is written
-*after* the tool_result. But the next user turn can re-parent to an earlier uuid, which is a real
-`ExecutionStream` discontinuity.
-
-Zero historically unresolved tool_uses exist in this corpus — but that is a property of *this*
-corpus (no session was killed mid-call), not an invariant. An `unresolved` state is still required.
-
-### 1.5 Subagents — three spawn mechanisms, not one
-
-This is where my earlier write-up was most wrong. There is no single field spanning all three.
-
-| Spawn kind | Count | Parent→child join |
-| --- | --- | --- |
-| **Agent tool** | 89 | `meta.json.toolUseId` → `tool_use` name `Agent` (89/89), *and* parent's `toolUseResult.agentId` |
-| **Skill fork** | 16 (4 sessions) | `toolUseResult.agentId` with `status:"forked"` only — meta.json has **no** `toolUseId` |
-| **Workflow** | ~2,579 | `Workflow` tool_result's `toolUseResult.transcriptDir` (absolute path of the run dir) + `toolUseResult.runId`; and `journal.jsonl` `{type:"started", agentId}` → 2,583/2,583 bijection |
-
-An earlier pass claimed workflow children have no parent `tool_use` and that `journal.jsonl` is
-their sole causal record. **Both refuted** — `toolUseResult.transcriptDir` is an exact_unique join
-from a parent tool_use straight to the directory holding every workflow child.
-
-Confirmed by full re-scan of all 2,686 agent files (218,393 records):
-
-- filename `agentId` == in-record `agentId`, 2,686/2,686; zero collisions; zero files with >1 `agentId`
-- zero files with >1 `sessionId`; **zero files whose `sessionId` differs from the parent directory** —
- the child always shares the parent's session, with no deviation
-- `isSidechain`/`agentId` partition the two file classes perfectly: 0 sidechain records in mains,
- 0 top-level `agentId` in mains
-- `agentId` conforms to `^a[0-9a-f]{16}$` in 224,130/224,130 occurrences
-- `spawnDepth`: 2,654×1, 31×2, 1 null. **No depth ≥ 3 exists.** All 31 depth-2 agents carry
- `parentAgentId`, resolving to Skill-forked parents.
-- the child's first user record is a bare string **byte-identical** to the parent `tool_use`
- `input.prompt`, 89/89
-- `"Task"` does not exist as a tool name anywhere — the tool is `Agent`
-
-**Async return.** `toolUseResult.status:"async_launched"` is the launch acknowledgement, distinct
-from the result — the field distinguishes them explicitly. Completion arrives as a
-`<task-notification>` XML blob whose `<task-id>` **is** the agentId.
-
-> **Caveat that breaks my earlier "exact end-to-end" claim:** the notification's `<tool-use-id>` is
-> *not* reliably resolvable in the transcript it lands in — **28 of 54 fail**, because nested
-> children notify the MAIN session while their spawning `tool_use` lives in an agent file. For
-> depth-2 the split is clean and exact: all 31 toolUseIds resolve in the parent *agent* file, all 31
-> agentIds appear as `<task-id>` in the *main* transcript. So the edge is exact but **cross-file** —
-> resolution must search the session's whole file set, not one file.
-
-`<status>` vocabulary is `completed | failed | killed | stopped`, and **1,183 notifications carry no
-`<status>` element at all** — parsing it unconditionally will null-deref.
-
-`agentType` does **not** identify a workflow spawn (a workflow step can request a named agent type).
-The reliable discriminator is on-disk path / journal membership.
-
-**The journal leads the filesystem.** For in-flight runs, journalled `started` agentIds had no
-`agent-*.jsonl` yet — one even had a `result` journalled while its transcript was still missing.
-A collector meeting a live session must treat this as normal, not as corruption.
-
-### 1.6 Context epochs and session identity
-
-**Compaction is explicit and exactly linked.** `{"type":"system","subtype":"compact_boundary"}` with
-`compactMetadata`. 26 boundaries corpus-wide, zero in any subagent file.
-
-The critical field is **`logicalParentUuid`** — an explicit pointer to the last pre-compaction
-message, present on 26/26 boundaries and on **no other record type anywhere** (0 occurrences among
-133,347 other main records and 221k+ subagent records). Exact rule, 26/26:
-`logicalParentUuid == compactMetadata.preservedMessages.allUuids | last`.
-
-So epochs *are* explicitly chained; `parentUuid: null` on the boundary is a DAG-root display
-convention, not an absence of linkage. This is precisely the explicit epoch boundary v0.8 §4.2
-demands, and it means compaction never has to be inferred.
-
-Also established:
-
-- `boundary.uuid == summary.parentUuid`, `isCompactSummary == true`, summary is the **very next
- line** — 26/26, zero exceptions.
-- The summary's timestamp is **always earlier** than its own boundary's, by 1 ms to 1,097 ms, 26/26.
- **Never order an epoch boundary by timestamp.**
-- `preservedMessages.allUuids` is a strict superset of `uuids`. `uuids` resolves 26/26; **`allUuids`
- contains 5 uuids that exist nowhere on disk** — records held in context but never written. Since
- `logicalParentUuid == allUuids | last`, one boundary's `logicalParentUuid` is itself unresolvable.
- Both must be typed unresolved-capable.
-- The preserved segment does **not** abut the boundary: the last preserved uuid is immediately
- before it in only 16/26; gaps reach 705 records. Resolve by uuid only, never by file proximity.
-- `trigger` is `auto` 24 / `manual` 2. `preCompactDiscoveredTools` is absent on 2/26 and `isMeta`
- absent on 5/26 — both must be optional in a Go struct.
-
-**Resume appends to the same file with the same sessionId.** Proven by a 659-hour gap where
-`version` jumps 2.1.220 → 2.1.251 and the chain still links. Across all 60 main + 2,851 subagent
-files: **zero dangling parentUuids, zero pointing cross-file.** Root count per file =
-`1 + N_compactions + N_replay_restarts`.
-
-**`/clear` starts a new sessionId and a new file**, and is detectable at the **head of the successor
-file** (line 3–4) as a fixed 3-record chain: a root `user` record with `isMeta:true` and
-`<local-command-caveat>`, then a `user` record whose content is the string
-`<command-name>/clear</command-name>`, then `system/local_command` with empty stdout. So "this
-session was born by clearing a predecessor" is detectable — though the record carries no pointer to
-the predecessor's sessionId.
-
-**There is essentially no durable on-disk conversation identity spanning sessionIds.** Candidates,
-all weak:
-
-| Candidate | Verdict |
-| --- | --- |
-| `bridgeSessionId` | spans 2 sessionIds in 1 of 20; can be the empty string; one session has two values. Not 1:1. |
-| `.session_id` (snake_case, ≠ `.sessionId`) | **contested** — see §1.9 |
-| `slug`, `ai-title`, `cwd`, `gitBranch` | per-record or per-session, mutate mid-session |
-
-This confirms v0.8's core premise: **the conversation ID must be supplied by the application.**
-No local heuristic is sound.
-
-### 1.7 OTLP — traces exist but are largely dead code
-
-Claude Code emits all three signals, on very different terms. Three instrumentation scopes:
-`com.anthropic.claude_code` (metrics), `com.anthropic.claude_code.events` (logs),
-`com.anthropic.claude_code.tracing` v1.0.0 (spans).
-
-Metrics (8 counters) and logs (~25 events) need only `CLAUDE_CODE_ENABLE_TELEMETRY=1` plus the
-relevant exporter var.
-
-**Traces are double-gated:** `CLAUDE_CODE_ENABLE_TELEMETRY=1` **and**
-`CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1`, plus `OTEL_TRACES_EXPORTER`. Without the beta flag every
-`startSpan` returns a no-op dummy span. A first-party managed-config toggle can set the beta flag
-for spawned sessions, so it is reachable from configuration, not only from a raw env var.
-
-> **Correction to my earlier claim.** I wrote that `claude_code.subagent.spawn` and
-> `claude_code.compaction` matter to the design. **They are dead code for external users
-> in 2.1.245** — gated behind a hardcoded-false function and an Anthropic-internal endpoint path,
-> confirmed in 2.1.224 too. Only **five** span names are reachable: `claude_code.interaction`,
-> `.llm_request`, `.tool`, `.tool.blocked_on_user`, `.tool.execution`.
-> `.subagent.spawn`, `.compaction`, `.hook`, `.bash.subprocess`, `.mcp.rpc` are unreachable.
-
-The decisive consequence for lineage: **events carry no `agent_id`.** Agent lineage
-(`agent_id`, `parent_agent_id`) exists **only on spans** — i.e. only under the enhanced-telemetry
-beta. On the metrics/logs path there is no subagent identity, no Conversation ID, and no Segment
-concept at all.
-
-What events *do* carry joins cleanly: `session.id`, `prompt.id`, `message.uuid`, `request_id`,
-`tool_use_id` — all with exact transcript counterparts. Two caveats: the slash-command
-`user_prompt` emit site emits **no** `message.uuid` (so that join is unresolved for every
-`/command`), and `tool_decision.source` is a composite string (`hook:…`, `rule:…`, `mode:…`) that
-must be split on `:`, not treated as an enum.
-
-Also noted: an always-on 300 s metric reader exists **outside** the telemetry guard for
-enterprise/team accounts.
-
-### 1.8 Raw provider bodies — no proxy needed
-
-**`OTEL_LOG_RAW_API_BODIES=file:<dir>` is a first-class, undocumented raw-body dump.** Verified live
-on 2.1.245 with subscription OAuth: 12 provider calls captured across 4 sessions including a
-subagent spawn. It writes the fully-serialized Messages request params to
-`<dir>/<randomUUID>.request.json` and the assembled response to `<dir>/<req_…>.response.json`.
-
-The joins are exact:
-
-- **The response file's basename *is* the Anthropic request-id, and equals the transcript's
- `.requestId`.** Independently confirmed by MITM: the proxy saw response header
- `request-id: req_011CeZxce27ffvNEPbBWiUUx` and the transcript wrote the same value.
-- `requestId → sessionId` is 1:1 with zero collisions across 85,652 distinct ids.
-- The request body carries `.metadata.user_id` holding the session_id.
-- **`.diagnostics.previous_message_id` chains request N to the response of request N−1 by `msg_`
- id** — the ordered-model-input chain the design needs, materialized in the wire body itself.
-
-**Independently reproduced on 2.1.245 with subscription OAuth.** A live capture in this project
-confirms the mechanism and shows the request body is far richer than the design assumed. Top-level
-keys: `betas, context_management, diagnostics, max_tokens, messages, metadata, model, output_config,
-stream, system, thinking, tools`.
-
-| Field | Observed |
-| --- | --- |
-| `system` | array of 4 blocks, 11,765 chars total — the complete system prompt |
-| `tools` | 13 tools **with full `input_schema`** |
-| `messages` | the exact ordered array as serialized |
-| `cache_control` | present, with ttl and scope, e.g. `{"type":"ephemeral","ttl":"1h","scope":"global"}` |
-| `metadata.user_id` | a JSON string holding `device_id`, `account_uuid`, **`session_id`** |
-| `diagnostics.previous_message_id` | the chain to the prior response |
-
-**Three independent exact joins back to the transcript**, all verified on the captured pair:
-
-1. the response filename is the request-id — found in the transcript (2 fragment lines)
-2. `metadata.user_id.session_id` — resolves to a real transcript file
-3. `system[0]` is an `x-anthropic-billing-header` block carrying `cc_prompt_id=…`, which equals the
- transcript's `promptId`, plus `cc_version` and `cc_entrypoint`
-
-This removes the largest and most privacy-fraught piece of the design, and makes
-`input_manifest.state = full` genuinely reachable — system blocks, tool definitions, ordered message
-refs and cache breakpoints all present as *observed* rather than inferred, which transcripts alone
-can never do (§1.3).
-
-**One asymmetry to plan around:** the response file is named `<request-id>.response.json`, but the
-request file is named by a client-side random UUID (`<uuid>.request.json`) — the request-id does not
-exist until the response arrives. Pairing a request to its response therefore needs either the
-`api_request` event (which carries both `client_request_id` and `request_id`) or the body's own
-`diagnostics.previous_message_id` chain. `clientRequestId` is absent from transcripts entirely.
-
-**Thinking is redacted here too** — the captured response's thinking block is `"<REDACTED>"`
-(§1.3). Raw bodies do not recover reasoning.
-
-**The proxy route is a measurement-perturbing observer and should not be the default.** With
-`ANTHROPIC_BASE_URL` pointed at a recorder, Claude Code drops the `cache-diagnosis-2026-04-07` and
-`advanced-tool-use-2025-11-20` betas, loses `.diagnostics` entirely, and collapses two
-globally-scoped system `cache_control` breakpoints into one. It also hard-fails with 403 under OAuth
-unless the forwarder speaks HTTP/2. It remains useful as a cross-check, not as the collection path.
-
-### 1.9 Contested and unresolved
-
-Recorded rather than silently resolved:
-
-1. **`.session_id` (snake_case) as a cross-session identity.** One pass calls it a
- conversation-lineage root surviving `/clear` and spanning 4 sessionIds over 13 days, gated by
- entrypoint (present on 11,721 of 12,959 `cli` records, 0 on `claude-vscode`/`sdk-cli`). Another
- measures 15 distinct `(sessionId, session_id)` pairs of which **11 agree**, with all 8,037
- disagreements traced to **one stale sticky value** inherited by 3 sessions — and shows zero uuid
- overlap with the claimed ancestor. Needs a decisive test before any use.
-2. **Whether the later copy of a duplicated uuid is ever richer.** Passes disagree. The
- keep-first-and-merge rule is safe either way, but the disagreement should be settled.
-3. ~~Does the raw-body dump retain thinking text?~~ **SETTLED — no.** Live capture on 2.1.245: the
- response's thinking block is `"<REDACTED>"`. Combined with the request's
- `thinking.display="omitted"`, reasoning text is unrecoverable by any local mechanism (§1.3).
- *Remaining sub-question:* raw-body behaviour under concurrency and inside subagents is still
- unmeasured.
-4. ~~Why one subagent file retains thinking text?~~ **SETTLED — the model.** Controlled comparison
- within one session at one version isolates it: haiku-4-5 retains text, opus-5 does not (§1.3).
-5. **Whether transcripts are ever truncated or rotated.** All evidence says append-only, which the
- storage design depends on (§2.6). Not proven.
-6. **Hook `agent_id` coverage** — reported present on every hook fired inside a subagent and absent
- on the main thread, from 5 live headless sessions. Needs wider confirmation.
-
-### 1.10 Hooks — the best lineage surface, with a sharp edge
-
-31 hook events, extracted from the binary's registry and verified by 5 live headless sessions with
-all events wired. The common envelope carries `session_id`, `transcript_path`, `cwd`, `prompt_id`,
-`permission_mode`, **`agent_id`**, `agent_type`, `effort`.
-
-`agent_id` is present on every hook fired **inside** a subagent and absent on the main thread —
-which is exactly the E0/E1 stream discrimination transcripts blur. The parent→child edge is
-delivered twice: `SubagentStop.agent_transcript_path` names the child file directly, and
-`PostToolUse(tool_name="Agent").tool_response.agentId` pairs the parent's `tool_use_id` to the
-child's `agent_id` (verified unambiguous with 3 concurrent subagents).
-
-Compaction is fully instrumented: `PreCompact{trigger, custom_instructions}` → `SubagentStop`
-(a phantom internal agent with `agent_type:""`) → `SessionStart{source:"compact"}` →
-`PostCompact{trigger, compact_summary}`.
-
-Three hard limits:
-
-- Hooks are **synchronous and on the critical path** (~5.6 ms/invocation measured, 60 s default
- timeout). A collector hook must never do work inline.
-- The naive `cat >> spool.jsonl` pattern **demonstrably corrupts lines above ~64 KB** under
- concurrent subagents. Appends must be single-`write(2)`-sized or go through a serializing writer.
-- Hooks never expose `message.id` or `requestId`, so they cannot produce ProviderCall manifests.
-
-Note the published docs are ahead of and inconsistent with the shipped binary — code to observed
-payloads. Also: the current `settings.json` on this machine has **no hooks block**, so no
-hook-based collection point exists yet.
-
-### 1.11 Sidecar surfaces worth collecting
-
-| Surface | Verdict |
-| --- | --- |
-| `file-history/<sessionId>/<sha256(abs_path)[:16]>@vN` | **Collect.** Directory name *is* the sessionId (26/26); blob name verified by recomputation. `file-history-delta.messageId` is the exact uuid of the assistant `tool_use` line that performed the edit (59/59). |
-| `history.jsonl` | **Collect, with care.** Neither complete nor a superset: 37 of 52 transcripts have zero history lines, but 330 of 345 sessionIds exist *only* here (transcripts pruned). |
-| `~/.claude.json` `.projects[<cwd>]` | **Collect.** Carries `lastSessionFirstPrompt` (verbatim first human prompt) and `lastSessionId`. Entry shapes vary 9–35 keys. |
-| `telemetry/1p_failed_events.*.json` | **Optional.** Not OTLP — a proprietary `ClaudeCodeInternalEvent` NDJSON spool of *undelivered* events, 41 `tengu_*` names, numeric payload base64-encoded in `additional_metadata`. Useful for cache-breakpoint and context-size signals absent elsewhere. |
-| `sessions/<pid>.json` | **Optional.** Live process registry, deleted on exit. `bridgeSessionId` joins the transcript's `bridge-session` record only after stripping a `session_` vs `cse_` prefix difference. |
-| `shell-snapshots/`, `session-env/`, `backups/`, `paste-cache/` | **Skip.** `session-env` is empty; `backups` rotates in minutes; 91% of paste-cache hashes have no surviving blob. |
-
-Two traps worth naming:
-
-- **`file-history` deltas and snapshots must be unioned.** Deltas only ever name `@v1` blobs (337);
- every blob at `@v2…@v49` (1,667 of 2,004) is reachable **only** through
- `file-history-snapshot.trackedFileBackups`. Neither surface alone is complete. The version counter
- is per-`(session, file)`, not per-file, so the exact key is `(sessionId, backupFileName)`.
-- **`history.jsonl` `.pastedContents` has two shapes.** 148 of 305 carry a full inline `content`
- string (≤1,006 chars); the rest carry only `contentHash`. Reading only `contentHash` drops half
- the payloads.
-
-### 1.12a Reconstruction: the `parentUuid` graph is a DAG, not a chain
-
-**This is the finding with the largest implementation consequence.** Parallel tool calls fork the
-graph: `tool_use#1 → { tool_use#2 (same message.id), tool_result#1 }`. An ancestor walk from the next
-provider call traverses one branch and silently drops the other `tool_use` and its `tool_result`.
-
-> **33,798 of 86,687 provider calls (39%) reconstruct under ancestor-walk to a message list carrying
-> a `tool_use` with no matching `tool_result` — a body the Messages API would reject.**
-> Line-order reconstruction is balanced in 2,622 of 2,623 tool-bearing files (the one exception being
-> a live in-flight file at snapshot time).
-
-**Rule: assemble by `(file, message.id)` grouping plus line order within an epoch. Never by ancestor
-walk.** Treat a fork as a topological hint, not a branch to choose between.
-
-Supporting measurements:
-
-- 4,073 fork nodes corpus-wide. **Every fork has exactly 2 children, never more.** Shapes:
- `assistant:tool_use + user:tool_result` = 3,905; `assistant:* + system:api_error` = 160; 3 others.
-- The *sequential* multi-tool case is a plain chain (`tool_use1 → tool_result1 → tool_use2 → …`,
- all one `message.id`). Only parallel dispatch forks.
-- Fork rate is 0.49% in main transcripts but **1.69% in subagents**, which hold ~69% of all provider
- calls — so main-transcript-only measurement understates exposure. 31.8% of subagent calls carry
- ≥2 `tool_use` blocks versus ~3% in main.
-- Fragments of one `message.id` are **never** interleaved with fragments of a different one
- (0 of 86,716 calls), which is what makes `message.id` grouping a genuinely independent second
- derivation from the chain.
-- Order inversion: for prompt-bearing turns the request puts reminders **first** and the user's text
- **last**, while the transcript writes the user record first with attachments as descendants. For
- tool_result turns the two orders agree. No chain walk discovers this.
-
-### 1.12b The duplicate-uuid blocks are evidence, not noise
-
-§1.2 records these as resume replays. **That framing is wrong and understates them badly.**
-
-Measured across all 10 affected files: the re-emitted block is **exactly** the set of user/assistant
-records lying before it that are *off* the following compaction boundary's ancestry.
-`dup_not_off = 0` in 10/10 files; `off_not_dup = 0` in 8/10.
-
-Claude Code is dumping precisely the context members its own `parentUuid` chain fails to reach. That
-makes these blocks an independent, chain-external recording of the real message set at 10 sites, and
-it establishes the reconstruction rule directly:
-
-> **context = chain ancestry ∪ off-chain siblings**
-
-The 2 exceptions are equally informative: 1f04957c has 330 and e84d2dab 37 off-chain records that
-were *not* re-emitted, clustering in whole abandoned runs. So off-chain records partition cleanly
-into **(a) parallel-tool siblings — re-emitted, genuinely in context** and **(b) abandoned or errored
-branches — not re-emitted, genuinely dropped.** That is ground truth for a decision the assembler
-would otherwise have to guess at.
-
-Also corrected: all 10 duplicate blocks terminate on the line immediately before a `compact_boundary`;
-only 2 of 10 are dense contiguous prefix replays; 0 of 1,530 lie in the ancestry of the following
-boundary's `logicalParentUuid`.
-
-### 1.12c What the continuity invariant can and cannot see
-
-Established by mutation testing — corrupting the reconstruction and measuring whether the check
-notices.
-
-**Blind to message content, completely.** Ten destructive mutations moved the failure count by
-**exactly 0** on both populations: `content[0]`-only, dropping thinking blocks, grouping by
-`requestId`, dropping the `isCompactSummary` record, dropping `isMeta` records, blanking every
-tool_result payload, blanking every text block, replacing blocks by their type string, replacing
-blocks by one indistinguishable token, and collapsing every record to a single indistinguishable
-block.
-
-Evaluated against transcripts, the check is therefore a function of exactly one thing: **the
-role-segmented record-count sequence along the chain.** It cannot observe a single byte of any
-message body.
-
-**Loud about intra-message order.** Ordering a message's fragments by uuid instead of line number
-takes main failures 178 → 9,714 and subagent 518 → 30,437.
-
-> v0.8 must describe this as a **structural cross-check on
-> `(message.id grouping × parentUuid tree × fragment order)`** — never as checking the request body.
-
-**Do not use it as a ranking oracle.** It scores three known-wrong reconstructions *better* than the
-correct one: keep-LAST duplicate handling (−19), dropping `<synthetic>` assistants (−13), and line-order
-reconstruction that readmits abandoned fork branches (−20 main, −518 subagent). Its safe use is
-strictly as a falsifier.
-
-**It never crosses a compaction boundary**, so it can never observe the one place where
-`next = prev + assistant + new` is definitely false.
-
-### 1.12d Semantic equality must ignore `cache_control`
-
-Measured on real adjacent capture pairs: **byte equality of the shared prefix fails 4/4; equality
-after stripping `cache_control` holds 6/6.** In one pair `messages[0]` is byte-identical at 22,477
-bytes while `messages[1]` and `[2]` differ by exactly 48 bytes each — entirely the removal of
-`{"ttl":"1h","type":"ephemeral"}`. Breakpoints migrate forward on every call.
-
-v0.8's "recursive semantic equality, not raw byte equality" is therefore **required, not
-stylistic**, and the design must state explicitly that the comparison ignores `cache_control` — a
-reconstructor cannot place these anyway, since transcripts contain none.
-
-On real wire bodies the invariant **holds**: across a 3-call chain,
-`next.messages[0..k] == prev.messages[0..k]` modulo `cache_control`, 100%, with the appended tail
-exactly `assistant(prev_response) + new user input`.
-
-### 1.12e What the log does not contain
-
-Not message content — that is complete (§1.3). What is missing is the **injected preamble prepended
-to `messages[0]` on the wire**:
-
-- an 8,338-char `<system-reminder>` block wrapping CLAUDE.md, `userEmail` and `currentDate`.
- `grep -c claudeMd` over transcripts returns 0. Subagents receive it too.
-- the wrapper text and ordering of agent/skill/deferred-tool listings — the underlying data survives
- as `attachment` records (88.8–98.7% recoverable) but not the rendered form
-- `tools[]` mutates between adjacent calls (96 → 116 one turn apart) via deferred loading, recorded
- only as a `deferred_tools_delta` attachment
-
-**Scale matters here and is easy to misread.** The preamble is a fixed ~8 KB (no project CLAUDE.md)
-to ~40 KB (a 31 KB project CLAUDE.md). Against a real session carrying 12,027,973 chars of
-conversation content that is ~0.3%. A character-weighted "99% not recoverable" figure measured on a
-26 KB probe session is not representative.
-
-**The attachment→request mapping is beta-gated and unstable**, which alone defeats transcript-only
-list reconstruction. Same machine, same day, two schemes: without the
-`mid-conversation-system-2026-04-07` beta the attachments become separate `<system-reminder>`-wrapped
-user text blocks *before* the prompt; with it they merge into a single 13,953-char **`role:"system"`
-message** placed *after* the user message. The `betas` list is not in the transcript, so a
-reconstructor cannot tell which applied. `role:"system"` inside `messages[]` is a fifth message class
-the model must accommodate.
-
-### 1.12f Human turns that live only in attachments
-
-**614 genuinely human-typed turns exist only as `queued_command` attachment records** with no
-`.message` twin. A reader that walks `.message` alone loses them.
-
-They must be filtered: `queued_command` carries `commandMode` (`prompt` | `task-notification`) and
-`origin.kind` (`human` | absent). Of 1,558 unique records only 960 are human input; including the
-other 598 in would insert machine-generated `<task-notification>` blobs as if the user had typed them.
-
-### 1.12g Corpus hazards for any absence test
-
-- **28.6% of main-transcript lines carry no `uuid` at all** — 38,340 of 133,887, across 14
- metadata-only record types. None is context-bearing, but a parser assuming every line has a uuid
- will crash or mis-position.
-- **177 of 2,935 files contain zero user/assistant/system records** — metadata only. A reconstructor
- treating every `.jsonl` as a session emits 177 empty conversations.
-- **The corpus is self-contaminated by this investigation.** Sessions analysing the data wrote the
- strings under test into their own transcripts. A bare `grep` for `cache_control` returns 18 files,
- all of them research prose. Any "no transcript contains X" claim must grep the *field*
- (`"uuid":"…"`) or apply a pre-analysis mtime cutoff.
-- **The corpus is live.** Subagent pair counts drifted 56,960 → 57,022 within 20 minutes. Quote
- counts with a timestamp; never treat them as constants.
-
-### 1.13 Two hazards that will bite an implementation
-
-**`sessionId` is not reliably lowercase.** 2 of 47 transcripts use an uppercase UUID
-(`41090DAB-113C-…`). Every join must be byte-exact and never case-folded.
-
-**Lines exceed `bufio.Scanner`'s default limit by 15×.** The longest single JSONL line measured is
-**979,632 bytes**; the largest file is 59 MB. `bufio.Scanner` silently fails at its 64 KB default
-token size. Use `bufio.Reader` with an explicit large buffer, or a streaming JSON decoder.
-
----
-
-## 2. Raw storage model
-
-### 2.0 Collection posture (decided)
-
-**Claude Code speaks exactly one protocol outward: OTLP.** Exporter selection in the binary accepts
-`console|otlp|prometheus` for metrics and `console|otlp` for logs and traces; any other value
-throws. There is no webhook, no custom sink, no alternative wire format. Pushing to us means running
-an OTLP receiver, or nothing.
-
-Three possible postures:
-
-| Posture | Mechanism | Cost to the user | Yield |
-| --- | --- | --- | --- |
-| **Pull** | tail `~/.claude/projects/**` | none — Claude Code does not know we exist | all structure and content (§1.2–1.6) |
-| **Push** | OTLP receiver (gRPC 4317 / HTTP 4318) | `CLAUDE_CODE_ENABLE_TELEMETRY=1` + `OTEL_EXPORTER_OTLP_*` | measurement only; no agent identity (§1.7) |
-| **Exec** | hooks — Claude Code runs our binary, JSON on stdin | edits `settings.json`; on the critical path | `agent_id`, liveness, compaction lifecycle (§1.10) |
-
-**v1 is pull only.** SkyWalking AI Sessionizer faces the filesystem, not the agent. Beyond the channel
-reasoning below, one practical argument dominates: **pull works on history that already exists** —
-2,900 files and 1.2 GB on the development machine today. Push and exec capture only from the moment
-they are configured and can never see yesterday. For a tool whose purpose is reconstructing
-conversations, that is a categorical difference.
-
-This also fixes the meaning of `src` in the envelope header: for a pull adapter it is the local
-source path, which is not redundant with the layout because the layout deliberately drops the
-project slug (we key by session id) and the workflow directory (we flatten children by agent id).
-`src` is the only place both survive, and it is what makes `off` verifiable. For a future push
-adapter `src` becomes the wire-side origin and `off` drops out.
-
-### 2.0.1 Root location and channel selection (decided)
-
-**`<root>` is assigned by SkyWalking AI Sessionizer configuration.** Default is a directory under the system
-temp dir. In tests, the root is placed side by side with the fixtures in the same folder, so a test
-case is self-contained and needs no absolute paths.
-
-**v1 collects transcripts only.** Rationale, from §1.7 and the event inventory:
-
-- Every structural edge — stream lineage, tool joins, the spawn chain, epoch boundaries — lives in
- transcripts at higher fidelity than any other channel.
-- OTLP **events carry no usable agent identity**: `query_source` is normalized to coarse
- `main | subagent | auxiliary` (the raw `agent:builtin:<name>` string is never exported), and
- `subagent_completed` is emitted with no `agent_id` at all.
-- OTLP **spans** would carry `agent_id`/`parent_agent_id`, but require
- `CLAUDE_CODE_ENHANCED_TELEMETRY_BETA`, and the two spans that would matter most
- (`subagent.spawn`, `compaction`) are dead code.
-
-What events *do* bring, and why they are a Plan 02+ addition rather than a v1 requirement:
-per-call `duration_ms` and `cost_usd` (absent from transcripts entirely), tool `duration_ms` and
-`error_type`, and `tool_decision` with a composite `source` (`hook:` / `rule:` / `mode:`).
-`assistant_response.message.uuid` is also useful — it is the uuid of the **last** transcript line of
-a split message, i.e. the terminal fragment that §1.3 shows is the only authoritative one.
-
-One asymmetry to remember if raw bodies are enabled later: the response file is named
-`<request-id>.response.json` and joins to the transcript directly, but the **request** file cannot
-be — the request-id does not exist until the response arrives, so it is named by a client-side id,
-and `clientRequestId` is absent from transcripts. Joining a request body therefore needs either the
-`api_request` event (which carries both `client_request_id` and `request_id`) or the body's own
-`.diagnostics.previous_message_id` chain, which works only first-party.
-
-The layout below reserves `log-*` and `trace-*` so adding either channel later requires no rework.
-
-### 2.1 Principles
-
-1. **Session is the primary partition.** Retention, purge, and the pipeline are all session-scoped.
- Purge must be a directory delete.
-2. **Files are write-once.** temp name → fsync → rename → `chmod 0444`. Never appended, never rewritten.
-3. **The landed file is a buffer, not a wire capture.** OTLP arrives cross-session and must be
- demultiplexed, which means re-serialization. We do not claim byte-exactness with what the client
- sent, and must not imply it.
-4. **Loss is detectable.** A monotonic per-session sequence makes a gap visible.
-5. **Collection never blocks the agent.**
-
-### 2.2 Layout
-
-```
-<root>/<session-id>/
- session.state next_seq · liveness · last_scan
- rebuild_progress assembler watermark
- streams/
- main/ transcript.cursor · transcript-<ts>-<seq>.jsonl ExecutionStream E0
- <agent-id>/ transcript.cursor · transcript-<ts>-<seq>.jsonl E1..En
- meta.cursor · meta-<ts>-<seq>.jsonl agent-*.meta.json
- runs/<run-id>/ journal.cursor · journal-<ts>-<seq>.jsonl workflow run journal
- manifest.cursor · manifest-<ts>-<seq>.jsonl workflows/wf_<run-id>.json
- script.cursor · script-<ts>-<seq>.jsonl workflows/scripts/*.js
-```
-
-**One directory per workflow run**, not three parallel ones keyed by the same id -
-mirroring "everything about one stream in one folder".
-
-**Cursors are named `<kind>.cursor`, not `<kind>-cursor`.** Landed files are
-`<kind>-<ts>-<seq>.jsonl`, so a hyphen would make a cursor share a prefix with the data it
-tracks and turn any prefix scan into a bug.
-
-**Cursors live beside their stream, not in one session-level file.** A session can hold ~2,700
-subagent streams, so a single cursor file would mean rewriting a ~200 KB file for a one-line change
-on every pass. Per-stream cursors mean only what changed is written.
-
-`session.state` stays session-scoped because `next_seq` must be monotonic **across all streams** —
-that is what collapses `rebuild_progress` to a single watermark. Consequence: **a session is the unit
-of work and is processed single-threaded**; parallelism is across sessions, never within one.
-
-**No `origin` file.** The path states the session and stream, the filename
-prefix states the kind, and the envelope header (§2.3a) already carries `src`, `session`, `stream`
-and `adapter` on every file. A separate metadata file would only restate them. Metadata gets its own
-file when content is large enough that finding or indexing it is the problem — which is not the case
-for a source path and two ids.
-
-For the same reason `meta.json` is not copied as a raw `.json` blob: it is source content, so it
-lands as an ordinary record stream (`kind: agent_meta`, header + one record) and needs no special
-read path.
-
-That is the complete v1 tree. Three prefixes are **reserved but not written in v1**, listed here so
-adding a channel later needs no restructuring:
-
-| Reserved | Would hold | Blocked on |
-| --- | --- | --- |
-| `log-<ts>-<seq>.jsonl` | the 25 `claude_code.*` log events, demultiplexed per session, one record per line — the source of per-call `duration_ms` / `cost_usd`, tool durations, and `tool_decision` | §2.0: measurement, not structure |
-| `trace-<ts>-<seq>.jsonl` | the 5 reachable spans (`interaction`, `llm_request`, `tool`, `tool.blocked_on_user`, `tool.execution`) | requires `CLAUDE_CODE_ENHANCED_TELEMETRY_BETA` |
-| `rawbody/<request-id>.{request,response}.json` | `OTEL_LOG_RAW_API_BODIES` output — the only path to a `full` input manifest | privacy; must be opt-in with redaction |
-
-`<ts>` is receipt time, compact RFC 3339 with nanoseconds (`20260831T091204.481523000Z`) —
-sortable, no `:`. `<seq>` is **one monotonic counter per session across all signals**, resumed on
-restart from the highest sequence on disk.
-
-**Every stream is a folder, including main.** The main transcript is ExecutionStream E0; giving it
-the same shape as the children means the pipeline reads one directory pattern rather than two code
-paths, and everything about one stream lives together.
-
-**Child folders are flat by `agent-id`, deliberately not mirroring the source tree.** Two reasons:
-
-1. `agentId` is globally unique (`^a[0-9a-f]{16}$`, 2,686 distinct, zero collisions, never in two
- files), so nesting is not needed for uniqueness.
-2. **The storage path must not encode a relationship the pipeline is supposed to derive.** Nesting
- by `wf-id` or by `parentAgentId` would make the tree assert a join; when such a join turns out to
- be wrong — as several did during this investigation — correcting it would mean *moving files*,
- breaking the write-once guarantee. This matters most for the 31 depth-2 agents, whose parent is
- another agent rather than the main stream.
-
-The grouping is preserved as *data*, not as path structure, via three independent routes: the
-envelope header's `src` (which retains the full source path including the workflow directory), the
-landed `meta` stream (`agentType`, `spawnDepth`, `toolUseId`, `parentAgentId`), and the journal's
-`started` records.
-
-**A child's owning session is known two independent ways** — from the source path
-(`projects/<slug>/<session-id>/subagents/…`) and from the records themselves, which carry the
-parent's `sessionId` (verified 2,686/2,686, zero deviation). The collector must cross-check them at
-land time and raise a conflict on mismatch rather than silently preferring one.
-
-Raw bodies are keyed by request-id because the source filename already *is* the request-id, and that
-value joins directly to the transcript's `requestId` (§1.8).
-
-### 2.3 Why `.jsonl` and not `.json`
-
-For write-once files the appendability argument is weak, so the real reasons are:
-
-- **Line number is a stable record ordinal.** A source reference becomes *(file, line)* — an
- integer — instead of a JSON pointer into nested structure.
-- **Byte offsets are valid resume points.** Our tailer cursor is a byte offset; in a line-delimited
- file an offset always lands on a record boundary.
-- **O(1) memory streaming.** Files reach 59 MB; whole-file parse is not viable.
-- **One toolchain.** Claude Code's own transcripts are `.jsonl`.
-- **A torn write costs one line, not the file.**
-
-For OTLP we therefore flatten each export to **one `LogRecord` or `Span` per line**, with resource
-and scope attributes attached, rather than storing the nested `ExportLogsServiceRequest`. The
-nesting buys nothing after demultiplexing and costs the ordinal.
-
-`.meta.json` and `rawbody/*.json` stay `.json` — they are single documents at source, not streams.
-
-### 2.3a Uniform record envelope
-
-Every landed **record stream** — regardless of source — uses one shape: a header line carrying
-everything invariant for the file, then one line per record carrying only what varies.
-
-```jsonl
-{"h":1,"seq":175,"at":"2026-08-31T09:12:06.114000000Z","kind":"transcript",
- "adapter":"claude-code/0.1.0",
- "src":"-Users-…-python/92a5d58e-…/subagents/agent-afdc6064f2f8dcb06.jsonl",
- "session":"92a5d58e-36cf-4838-a58c-66058206b903","stream":"afdc6064f2f8dcb06",
- "state":"available","derivation":null}
-{"ord":1042,"off":532231,"sha":"9f3a1c2b…","payload":{…}}
-{"ord":1043,"off":533104,"sha":"1b2c4d5e…","payload":{…}}
-```
-
-**Source kinds.** Note that `log` and `event` are not two sources: Claude Code's events *are* OTLP
-log records, with body = the string `claude_code.<event_name>`.
-
-```
-transcript | otlp_span | otlp_log | provider_body | agent_meta | journal | derived
-```
-
-`derived` is SkyWalking AI Sessionizer's own output; the slot is reserved now so Plan 02 needs no format
-change.
-
-**Why header + records.** `kind`, `src`, `session`, `stream` are constant for a whole delta file.
-Repeating them per record costs ~200 bytes × ~350k records ≈ 70 MB of pure redundancy. Hoisting them
-also makes each file self-describing — it stays interpretable if moved out of its directory, which a
-bare stream of source lines is not. For OTLP the header pays off twice: resource and scope
-attributes are per-export constants and belong there rather than on every record.
-
-**Per-record fields.**
-
-| Field | Purpose |
-| --- | --- |
-| `ord` | ordinal **in the source**, not in this file — deltas start mid-file, so these differ, and only `ord` is stable across passes |
-| `off` | byte offset in source; lets any record be re-verified against the original while it exists |
-| `sha` | digest of the payload bytes |
-| `payload` | the source record, **byte-for-byte** |
-
-**Byte-fidelity is preserved.** The payload is copied through as raw bytes rather than re-serialized
-(`json.RawMessage` in Go), so key order, whitespace and number precision all survive. The envelope
-does not cost us the exactness of the wrapped record.
-
-**Per-source mapping.**
-
-| Source | `kind` | `ord` | `off` | Notes |
-| --- | --- | --- | --- | --- |
-| main / subagent transcript | `transcript` | source line no. | byte offset | payload verbatim |
-| workflow journal | `journal` | source line no. | byte offset | payload verbatim |
-| `agent-*.meta.json` | `agent_meta` | 0 | — | single document: header + 1 record |
-| OTLP logs *(reserved)* | `otlp_log` | index in export | — | `derivation:"otlp_flatten"`; resource+scope in header |
-| OTLP spans *(reserved)* | `otlp_span` | index in export | — | same |
-| raw bodies *(reserved)* | `provider_body` | 0 | — | header + 1 record |
-| sessionizer output | `derived` | — | — | Plan 02+ |
-
-`state` is the record-level content state (`available | redacted | truncated | omitted |
-unavailable`). In v1 nothing is redacted at collection time, so it is `available` in the header and
-never overridden; the slot exists for privacy modes later. Field-level redaction — such as the
-stripped thinking text in §1.3 — is a coverage concern for Plan 02, not a collection-time one.
-
-**State files are not record streams.** `cursor`, `session.state` and `rebuild_progress` stay
-plain line-oriented key-value text (§2.5). The rule is: streams get envelopes, state files stay
-simple.
-
-### 2.4 Sequence, duplicates, and what digests can honestly prove
-
-Protobuf serialization is **not canonical** — the spec does not guarantee byte-identical output for
-the same logical message. "Same content ⇒ same bytes" is false in general.
-
-What *is* true is narrower: an OTLP exporter retrying a failed export re-sends the same serialized
-buffer. That is a property of exporter behaviour, not of protobuf.
-
-Therefore:
-
-- **Drop duplicates at the receiver's edge**, in memory, on the digest of received wire bytes *before*
- demultiplexing. A retry hits the cache and is dropped.
-- **No digests in filenames for duplicate checks.** After demultiplexing the bytes are ours; a digest attests
- to our buffer's integrity, not to what the client sent.
-- **The assembler must tolerate duplicate records regardless** — an intermediate Collector can
- re-batch, delivering the same record inside two payloads no digest will match. It must also drop duplicates
- transcript records by `(file, uuid)` for the independent reason in §1.2.
-
-Sequence gaps detect *our* losses. They cannot detect client-side drops, because OTLP carries no
-client sequence number. That should be reported as unknown, not papered over.
-
-### 2.5 State files
-
-All are plain line-oriented text, rewritten atomically via temp-and-rename, holding current state
-rather than history. If an audit trail is wanted later, that is a separate append-only log.
-
-#### Two cursor kinds
-
-The six source kinds (§2.8) divide into growing files and rewritten files, and they need different
-cursors.
-
-**Append cursor** — for `.jsonl` sources that only grow:
-
-```
-# streams/afdc6064f2f8dcb06/cursor
-schema 1
-kind append
-source subagents/agent-afdc6064f2f8dcb06.jsonl # relative to source_root
-dev 16777232
-ino 8461299
-offset 532231 # bytes consumed → next read starts here
-ord 1042 # source line consumed through → next batch starts at ord 1043
-last_uuid 5acb6271-ea30-4e55-a27f-ff8a41f579c0
-tail_sha256 1a2b3c… # sha256 of [offset-1MiB, offset)
-last_seq 175
-size_at_scan 532231
-updated_at 2026-08-31T12:00:00.000000000Z
-state active # active | source_gone | conflict
-```
-
-Each field earns its place against a specific failure:
-
-| Field | Catches |
-| --- | --- |
-| `dev` + `ino` | the file was replaced or rotated under the same path |
-| `offset` vs `size` | truncation, and "is there anything new" without trusting mtime |
-| `tail_sha256` | rewrite of recently-consumed bytes |
-| `ord` | avoids re-counting lines from byte 0 to continue numbering |
-| `last_uuid` | semantic drift — **diagnostic only** (28.6% of lines carry no uuid, §1.12g) |
-| `state` | distinguishes a pruned source (normal) from a conflict (stop) |
-
-`offset` is both "position consumed" and "next round's starting point" — one value, no second field.
-They diverge only for the in-flight tail: if the file is 90,000 bytes but the last complete newline
-is at 85,820, we record 85,820. Recording what was *scanned* rather than *consumed* would be wrong.
-
-**Why a tail window rather than a full prefix digest.** Re-hashing `[0, offset)` every pass means
-re-reading up to 59 MB per cycle. The window is sized at 1 MiB, comfortably above the largest
-observed line (979,632 bytes), so it always spans at least one complete record boundary. Combined
-with `ino` (rotation) and `size < offset` (truncation), the only uncovered case is an in-place edit
-far behind the cursor — not a Claude Code behaviour. Full-prefix hashing becomes an opt-in strict
-mode.
-
-**Snapshot cursor** — for whole-file JSON that is *rewritten*, not appended:
-
-```
-# manifest/wf_0477d6c4-5e3/cursor
-schema 1
-kind snapshot
-source workflows/wf_0477d6c4-5e3.json
-content_sha256 9f3a1c2b…
-size 18422
-mtime 1788046628
-last_seq 176
-updated_at 2026-08-31T12:00:00.000000000Z
-state active
-```
-
-A snapshot cursor lands a **new immutable version whenever the digest changes**, turning a mutable
-source into an immutable version history. `wf_<run-id>.json` genuinely needs this — it carries
-terminal state (`completed`, `killed`), `durationMs`, `totalTokens` and `result`, so it is rewritten
-as a run progresses. `agent-<id>.meta.json` uses the same cursor but in practice lands exactly once
-(§2.8).
-
-#### `rebuild_progress`
-
-Assembler state. Because `<seq>` is monotonic **across all streams**, it collapses to one watermark
-regardless of how many child streams exist:
-
-```
-updated_at 2026-08-31T09:14:02Z
-pass 41
-landed_seq 174
-```
-
-That is the point of the shared counter — without it this file would need thousands of per-stream
-lines. It is safe because the collector assigns the sequence at write time, so a file never appears
-later with a lower sequence and the watermark cannot skip a hole.
-
-### 2.6 The mutable-source problem
-
-Transcripts are appended to **live**, so they cannot be referenced in place — they violate the
-write-once model everything else depends on.
-
-Proposal: **tail them into immutable deltas.** Each `transcript-<ts>-<seq>.jsonl` contains only the
-bytes appended since the last cursor. Mutability is then confined to the tailer.
-
-Evidence supports this strongly: resume appends to the same file, and across all 60 main + 2,851
-subagent files there are **zero dangling parentUuids and zero cross-file references** (§1.6). The
-replay phenomenon (§1.2) is also an *append* — a contiguous block written at the resume point — so
-the file stays append-only and the prefix digest holds. Whether truncation ever occurs is unproven
-(§1.9), which is exactly what the prefix-digest check exists to catch.
-
-Open question: continuous tailing (a daemon) versus on-demand scanning at rebuild time. On-demand is
-simpler and loses nothing **provided** transcripts are never truncated. The §1.9 item decides this.
-
----
-
-### 2.7 Pull-mode positioning and completeness (Phase 1)
-
-Positioning is a **pull-only** concern. Phase 2 (push) has no cursor — it has an at-least-once
-delivery problem instead. Different failure modes, which is the main reason the two postures should
-not share an abstraction beyond "produces landed records".
-
-**A session is never "complete".** Resume appends to the *same* file with the *same* sessionId —
-proven by a 659-hour gap in one transcript where `version` jumped 2.1.220 → 2.1.251 and the chain
-still linked (§1.6). A file quiet for a month can grow again. The collector can therefore never mark
-a session done and stop watching; it can only report **quiet since T**. This propagates: retention
-cannot trigger on "completed", and segment closing needs the `close_candidate` → gates → `committed`
-staging of v0.8 §3.3 rather than a bare idle timeout.
-
-Five levels of completeness, each with its own rule:
-
-| Level | Rule |
-| --- | --- |
-| **Byte** | consume only up to the last `\n`; a trailing partial line is an in-flight write — never parse it, never advance past it |
-| **Record** | free once byte-level holds (this is why `.jsonl`, §2.3) |
-| **Provider call** | close call C when a line with a different `message.id` arrives. **Do not wait for the terminal fragment** — 7.14% of calls never get one (§1.3), so waiting would hang on one call in seven. Mark `terminal_missing` instead. |
-| **Stream** | a subagent is finished per the journal `result`/`failed`, the `<task-notification>` status, or the parent's tool_result — but **the journal leads the filesystem** (§1.5), so "told it finished, haven't seen its data yet" is a normal state |
-| **Session** | no session-end record exists. The only liveness signal is `~/.claude/sessions/<pid>.json`, present while the process runs and deleted on exit. State model is `live \| quiet \| unknown` — never `complete`. |
-
-**Cursor shape** — four fields, each catching a different failure:
-
-```
-src <path> dev=16777232 ino=8461299 offset=8823104
- uuid=ccb3597f-… sha256=1a2b3c…
-```
-
-- `offset` — resume point, always on a line boundary
-- `dev`+`ino` — catches **rotation**: same path, different inode means replaced, not appended
-- `uuid` — cheap semantic anchor; the last record consumed should still be there
-- `sha256` — digest of bytes `[0, offset)`, the authoritative rewrite check
-
-Re-hashing 59 MB every pass is wasteful. Check the cheap invariants first — `size >= offset`, inode
-unchanged — and do a full re-verify only when one trips. Go's `sha256` implements
-`encoding.BinaryMarshaler`, so the rolling state can be persisted and extended rather than
-recomputed in the forward direction.
-
-### 2.8 Pull-mode collection process (Phase 1)
-
-#### The seven source kinds
-
-| # | Source | Nature | Cursor |
-| --- | --- | --- | --- |
-| 1 | `<session-id>.jsonl` | append-only | append |
-| 2 | `subagents/agent-<id>.jsonl` | append-only | append |
-| 3 | `subagents/agent-<id>.meta.json` | write-once | snapshot |
-| 4 | `subagents/workflows/<wf>/agent-<id>.jsonl` | append-only | append |
-| 5 | `subagents/workflows/<wf>/journal.jsonl` | append-only | append |
-| 6 | `workflows/wf_<run-id>.json` | **rewritten** | snapshot |
-| 7 | `workflows/scripts/<label>-wf_<run-id>.js` | rewritten | snapshot |
-
-**Workflow scripts carry their run id in the filename**, which is what makes them
-collectable despite living outside their session's directory:
-
-```
-<label>-wf_<run-id>.js session from the path, run id from the name
-```
-
-Split on the **LAST** `-wf_`: a run id contains hyphens (`wf_afa31e47-f6c`), so a greedy
-match starts at the first occurrence and swallows any later one. Verified on a real corpus:
-172/172 filenames parse, and 172/172 run ids resolve to a manifest in their own session.
-
-A script is JavaScript, not JSON, so it lands with `state: "raw"` - wrapped as a JSON string
-so the landed file stays parseable, with the bytes recoverable exactly.
-
-**`meta.json` is write-once.** 200 of 200 sampled meta files are *older* than their own
-`agent-<id>.jsonl` — zero equal, zero newer. Its content is spawn-time only (`agentType`,
-`description`, `spawnDepth`, `toolUseId`, `parentAgentId`, `model`); no status, result or duration.
-There is **no registry file** — each agent owns its own sidecar, so nothing is ever removed from a
-list. Pairing is exact: 2,697 meta ↔ 2,697 jsonl, zero orphans either direction.
-
-**`journal.jsonl`** records `{"type":"started","key":"v2:<sha>","agentId":…}` and `{"type":"result",…}`
-— 389 started / 347 result in the sample, the shortfall being in-flight and killed agents. It is
-**workflow-only**; Agent-tool spawns and Skill-forks have no journal. And it **leads the filesystem**:
-a `started` — occasionally even a `result` — appears before the child transcript exists. Normal, not
-a gap.
-
-**No single kind carries spawn lineage.** Of 300 sampled meta files, 210 hold only
-`{agentType, spawnDepth}` — **70% of children carry no parent pointer in their own sidecar**, because
-they are workflow subagents. The link is distributed:
-
-| Mechanism | Parent side | Child side |
-| --- | --- | --- |
-| Agent tool | main: `tool_use{name:"Agent"}` + `toolUseResult.agentId` | `meta.json.toolUseId` — redundant, so cross-checkable |
-| Skill fork | main: `toolUseResult.agentId`, `status:"forked"` | **nothing** |
-| Workflow | main: `Workflow` result → `toolUseResult.transcriptDir` + `runId` | `journal.jsonl` `started` records |
-| Depth 2 | — | `meta.json.parentAgentId` |
-
-So all six kinds are collected. The main transcript alone cannot enumerate workflow children; the
-child files alone cannot name the parent for 70% of children.
-
-#### Discovery is session-first, not project-first
-
-**A session's files scatter across several project directories.** Measured: **7 of 61 sessions span
-more than one slug**, one of them across six.
-
-The cause is **workflow scripts, not execution streams.** Claude Code files a run's script under
-whatever working directory the agent had at the time; the conversation data - main transcript, child
-streams, journals, manifests - stays in one directory. Before scripts were collected, those extra
-directories yielded nothing and were correctly not recorded. Now that scripts are collected the
-directories are real again, which is why the exclusion matcher keys on the session's PRIMARY
-directory: a script filed elsewhere must not disqualify the conversation it belongs to.
-
-```
-f9d8df3b-c47b-4f56-829c-37f4d81486cf
- transcript -Users-wusheng-github-skywalking-horizon-ui
- dir -Users-wusheng-github-skywalking-horizon-ui
- dir -Users-wusheng-github-skywalking-horizon-ui-test-e2e
- dir -Users-wusheng-github-skywalking-horizon-ui-apps-bff-src
- dir -Users-wusheng-github-skywalking-horizon-ui-apps-bff-src-store-audit
- dir -Users-wusheng-github-skywalking-horizon-ui-apps-bff-src-client-banyandb
-```
-
-Cause: a subagent running in a different working directory is filed under *that* cwd's slug, not the
-session's launch slug — the `cwd`-variance of §1.2 made visible in the filesystem.
-
-**Rule:** scan every project directory for UUID-shaped entries — both `.jsonl` files and directories —
-then **group by session id across slugs and union them**. A project-first walk misses most subagents
-for 11% of sessions.
-
-This also makes filters session-scoped: excluding a slug must not cut streams out of a session
-being collected, so `include`/`exclude` are evaluated as "does any part of this session match", never
-per-directory.
-
-Two further discovery rules from the same scan:
-
-- **Filter directory names by UUID shape.** `memory/` is a sibling directory inside project
- directories and is **not** a session. It is excluded.
-- **Pruning is real and not atomic.** 12 of 41 session directories have no main transcript — the
- `<session>.jsonl` was deleted while the whole `subagents/` tree survived. Expect child streams with
- no parent. Separately, 32 of 61 main transcripts have no subagent directory at all, which is simply
- a session that spawned no agents.
-
-#### Directory names are slugified cwd, and irreversible
-
-`/Users/wusheng/github/skywalking-horizon-ui` → `-Users-wusheng-github-skywalking-horizon-ui`.
-Verified across all 25 directories carrying a transcript; two mismatches revealed the full rule —
-`/`, `.` **and space** all collapse to `-`:
-
-```
-/Users/…/Application Support/JetBrains/GoLand2026.1/projects/…
- → -Users-…-Application-Support-JetBrains-GoLand2026-1-projects-…
-/private/tmp/agentsessionizer-cc-v04.psixvf
- → -private-tmp-agentsessionizer-cc-v04-psixvf
-```
-
-So a slug cannot be reversed to a path. **Forward slugification is deterministic; reverse is not** —
-which is why config accepts real paths and slugifies them to match, and never attempts the inverse.
-The real `cwd` is recoverable from record content, but it varies within a session (one session showed
-10,379 records at the project root plus Downloads, a scratchpad and several subdirectories), so it is
-not a session-level key.
-
-#### Configuration
-
-```yaml
-storage:
- root: ./data # beside the binary; tests point it beside their fixtures
-
-adapters:
- - name: claude-code-local
- enabled: true
- # source_root resolution: this value → $CLAUDE_CONFIG_DIR/projects
- # → $XDG_CONFIG_HOME/claude/projects → ~/.claude/projects
- source_root: ""
- # Entries starting with "/" are real paths, slugified forward for matching.
- # Anything else is a glob against the directory name. Evaluated per SESSION.
- include: [] # empty = all
- exclude:
- - "/private/tmp/**" # Claude Code's own agent scratchpads
- collector:
- mode: watch # watch | once
- interval: 5s
- max_delta_bytes: 8MiB
-```
-
-`claude-code-local` names both the runtime and the collection posture, leaving room for a future
-`claude-code-otlp` reading the same runtime by push. `collector` is an adapter attribute, not a
-global section, because cadence is a property of how a given adapter reaches its source.
-
-`mode: once` matters as much as `watch`: the first run over existing history is a backfill of ~1.2 GB
-across ~2,900 files, not a tail. Same code path, different exit condition.
-
-A `sources list` command is needed so nobody has to guess at slugs — printing slug, resolved `cwd`
-(from record content) and session count.
-
-#### The consuming workflow
-
-```
-┌─ DISCOVER (session-first) ────────────────────────────────────────┐
-│ walk all project dirs → UUID-shaped .jsonl files AND directories │
-│ group by session id ACROSS slugs → union │
-│ skip non-UUID dirs (memory/) │
-│ liveness: does ~/.claude/sessions/*.json name this session? │
-│ CHANGE TEST: append → stat().size vs cursor.offset │
-│ snapshot → content digest vs cursor.content_sha256 │
-└───────────────────────────┬───────────────────────────────────────┘
- │ sessions with ≥1 changed source
-┌─ PER SESSION (single-threaded) ───────────────────────────────────┐
-│ │
-│ VALIDATE ino ≠ cursor.ino → rotation → conflict, stop │
-│ size < cursor.offset → truncation → conflict, stop │
-│ file gone → source_gone (NOT an error) │
-│ else verify tail_sha256 │
-│ ▼ │
-│ TAIL seek(offset); read to the LAST COMPLETE '\n' │
-│ bytes past it = in-flight write → leave them │
-│ unparseable line mid-stream → conflict │
-│ ▼ │
-│ LAND seq = session.state.next_seq++ │
-│ write .tmp: header + records (payload verbatim) │
-│ fsync → rename → chmod 0444 │
-│ ▼ │
-│ COMMIT cursor → tmp → rename │
-└───────────────────────────────────────────────────────────────────┘
-```
-
-**Land before commit — deliberately at-least-once.** A crash between the two re-lands the same
-records next pass. That is safe *for free*, because the assembler must already drop duplicates by
-`(file, uuid)` for an unrelated reason (§1.2 — Claude Code itself writes duplicates). The reverse
-order would lose data. No write-ahead log is needed.
-
-**Change detection is `size` vs `offset`, never mtime.** One `stat()` per file; ~2,900 stats is
-milliseconds, and it needs no assumption about clock skew or timestamp granularity.
-
-**Source deletion is normal.** 330 of 345 session ids in `history.jsonl` have no surviving
-transcript. The cursor moves to `state=source_gone` and the landed deltas remain — so the landing
-zone outlives the source, and retains conversations Claude Code has already discarded.
-
-**Files appearing mid-pass is normal**, because the workflow journal leads the filesystem. Each pass
-re-discovers rather than caching a file list.
-
-#### Open: conflict recovery
-
-Rotation or a tail-digest mismatch means the source changed underneath us. Stopping and flagging is
-right; what follows is not settled. Proposed default: **re-land from offset 0 into a new stream
-generation** (`streams/<id>@2/`), so nothing is lost and nothing is silently merged. Evidence says
-this should never fire — transcripts look strictly append-only — but §1.9 lists that as unproven.
-
-## 3. Deferred
-
-**Phase 1 (local logs) — remaining plans:**
-
-- **Plan 02 — per-session pipeline:** duplicate uuids removed first, then fragment reassembly by
- `(file, message.id)` **in line order, never by ancestor walk** (§1.12a), stream partitioning, the
- three spawn joins, epoch boundaries via `logicalParentUuid`, segment gates, continuity audit
- scoped as a structural cross-check (§1.12c).
-- **Plan 03 — output:** conversation bundle, export projection, preview.
-
-**Phase 2 (OTLP) — additive, no Phase 1 rework:**
-
-- OTLP receiver, per-record demultiplex by `session.id`, landing into the reserved `log-` / `trace-`
- prefixes (§2.2).
-- Measurement only: per-call `duration_ms` and `cost_usd`, tool durations, `tool_decision`.
- No structural contribution (§1.7).
-- Metrics are a third, separate pipeline with its own retention. Nothing in metrics contributes to
- rebuilding a conversation.
-
-**Orthogonal opt-in, either phase:**
-
-- Raw provider bodies. The only route to a verified `input_manifest` and the only way to check the
- continuity invariant against what was actually sent (§1.8, §1.12d). Highest privacy risk in the
- system; must be opt-in with redaction.
-
----
-
-## 4. Open questions for discussion
-
-1. **Does the collector own an OTLP endpoint, or read files only?** Given §1.7 — no `agent_id` on
- events, and lineage spans unreachable without a beta flag — the OTLP channel now looks
- substantially *weaker* than transcripts for structure. It contributes timing and cross-validation.
- This weakens the case for a receiver in v1.
-2. **Should hooks be the second channel instead of OTLP?** §1.10 says hooks carry `agent_id` on
- every subagent hook, name the child transcript directly, and expose the full compaction lifecycle
- — everything the unreachable spans would have given us, without a beta flag. Cost: they are on
- the critical path and require editing the user's `settings.json`.
-3. **Is `OTEL_LOG_RAW_API_BODIES` in scope for v1?** It makes exact model-input membership reachable
- and needs no proxy. It is also the highest-privacy-risk surface in the system and must be opt-in
- with redaction.
-4. **Where does `<root>` live** — under `~/.claude/`, alongside it, or a configured path?
-5. **Retention:** per-session TTL, or unbounded until explicitly purged?
diff --git a/design-notes/02-parse-and-assembly.md b/design-notes/02-parse-and-assembly.md
deleted file mode 100644
index 2717c7a..0000000
--- a/design-notes/02-parse-and-assembly.md
+++ /dev/null
@@ -1,623 +0,0 @@
-# Design Plan 02 — Parse and Assembly
-
-**Status:** implemented. The pipeline is `internal/assemble`, the round chain is `pkg/sessionflow`, the
-round driver is `internal/parse`, and `asz parse` / `asz conversation` drive them.
-**Scope:** turning landed records into a conversation structure. Collection is Plan 01 and is
-implemented; export and preview are Plan 03 and are not designed.
-
-Every rule below is forced by a measurement. Where a rule looks arbitrary, the evidence that forced
-it is cited, because several of them are the opposite of the obvious choice.
-
-Numbers marked **(corpus)** were re-measured over the whole corpus — 2,970 files, 1.09 GB, 365,825
-records, 25 project directories — and each was then re-derived independently by a second pass that
-was told to refute it. Where the two disagreed, the number below is the one that survived. Several
-claims in the first draft did not.
-
----
-
-## 0. What parse consumes
-
-Parse reads the **index**, not the payloads.
-
-```
-365,554 index entries describing 1.0 GB of landed payload
-47 MB total index ~12x smaller
-16 MB / 4.5 ms largest single session, load + rebuild every lookup map
-```
-
-Structure resolution needs identifiers, not content: removing duplicate records needs record ids, provider-call
-grouping needs message ids, tool joins need tool-use ids, spawn joins need agent and run ids. None
-of it reads message text. Payloads are read only in the final materialisation step, and only for
-records that appear in the output.
-
-The index is derived and disposable; deleting it rebuilds from landed files.
-
----
-
-## 1. The pipeline
-
-Eight stages. The order is forced: each depends on the one before, and two of them are wrong in the
-obvious formulation.
-
-### 1.1 Remove duplicate records, keeping the first
-
-**Nothing downstream is correct before this.** Record ids are not unique within a source file.
-
-**(corpus)** 1,533 repeated ids among 318,199 distinct ids across 2,970 files. **Multiplicity is
-exactly 2, never 3**, so a single pass is provably enough — no iteration, no fixed point. They occur
-in 11 of 62 main transcripts and in **0 of 2,908 subagent transcripts**.
-
-**They are not a resume.** The first draft said they were. In 11 of 11 files the repeated run ends on
-the record immediately before a `system`/`compact_boundary` — every time, with no exception. This is
-the runtime re-writing a block of history just before it resets model context. It matters
-operationally: duplicate removal and epoch cutting touch the same lines, so **removal must run
-first**, and the epoch cut must not read the re-written block as new content.
-
-**First occurrence wins, and the reason is data, not order.** The later copy is the worse one:
-
-- `promptId` is rewritten on 538 of 1,533 pairs, collapsing onto the cycle that was active at reset
- time. Keeping the last would attribute those records to the wrong turn.
-- `toolUseResult` is present on one side of 536 pairs, and the first copy is larger in 430 of them
- and smaller in **zero**. 43.4% of captured tool output would be discarded, and in 314 pairs the
- later copy has `stdout` blanked to `""`.
-- `parentUuid` differs on 40 pairs, with both parents present in the file, so keeping the last
- changes the graph that stages 1.3 and 1.7 walk.
-
-The `message` object itself is byte-identical in 1,533 of 1,533 pairs. Only the envelope differs —
-which is exactly why this looks harmless and is not.
-
-**Nothing distinguishes an original from a copy except line order.** Timestamps are identical
-1,533/1,533, message ids identical where present, block ids identical. So "first" has to mean first
-in file order; there is no content test to fall back on.
-
-**44,585 of 365,825 lines carry no record id at all** — sidecar types keyed only by session. A pass
-that indexes by id must not treat a missing id as one colliding key. `internal/index` skips them:
-an entry with no identity is always its own record.
-
-> The index applies all of this. `Canonical()` is the deduplicated view, and a repeated copy is
-> excluded from **every** derived lookup, not only from id resolution.
-
-### 1.2 Partition streams
-
-`main` and one stream per agent id, already separated by the landing layout. A child stream **always
-shares the parent's session** — **(corpus)** 2,908 subagent files, zero exceptions, and no file
-carries more than one session id.
-
-**Stream membership comes from the file, never from a field.** `isSidechain` looks like the
-discriminator and is not: it is absent on 28.7% of main records, so a truthiness test misclassifies
-nearly a third of the parent lineage. Inside a subagent file it is a constant, so it carries no
-per-record information there either. It is not indexed.
-
-**(corpus)** Nesting stops at depth 2: 2,683 sidecars at depth 1, 31 at depth 2, one with the field
-absent — which must default to 1 rather than fail. The layout carries **no** depth information; all
-2,715 agent transcripts sit flat in two directories. Depth is only in the sidecar. All 31 depth-2
-agents come from one session, so any "sample N sessions" method has a good chance of seeing none and
-concluding wrongly that the maximum is 1.
-
-**A recursive walk is required.** A non-recursive listing of `subagents/` finds 107 files where a
-recursive walk finds 2,910 — a 96.3% miss, because most children live under
-`subagents/workflows/<run-id>/`.
-
-### 1.3 Assemble provider calls by `(file, message.id)` in **line order**
-
-**Never by walking `parentUuid` forward.** The graph is a DAG, not a chain: parallel tool dispatch
-forks it, so a forward walk traverses one branch and silently drops the other `tool_use` and its
-result.
-
-The ban is on the **direction**, not on the edge. Walking *backward* - from a record to its nearest
-ancestor carrying a given field - is single-valued and correct, because a record has exactly one
-containment parent. Stage 1.7 depends on that. Only forward traversal is ambiguous.
-
-**(corpus)** The forward walk drops a `tool_use` on 16,063 of 85,057 tool-bearing calls — 18.9%
-pooled, 2.1% on the parent lineage, 25.9% on child streams. The first draft explained this as
-parallel dispatch forking the graph. That is wrong: no fragment ever has two children sharing its
-message id, so there is no branch to choose. The real reason is that a call's fragments are
-**interleaved with their own tool results**, in the parent chain and in line order alike — 24.0% of
-multi-fragment calls have foreign records inside their line span. So grouping must be a full scan
-keyed by message id, and must not assume a call occupies a contiguous run of anything.
-
-Group by `message.id`, **not** `requestId`: a `<synthetic>` companion record can reuse a real call's
-`requestId`, which would put fabricated content inside a real call.
-
-Synthetic filter: `message.model == "<synthetic>"`. Not `requestId == null` — 24 of 97 synthetics
-carry a non-null `requestId`, and 2 genuine calls have none.
-
-**Usage: take the last fragment in LINE ORDER. Never sum, and never use the stop reason to find it.**
-
-The stop reason is not a terminal marker. **(corpus)** a main transcript stamps the same stop reason
-on every fragment of a call (23,324 of 23,329 groups); a child stream stamps it only on the last. A
-stop-reason test therefore picks the first fragment on main and the last on a child. Line order is
-the only rule that works on both.
-
-Summing is wrong for a different reason on each side. Main repeats the call's final usage on every
-fragment, so adding them multiplies the real number by the fragment count — **(corpus)** 2.18x
-pooled, 9.0x worst case. Child streams write streaming partials that climb 1, 2, 3, where the last
-value is the whole count and summing adds only 2%.
-
-**(corpus)** ~10% of calls never report a stop reason (0.02% main, 14.5% subagent). Two traps here:
-
-- Their usage block is **present and wrong** — output counts of 1 to 5, a streaming stub. 5,739 of
- 7,410 such calls have a last value of 5 or less, against 101 of 66,751 finished ones. So usage
- availability is gated on the stop reason, never on the field being present.
-- They are **not in flight**. Only 72 of 7,410 sit at the end of a file; the rest are mid-file with
- the conversation continuing past them, and 7,311 had their tools actually run. Nothing waits.
-
-Fragments per call: **(corpus)** 2.09 pooled, ranging 1.59 to 2.51 per session. Treating records as
-calls inflates every count by that factor.
-
-Synthetic records have a further signature: **(corpus)** all 122 have a UUID-shaped message id while
-all 184,330 real ones are prefixed `msg_`, with no overlap. So a fabricated record can never collide
-with a real call's id.
-
-### 1.4 Join tools
-
-`tool_use_id` → the index's tool lookup. **(corpus)** every `(file, tool_use_id)` maps to exactly one
-result once duplicates are removed. `tool_use` → `tool_result` is strictly one to one; a provider
-call to its tools is one to many, up to 16.
-
-**There is a second, fully redundant join the first draft missed.** Every tool result carries a
-top-level `sourceToolAssistantUUID`, and it matched the record holding the corresponding `tool_use`
-in **105,409 of 105,409 cases** — a direct record-to-record edge, and a free integrity check on the
-id join.
-
-**`srvtoolu_` is not a tool-use id.** The first draft said it was a second prefix to accept. It is an
-id *inside a WebSearch result payload*, one level below the tool call, and the assembler never joins
-on it. Nothing may match on a prefix either way.
-
-**Absence of `is_error` does not mean success** — it is present on 85.5% of result blocks.
-
-**`toolUseResult` is a parent-lineage field.** Present on 100% of main-transcript tool results and
-2.1% of child-stream ones, so everything read from it — captured output, and the spawn joins in §1.5
-— is implicitly a parent-lineage mechanism.
-
-**It is not always an object.** On roughly 7% of results it is a bare string. Decoding it into a
-struct fails, and because the failure covers the whole record, every identifier on that record is
-lost with it. It must be read as raw JSON and shape-checked.
-
-Execution timing is reported `unavailable`. "No local record" is slightly too strong: three tools do
-give a real duration (`WebFetch.durationMs`, `WebSearch.durationSeconds`, `Agent.totalDurationMs`)
-and could populate it for exactly those. A configured budget such as a timeout is not a measurement,
-and a timestamp delta includes model and harness latency, so neither is used.
-
-### 1.5 Join child agents to the calls that started them
-
-No single field spans them. The first draft ordered these by how obvious they look; the corpus
-orders them by how much work they do, and the two orders are almost reversed.
-
-| Mechanism | Join | Share **(corpus)** |
-| --- | --- | --- |
-| **Workflow** | the launch result's `runId`, plus `journal.jsonl` `started` records naming each child | ~96% |
-| **Agent tool** | the parent's `toolUseResult.agentId` | |
-| **Skill fork** | the same field with `status:"forked"`; there is no separate pointer to the call, so the tool id comes from the result's own content block | |
-| **Nesting** | the child sidecar's `parentAgentId` | 31 children |
-| | *together* | ~4% |
-
-**The sidecar is not a general parent pointer.** The first draft said 70% of children lack one; the
-real figure is worse and the conclusion different. A sidecar normally holds only `{agentType,
-spawnDepth}` — nothing about who started it. It names a parent for a *nested* child and nothing else,
-so it is a 3.3% edge case, not a primary join. Without `parentAgentId` a child of a child is
-attributed to the main stream, flattening all 31 depth-2 spawns.
-
-**The journal answers one question only:** which children belong to a run. It carries no timestamp,
-no name, no parent pointer and no tool id — just an agent id and a content hash. So the launch
-itself still has to come from the parent's own result, which names the run. That is how a workflow
-spawn becomes an edge between the call and the child, with no invented node standing for the run.
-
-**`status` has three values, not two.** `async_launched` is a launch acknowledgement and not a
-result. `forked` is a skill fork. `completed` is a **synchronous return, and it IS the result** — the
-first draft did not mention it, and treating it as an acknowledgement loses a real output.
-
-Completion arrives as a notification, and its two ids are not equally good. **`<task-id>` is a
-generic handle** whose namespace depends on what was launched — an agent id only ~8% of the time,
-otherwise a background-command id, a workflow task id, or a monitor id. Joining on it
-unconditionally matches the wrong thing most of the time. `<tool-use-id>` is exact and is the primary
-key; the call it names already has a spawn edge, so resolving through the call works wherever the
-task id does not. A single notification can also carry **several** `<task-id>` elements.
-
-Two further cautions hold as written. The `<tool-use-id>` is **not always resolvable in the file it
-lands in** — a nested child notifies the main lineage while the call that started it lives in an
-agent file — so resolution searches the whole session. And `<status>` is **absent entirely** on a
-large class of notifications, so nothing may require it.
-
-**A child with no parent at all exists.** 99.93% resolve; two in the corpus do not. That is a state
-to record, not an assumption to assert — an orphan stream stays attached to the session and to no
-Talk, because guessing a parent is worse than saying none was found.
-
-**`agent.output` ownership is a policy, not an observation.** The first draft read as if the parent
-simply did not receive the child's output. It does: a `status:"completed"` result and a notification
-both carry a copy. The rule is that we *deliberately drop* the parent's copy and keep a reference,
-because otherwise a rendered conversation repeats every subagent's work inside its parent. The index
-marks those records so a renderer knows there is something to drop.
-
-`agent.output` belongs to the **child** stream. The parent gets a compact boundary that references
-it, never absorbs it — this is what stops a rendered conversation duplicating every subagent's work.
-
-### 1.6 Cut context epochs
-
-`system/compact_boundary` carries **`logicalParentUuid`**, an explicit pointer to the last message
-before the reset. **(corpus)** present on 27 of 27 boundaries and on no other record type anywhere.
-Epochs are never inferred.
-
-- **the boundary re-roots the chain.** Its own `parentUuid` is null on 27 of 27, so
- `logicalParentUuid` is the *only* link back. Anything walking parents across a reset stops dead
- without it.
-- boundary `uuid` == the `parentUuid` of the following `isCompactSummary:true` record, so the pair is
- joined by containment
-- the summary's timestamp is **always earlier** than its boundary's, by up to ~1.4 s — so an epoch is
- ordered by landed position and never by time
-- `logicalParentUuid` is **never** the immediately preceding line, so proximity cannot substitute
-- `preservedMessages.allUuids` may name records that exist nowhere on disk, so both it and
- `logicalParentUuid` have to be unresolved-capable
-- the summary carries a `promptId` but **no `origin`** — which is exactly the case §1.7 has to handle
- without an explicit trigger
-- child streams never reset — **(corpus)** 0 in 221,592 child records, so every child stream is
- exactly one epoch
-- searching for the literal text `compact_boundary` is unsafe: it appears far more often in message
- content than in real boundary records
-
-### 1.7 Build Talks and Runs
-
-This is the stage that produces the conversation a person reads, and the one with the least
-runtime support: neither Talk nor Run is a Claude Code concept. Both are derived, so the boundary
-rule has to be stated exactly.
-
-**Two runtime signals map onto one model concept. Conflating them is the trap.**
-
-`promptId` and `origin` are Claude Code's own undocumented fields. Neither is a model concept and
-neither may appear outside this adapter; the model has Talk, and nothing below it.
-
-| Side | | Meaning | Sampled session |
-| --- | --- | --- | --- |
-| runtime | `promptId` | one *prompt cycle* | 65 |
-| runtime | `origin.kind` | what triggered that cycle: `human` or `task-notification` | 36 / 29 |
-| **model** | **Talk** | a human trigger plus the notification cycles it caused | **36** |
-
-The identity is close but not exact:
-
-```text
-distinct promptId == count(origin.kind=="human") + count(origin.kind=="task-notification") + cycles with no origin at all
-```
-
-The third term is the correction. **A local slash command is a person acting and the runtime records
-no origin for it**, so a rule that required an explicit human marker would leave those cycles
-attached to whatever came before. The rule is therefore:
-
-> **A Talk starts on an external trigger, OR on a cycle where no record states a trigger at all.**
-> Everything else continues the interaction in progress.
-
-**(corpus)** the identity also only holds *after* duplicate removal: re-emitted records rewrite
-`origin`, so a splitter running before removal over-counts human triggers by up to 15%.
-
-**`origin` is not in one place.** It sits on the record and also on the attachment inside it —
-missing the second location loses 27.6% of all trigger markers. And the value is an open set: a third
-kind exists beyond the two obvious ones, so anything switching on exactly two values will meet
-something it does not expect.
-
-A background agent finishing and the parent resuming is mechanically a new prompt cycle, but nobody
-said anything — it is the same interaction continuing. This is exactly the
-`agent.call -> launch_ack -> child stream -> agent.output -> runtime.notification -> parent resumes`
-chain of §1.5, and it lives INSIDE one Talk.
-
-**Prompt cycle ids are not unique across streams.** A child stream is written under a cycle id that
-also appears in the parent, and one id was observed in three different child streams. Keying a Talk
-on the cycle alone therefore puts a child's records inside the parent's turn and leaves the child
-with no run of its own. The key is (stream, cycle).
-
-*Verification on a real 59,495-record session:* the parent lineage assembles to **307 Talks**, which
-is exactly the number of `origin.kind=="human"` records measured independently on the raw file, from
-336 distinct prompt cycles.
-
-**Membership is resolved by walking backward, never by line proximity.**
-
-`promptId` appears on `user` records only - assistant records carry none, verified across 115,184.
-But every assistant record reaches one through `parentUuid`:
-
-```text
-assistant records tested 400
-reached a promptId via parentUuid 400 (100%)
-hops needed min 1, median 2, max 7
-```
-
-**(corpus)** the walk must not be capped at a small constant: 140 of 44,973 records need more than 7
-hops. Attachments are ordinary hops in the chain — they carry a parent and are named as a parent by
-later records — so the walk must pass through them rather than skip anything that is not a message.
-
-So a record's Talk is found by walking back to the nearest ancestor carrying a cycle id, then reading
-that record's trigger. (The index exposes these as `Cycle` and `Trigger`; the runtime field names
-stop at the adapter.) Line-order proximity would agree in the simple case and diverge exactly where
-it matters — a forked chain, a re-emitted block, an interrupt that reattaches to an earlier record —
-putting records in the wrong turn.
-
-**A child stream is exactly one Talk**: the delegated prompt in, the final output out, nobody outside
-the agent involved. Child streams never reset context, so each is also exactly one epoch.
-
-This one is asserted from the stream's nature, not measured from the data. **(corpus)** child
-records carry `promptId` on 82,378 of 221,592 records but `origin` on exactly **one** record in the
-whole child corpus — so a child stream has many cycles and essentially no triggers, and the rule
-cannot be derived the way it is on the parent lineage. The implementation creates the single Talk up
-front and files every cycle in it as a Run.
-
-**Human turns that exist only as attachments must be recovered.** These are `queued_command` records
-with no message twin, and a reader following messages alone loses real human input — **(corpus)** a
-third of all human utterances live only here. They are filtered on `commandMode == "prompt"`; without
-the mode, completion blobs queued the same way are read as things the user typed.
-
-**They are not Talk starts.** **(corpus)** 886 of 918 are mid-turn steering absorbed into a cycle
-already running. They belong *inside* the enclosing Talk as human content, placed by ancestry — and
-no cycle id may be invented for them.
-
-**Ownership rules** from the model: `context.injection` is first-class, `message.synthetic` is never
-counted as agent output, `agent.launch_ack` is not a result, and `agent.output` is owned by the
-child stream and only referenced by the parent.
-
-### 1.8 Close segments — candidate, then gates
-
-A quiet period creates a **close candidate**, not a commit, because a long gap often falls inside a
-turn rather than between two.
-
-**(corpus)** the "~32%" of the first draft was one number answering two different questions. Both
-matter, and the gate depends on the second:
-
-| Gap threshold | Candidates | Mis-splits a prompt cycle | Mis-splits a **Talk** |
-| --- | --- | --- | --- |
-| 10 min | 697 | 23.67% | **38.74%** |
-| 15 min | 444 | — | 30.41% |
-| 20 min | 302 | 9.60% | 21.52% |
-
-The Talk figure is the one to gate against: splitting at a notification cycle still severs a Talk.
-There is nothing special about ten minutes — it is the worst point on the curve below fifteen. The
-threshold is a setting, recorded in the round header's policy version, and 20 minutes buys a large
-reduction in mis-splits for 57% fewer candidates. Whether provider-error records sit on the
-segmentation timeline moves the first column by five points, so that choice is stated too.
-
-**Time runs backwards sometimes.** **(corpus)** 0.315% of consecutive records step back, by up to
-4.06 hours, and provider-error records are the main cause: their timestamp is when the failed
-request began, not when the record was written. Anything ordering by time — the gap computation
-included — must clamp a negative delta rather than read it as an enormous gap.
-
-Gates before committing:
-
-| Gate | Why |
-| --- | --- |
-| `activity_boundary` | the idle gap is real |
-| `no_crossing_open_operation` | **no tool request whose result lands on the far side of the gap, and no child still running.** Without this a request could be frozen into a committed segment while its result arrives in the next one, and a committed segment is never rewritten |
-| `lateness_watermark` | no late evidence still expected |
-| `conversation_identity` | the conversation id is resolved |
-
-The second gate does real work rather than being a safety net: it is what makes "committed segments
-are immutable" safe to promise.
-
-**(corpus)** it is not about a request left dangling at the end of a file — that is 0 of 25,892 on
-the parent lineage, because the runtime always writes a result, including a failure. It is about a
-result arriving after the quiet period, which happens on 85 of 697 long gaps (12.2%). The two are
-different things and the first draft conflated them.
-
----
-
-## 2. Output structure — the round chain
-
-Implemented in [`pkg/sessionflow`](../pkg/sessionflow/). Its tests are the specification;
-what follows is why it has this shape.
-
-**A round is an append-only immutable delta, and the conversation is the fold of every round.**
-
-```text
-state₀ = empty
-stateₙ = apply(stateₙ₋₁, roundₙ)
-```
-
-The obvious shape — rewrite the whole structure every round — works against everything beneath
-it. Landed files append, cursors advance, the index appends. Rewriting a 15 MB structure every five
-seconds to add a handful of nodes is the one place the design would stop being incremental.
-
-Pure append does not work either, because **assembly revises what it already emitted**. A `tool`
-node lands with `result: unavailable` and gains its result later. A provider call gains fragments.
-An `agent.call` gains its child stream once the child transcript appears — and the workflow journal
-names children before their transcripts exist, so this is routine, not an edge.
-
-The resolution is that a revision is a *new line in a later round*, never an edit to an earlier one.
-That is what lets a round be digested, archived or shipped the instant it is written.
-
-### 2.1 Layout
-
-```text
-data/_conversations/<conversation-id>/
- conversation.state mutable head pointer
- rounds/r000001-<digest12>.sf immutable, 0444
- rounds/r000002-<digest12>.sf
-```
-
-The split is deliberate. **Rounds carry nothing mutable and nothing temporal**; `conversation.state`
-carries the head round, its digest, how far landed evidence has been consumed, and when. Rounds are
-linked by digest, not by filename: round N names the digest of round N−1, so a holder of round 3
-that has not seen round 2 can store it but cannot apply it.
-
-### 2.2 Frames
-
-```jsonl
-{"t":"header","schema":"sf/1","conversation":"C1","session":"S1","round":2,"previous":"<r1 digest>",
- "from_seq":46,"through_seq":57,"input_digest":"<chained landed digest>","parser":"v1","policy":"v1"}
-{"t":"node","id":"main/t8","revision":2,"kind":"talk","stream":"main"}
-{"t":"node","id":"main/t8/r1/c3","revision":2,"kind":"llm.call","parent":"main/t8/r1",
- "ref":{"seq":12,"row":340},"refs":[{"seq":12,"row":340},{"seq":12,"row":341}]}
-{"t":"relation","id":"rel/starts/main-t8-r1-c3/stream-a4646a5c9","revision":2,"type":"starts",
- "from":"main/t8/r1/c3","to":"stream/a4646a5c9","quality":"exact_unique","via":"toolUseResult.agentId",
- "evidence":[{"seq":12,"row":345}]}
-{"t":"unresolved","id":"unres/tool_result/toolu_01Y","revision":2,"kind":"tool_result",
- "ref":"tool/toolu_01Y","state":"open"}
-{"t":"commit","digest":"<sha256 of lines 1..n-1>","counts":{"nodes":1284,"relations":96,"unresolved":3}}
-```
-
-The commit digest covers every line before it, so a round verifies itself: recompute over lines
-1..n−1 and compare. It is also the value the next round names as its `previous`.
-
-### 2.3 The four rules
-
-**1. Entity ids come from stable evidence, never from position.** A node keyed by "the 14th record in
-the stream" gets a different key the moment an earlier source is backfilled — and a changed key
-cannot supersede its own earlier revision, so the fold would hold both, one stale and one current,
-with no way to tell them apart. Ids are built from the runtime's own identity for a thing, scoped by
-session and stream; where the runtime supplies no identity, from the landed location, which is
-itself immutable once written.
-
-**2. Last writer wins, per id, whole-entity.** A later revision replaces the earlier one entirely.
-Not field-by-field: a partial merge cannot express "this attribute is now absent", so a correction
-that removes something could never be recorded.
-
-**3. Absence means unchanged.** An entity missing from round 7 is exactly the entity round 4
-published. Removal is therefore explicit — a revision with `tombstone` set — and never inferred from
-silence. Likewise an unresolved entry that gets resolved is *superseded* with `state:"resolved"`, not
-deleted: absence would be indistinguishable from "never noticed".
-
-**4. Order is chain order, not timestamp order.** Rounds fold in round order, entities within a round
-in file order. Timestamps come from the observed runtime and can go backwards across sources; the
-chain cannot.
-
-### 2.4 Determinism, and what it forced
-
-The property worth protecting is: *same previous digest + same landed range + same parser and policy
-versions ⇒ same round bytes and same digest*. Without it a chain can only be trusted, never
-re-derived. Three things in the earlier draft broke it, and each had to change:
-
-- **`revision` was a per-entity counter.** Deriving it required replaying the chain, and two
- producers reaching the same state by different paths would disagree. It is now **the round number
- that produced the entity** — readable straight off the header, and identical for any producer of
- that round.
-- **The header carried a timestamp.** Wall-clock time cannot be reproduced, so it would put every
- re-derivation in disagreement with the original. Time moved to `conversation.state`, outside every
- digest.
-- **`input_digest` was described as a digest over all evidence read so far.** Recomputing it would
- make each round cost proportional to the whole conversation — a long-running one would slow down
- without bound, which is the exact property an incremental design exists to avoid. It is now
- chained: `H(previousₙ₋₁ ‖ sorted digests of what round n consumed)`, proportional to what is new
- and still binding every round transitively to all evidence before it. Inputs are sorted because
- directory iteration order is not stable.
-
-### 2.5 What verification checks
-
-`Chain.Verify` walks the chain and checks four things, each catching a failure the others miss:
-
-| Check | Catches |
-| --- | --- |
-| rounds are 1..N with no gap | a missing round — its successors are unverifiable without it |
-| each round names its predecessor's digest | a fork, or an edited round |
-| each round's digest matches its bytes | tampering or truncation (`Read` does this) |
-| `from_seqₙ == through_seqₙ₋₁ + 1` | **skipped landed evidence — no digest reveals this** |
-
-The last is the one worth stating separately. Digests prove the rounds are the rounds; only the
-sequence check proves they read everything.
-
-### 2.6 What shipping does not solve
-
-A round references content by `{"seq":…,"row":…}` into landed files. A receiver holding only rounds
-can render the structure but not the text. Either the landed files travel too, or the export inlines
-content for the nodes it describes — an export contract decision, deferred to Plan 03.
-
-**Content is referenced, never inlined**, for the same reason: storing each call's input would
-multiply storage by conversation depth — roughly 220 MB for a session whose actual content is 10 MB.
-
-**Input manifests are deltas.** `base` + `appended`, which is exactly the continuity invariant, so
-the representation *is* the claim. A full manifest appears only where the chain genuinely breaks:
-compaction, fork, stream start. Those bases coincide with epoch boundaries.
-
-**Containment is single-parent; causality is a sparse typed relation** carrying `via` and `quality`,
-so an exact id match is distinguishable from an inference.
-
----
-
-## 3. Continuous build
-
-Rounds and segments are independent axes: the round is the **append** unit, the segment is the
-**commit** unit. Many rounds accumulate while work continues; a segment closes only on an idle gap
-plus gates. A segment close is recorded as an ordinary round — one that publishes the segment
-boundary — not as a separate mechanism or a rewrite.
-
-Watermark chain, each stage trailing the last:
-
-```text
-assembled_seq <= indexed_seq <= next_seq - 1
-```
-
-where `assembled_seq == conversation.state.through_seq`. A crash between publishing a round and
-saving state leaves a round on disk that state does not mention, so **the filesystem is the
-authority**: recovery lists the rounds directory and takes the highest, rather than trusting the
-pointer.
-
-**A per-stream record-id index must persist across commits.** A resume replays records whose first
-occurrence may sit in an already-committed segment; without it, a replayed block lands as a
-duplicate node. (This is the same rule the derived index enforces in `internal/index`: first
-occurrence wins, and the duplicate is excluded from *every* lookup, not only from id resolution.)
-
-**The invariant that keeps this honest:**
-
-```text
-fold(rounds 1..N) == full parse of landed evidence 1..through_seqₙ
-```
-
-A full rebuild from the index is the reference implementation — simple and obviously correct — and
-the fold is checked against it. Where they disagree the fold is wrong, and the disagreement is a
-test failure.
-
-The earlier draft asserted something weaker and wrong: that revision N+1 is a *prefix extension* of
-revision N. It is not, and cannot be. Late evidence supersedes entities emitted rounds ago, which is
-the whole point; a replay reordering after duplicates are removed or a compaction re-rooting an epoch changes what an
-earlier round said. Prefix extension holds for the round *chain* — no round is ever rewritten — but
-never for the folded result.
-
-**Parser and policy versions are chain generations.** A change to either that alters meaning does not
-retroactively fix earlier rounds and must not silently mix with them: it starts a new chain. Both
-versions are in every header so a reader can detect the mix rather than fold through it.
-
----
-
-## 4. The unresolved catalog
-
-Recorded as data, with two disciplines that make it useful rather than noisy.
-
-**Transient vs permanent**, because both genuinely occur:
-
-| Reference | Kind |
-| --- | --- |
-| journal `started` names an agent whose transcript does not exist yet | transient **while the run is live**; once the run is over it is a pruned or lost file, so it is terminal |
-| `<task-notification>` `<tool-use-id>` absent from the file it landed in | transient — resolves elsewhere in the session |
-| `tool_use` with no `tool_result` | transient unless the stream ended. **(corpus)** 0 of 25,892 on the parent lineage and 7 of 79,418 on child streams, always the final line of a truncated file |
-| workflow child `started` with no `result` | **not an error** — **(corpus)** 10.24% of workflow children never get one |
-| `allUuids` entry existing nowhere on disk | terminal — held in context, never written |
-| `logicalParentUuid` unresolvable | terminal |
-| child stream nothing names as spawned | terminal — **(corpus)** 2 exist |
-| source in `source_gone` | terminal — pruned before collection |
-
-**A denominator.** "3 unresolved" means nothing; "3 of 8,045 tool_use ids" means something.
-
-A transient reference is promoted only on **terminal evidence** — a pruned source, a finished run, a
-runtime that wrote no such record. Never on elapsed rounds: "still open after N rounds" is a
-statement about how often we looked, not about the data. That is why `state` has three values
-(`open`, `resolved`, `terminal`) rather than a counter.
-
----
-
-## 5. Open questions
-
-1. **Does a context reset force a segment close?** I lean no — an epoch is a model-context lifetime,
- a segment is a commit window; treating them as one makes segments depend on the runtime's reset
- timing. There is a reason to revisit it: a reset is preceded by a block of re-emitted records, so
- the two events are already entangled at the record level.
-2. **Where on the threshold curve to sit.** The table in §1.8 is measured; the choice is not made.
- 10 minutes mis-splits 38.74% of Talks and 20 minutes mis-splits 21.52% at 57% fewer candidates.
- The default is 10 minutes because the gates catch what the threshold gets wrong, but that reasoning
- deserves a measurement of its own — how often the gates actually save a bad candidate.
-3. **Where a session's conversation id comes from** when an application supplies one, and how it is
- recorded so cold reactivation preserves it. Today `asz parse` uses the session id, which is the
- adapter's documented default and the only sound one.
-4. **How a viewer folds the chain.** The fold this document describes is the assembler's: apply every
- round in order and keep the result. A viewer needs the same answer but not necessarily the same
- path — it may want the state as of an earlier round, or the history of one entity across rounds,
- or to stream rounds as they arrive rather than read them all first. That is Plan 03 work and is
- deliberately not designed here; what Plan 02 guarantees is only that the rounds contain enough to
- support any of those.
-
-5. **Retention now spans two directories.** Landed files under `data/<session>/` and rounds under
- `data/_conversations/<id>/` age independently, and a round references landed records by
- `(seq, row)`. Pruning landed data while keeping rounds leaves a structure whose content cannot be
- read; pruning rounds while keeping landed data is safe but discards the parse. There is no policy
- for this yet, and the coupling should be made explicit before either directory gets a TTL.
diff --git a/design-notes/03-read-and-export.md b/design-notes/03-read-and-export.md
deleted file mode 100644
index 14418b5..0000000
--- a/design-notes/03-read-and-export.md
+++ /dev/null
@@ -1,626 +0,0 @@
-# Design Plan 03 — Read and Export
-
-**Status:** sections 0-4 are implemented. Section 5, the viewer, is designed and
-not built.
-**Scope:** getting a conversation out of the round chain and in front of a reader. Collection is
-Plan 01 and assembly is Plan 02; both are implemented and neither changes.
-
-The plan changed shape once already. It began as "export and preview" and the first real question -
-what does a viewer read? - answered itself immediately: it reads the round chain, because a second format would
-only drift from the first. The interesting question turned out to be the one behind it, in
-[section 3](#3-the-contract-is-the-structure-file-and-that-exposes-a-hole): the model is
-runtime-agnostic about structure and not about content, and a viewer is the first thing that would
-notice. [Section 4](#4-two-formats-sf-and-sd) proposes the two formats that fix it.
-
-Every number below was measured on the same corpus Plans 01 and 02 were measured on: 62 sessions,
-1,100.1 MB of landed records, 319,264 assembled nodes. Where a number is missing it says
-`unmeasured` rather than an estimate.
-
----
-
-## 0. Why this plan exists
-
-**Nothing has read the output yet.** Plan 02 produced 319,264 nodes across 62 sessions and no reader
-has looked at one. That matters more than it sounds: most of the model's decisions were justified by
-a claim about reading — a tool use is one step because *a reader thinks of it as one event*, a child
-agent's output belongs to the child because otherwise *a rendered conversation shows the same work
-twice*, a delegation stays inside one Talk because *a reader sees one interaction*. Those are
-falsifiable claims about reading, and only a reader can falsify them.
-
-So this plan's real output is not a viewer. It is the set of decisions a viewer forces.
-
-### Three things a reader needs that the chain does not carry
-
-**1. There is no time anywhere.** Measured: 0 of 319,264 nodes carry any temporal field.
-
-```text
-100% id · kind · revision
- 99% parent · ref · stream
- 64% attrs
- 0% anything temporal
-```
-
-This is not an oversight. A round is deliberately time-free so that the same evidence reproduces
-the same bytes and the same digest. But a node's *observed* time — when the runtime says the thing
-happened — comes from a landed record that is immutable, so it is just as reproducible as the
-node's parent or kind. It could be carried; it is not. A viewer cannot draw a timeline, group by
-day, or say "twenty minutes later" without opening every referenced record.
-
-**2. A Talk has no name.** A reader opening a session sees a list of ids. The largest session has
-922 of them. Nothing in the chain distinguishes one from another.
-
-**3. Nothing resolves a reference to bytes.** A node says `{"seq":1,"row":1004}`. The only thing
-that turns that into text is `asz show`, which prints one record. There is no path from a folded
-conversation to what anybody said.
-
-### And one the maintainer raised
-
-**A viewer's fold is not the assembler's fold.** The assembler applies every round in order and
-keeps the result. A viewer may want the state as of an *earlier* round, or the *history* of one
-entity across rounds, or to render rounds as they *arrive* rather than reading them all first. The
-rounds hold enough to answer any of these. Which of them to serve, and how, is undecided.
-
----
-
-## 1. Where the bytes are
-
-The whole corpus, by what the bytes actually contain:
-
-| MB | share | what |
-| ---: | ---: | --- |
-| 370.0 | 33.6% | scaffolding — record ids, usage blocks, envelope |
-| 334.8 | 30.4% | tool response, content block |
-| 146.4 | 13.3% | **thinking signatures** |
-| 83.8 | 7.6% | tool response, **a structured duplicate of the same output** |
-| 75.0 | 6.8% | tool command |
-| 58.7 | 5.3% | injected context |
-| 18.4 | 1.7% | message text |
-| 14.3 | 1.3% | assistant text |
-| 0.1 | 0.0% | **thinking text** |
-
-Four readings of that table, each of which constrains the contract:
-
-**Tool responses are 38.1% and are 6× the size of the commands.** Asking what a tool *did* is cheap;
-carrying what it *returned* is the whole problem.
-
-**What a person came to read is 3.0%.** Message text plus assistant text is 32.7 MB of 1,100 MB. The
-conversation as read is a rounding error on the conversation as stored.
-
-**13.3% is cryptographic signature on thinking blocks whose text is 0.1 MB.** A signature exists so
-a provider can verify a block handed back to it. It means nothing to a reader, and it is the second
-largest single category in the corpus.
-
-**7.6% is the same tool output stored a second time.** A main-stream tool result carries both the
-content block and a structured form with `stdout`/`stderr` split out. Both are real — the structured
-one has detail the block lacks — but a reader wants one of them.
-
-### The share is not stable, which rules out a fixed policy
-
-Per session, over the 28 sessions larger than 1 MB:
-
-```text
-tool-response share min 17.3% median 35.1% max 83.5%
-```
-
-A rule tuned to the median is wrong by a factor of two at both ends. Whatever the contract does
-about tool output, it cannot be a constant.
-
-### Compression is not the answer either
-
-| | size | ratio |
-| --- | ---: | ---: |
-| largest session, landed | 200 MB | — |
-| zip / gzip | 68 MB | 2.8× |
-| zstd -19 | 45 MB | 4.3× |
-| **its round chain** | **21 MB → 1.4 MB** | **15×** |
-
-Rounds compress fifteen times because they are tens of thousands of near-identical structured lines.
-Landed content does not, because a third of it is high-entropy signature and captured output.
-
-Removing the two known-wasteful categories *before* compressing beats compressing harder: 45.2 MB →
-26.9 MB, a 40% cut that no compression level recovers on its own.
-
----
-
-## 2. What a reader asks for
-
-A Talk is the natural unit — it is the thing a reader opens. Measured across all 4,941 of them,
-structure only, no content:
-
-```text
-one talk's subtree p50 12.7 KB 42 nodes
- p90 43.4 KB 140 nodes
- p99 111.1 KB 348 nodes
- max 413.8 KB 1,475 nodes
-```
-
-The median talk's structure fits in a network round trip. The whole largest session is 16 MB of
-structure — too much for a first screen, fine for a background load.
-
-That asymmetry is the shape of the answer: **structure is small enough to be eager, content is not.**
-
-### 2.1 Nothing needs to be cached
-
-A design review set the threshold: if rendering a p99 talk takes longer than 200 ms, build an index
-for it; otherwise never. Measured on the largest real session, 53,106 nodes and 922 talks:
-
-| | |
-| --- | ---: |
-| fold the whole chain | 302 ms |
-| build and order the talk list | 1 ms |
-| **one talk's subtree, p99** | **13 µs** |
-| walk every subtree in the session | 11 ms |
-
-So: fold on demand, cache nothing. Four independent designs each proposed a derived read artifact -
-a bundle, a view directory, four projections, a locator index - and all four were specified before
-anyone had rendered a single talk.
-
-The first measurement said p99 was 104 ms, which looked like it might justify one. It was
-`View.Children` scanning all 53,106 nodes on every call, so a 42-node subtree cost 2.2 million
-comparisons. A parent-to-children map built once took the same walk from 22 seconds to 11
-milliseconds. The lesson is worth keeping: **measure the naive path before designing around it**,
-because the first number may be measuring the implementation rather than the problem.
-
----
-
-## 3. The contract is the structure file, and that exposes a hole
-
-**A viewer reads the round chain. It does not read landed files, and it does not read a format
-invented for viewers.** The chain already is the read contract: it carries the structure, it references the
-content, and it is verifiable on its own. Inventing a second shape for viewers would mean two things
-to keep in step, and the second would be the one that drifts.
-
-But the chain references content; it does not describe it. And that is where the layering breaks.
-[Section 4](#4-two-formats-sf-and-sd) is the answer that came out of it.
-
-### 3.1 What is runtime-agnostic today, and what is not
-
-The project's rule is that a runtime's vocabulary stops at its adapter. Measured against the code,
-that is half true:
-
-| | who interprets it | runtime-agnostic? |
-| --- | --- | --- |
-| **structure** | the adapter maps its identifiers onto index roles | **yes** |
-| **content** | nobody — the payload is the source bytes, untouched | **no** |
-
-Landing preserves payloads byte-for-byte, on purpose: it is what makes a landed record replayable
-and its digest meaningful. The consequence is that the only description of what is *inside* a
-payload is the runtime's own schema.
-
-So a viewer rendering one `message.assistant` node has to do this:
-
-```text
-node.ref {seq:1, row:1004}
- -> landed record
- -> payload.message.content[0].text <- Claude Code's shape, not the model's
-```
-
-Nothing stops it, and nothing tells it that walking `.message.content[]` is an adapter's business.
-The first viewer that ships would hard-code one agent product's JSON into the reader, and the
-model's whole claim to be runtime-agnostic would be true only of the parts nobody looks at.
-
-**There is also no adapter interface at all.** `internal/adapters/claudecode` is a package of free
-functions that `cmd/asz` calls by name. Nothing states what an adapter must provide, so a second one
-has nothing to implement against.
-
-### 3.2 Two axes, currently collapsed into one
-
-A landed header carries a single `adapter` string, `claude-code-local/0.1.0`. It is doing two jobs:
-
-- **transport** — how records arrive: reading local files, or receiving a push
-- **dialect** — how records are interpreted: Claude Code's schema, Codex's, another agent's
-
-Those vary independently. A push receiver for Claude Code shares every interpretation rule with the
-local reader and no acquisition code. A local reader for Codex shares the acquisition shape and no
-interpretation rules.
-
-```text
- dialect: claude-code dialect: codex dialect: …
-transport: local implemented — —
-transport: push — — —
-```
-
-Keeping the *transport* in the adapter name is right and already documented: a push adapter is a
-separate entry rather than a mode flag, because the collection posture changes what can be promised
-about completeness. Keeping the *dialect* there too is what makes the table above impossible to
-fill in without duplication.
-
-### 3.3 The surface an adapter has to provide
-
-Three responsibilities, of which only the first two exist and only one is named:
-
-| | today | needed |
-| --- | --- | --- |
-| **acquire** — get records into the zone | `claudecode.Collector`, called directly | a transport interface; pull and push are two implementations |
-| **interpret structure** — identifiers onto roles | `claudecode.IndexEntry`, a free function | a dialect interface, dispatched by the landed header |
-| **interpret content** — a payload into something readable | *missing* | a dialect method returning a runtime-agnostic content unit |
-
-The third is the one a viewer is blocked on. Its shape is constrained by things already measured:
-
-- It must be able to return **part** of a payload. A single tool result runs to 1.1 MB, and the
- tool-response share of a session varies from 17.3% to 83.5%, so truncation cannot be a caller's
- afterthought.
-- It must report **content state**, not just bytes. `available`, `truncated`, `redacted`,
- `unavailable` are already in the model; a reader has to be told which one it got, or the model's
- honesty rule stops at the last hop.
-- It must be addressable by exactly what a node carries: a landed record and an optional block
- ordinal. Nothing else is in scope, because nothing else is in a round's reference.
-
-An adapter that cannot interpret a payload it produced is still useful — the structure is already
-assembled — so this is `unavailable` rather than an error, and a reader is told the content cannot
-be described rather than shown a guess.
-
-### 3.4 What this changes about the plan
-
-The viewer is no longer the first thing to build, and it was never the point. **The adapter contract
-is,** because a viewer written before it exists will encode Claude Code into the reader, and the
-second agent product will find that out the expensive way.
-
-It also reframes what Plan 03 is for. It is not "export and preview". It is the point at which the
-model has to be honest about content the way it is already honest about structure.
-
----
-
-## 4. Two formats: `.sf` and `.sd`
-
-The names say what each is for. **Session Flow** is the structure — how the conversation went.
-**Session Data** is the detail — what was actually in it.
-
-```text
-data/<session>/streams/main/
- transcript-<ts>-000001.jsonl landed the source bytes, byte-identical, 0444
- transcript-<ts>-000001.sd derived the same records, normalised
-
-data/_conversations/<id>/
- rounds/r000001-<digest>.sf published the structure, immutable, digest-chained
- conversation.state cache the head pointer
-```
-
-`.sf` is what was called `asb`. Nothing about it changed but the name and the schema string.
-
-### 4.1 Why a second file rather than a read-time conversion
-
-The alternative is to leave landed payloads as they are and have the dialect interpret them each
-time something reads one. That was the shape of §3.3, and it is worse in three ways.
-
-**The conversion is already happening.** Collection parses every payload to build the index. Doing
-it again per read is the same work, an unbounded number of times.
-
-**The reader cannot be trusted to stay clean.** A dialect method that returns content on demand is a
-rule that has to be *obeyed*. A normalised file on disk is a rule that is *enforced* — there is no
-runtime shape to reach for, because the reader never receives one.
-
-**The dialect surface is bigger than it looks.** Measured: `toolUseResult` takes **49 distinct key
-sets** across the corpus and is a bare string 7.2% of the time. Bash returns
-`stdout`/`stderr`/`interrupted`, Edit returns `structuredPatch`, Workflow returns
-`runId`/`taskId`/`transcriptDir`. That is a per-tool schema, and it is exactly what must not reach a
-reader.
-
-The cost of a second file is that it stores content twice. That is acceptable only because **`.sd`
-is derived and disposable**, in the same sense the index is: raw payloads stay, so a wrong
-normalisation is a rebuild, not data loss. Nothing may treat `.sd` as authoritative.
-
-### 4.2 The vocabulary is small because the industry settled it
-
-Every content block in 3,032 files, 1.1 GB:
-
-| count | Claude Code block | keys |
-| ---: | --- | --- |
-| 108,243 | `tool_result` | `content` `is_error` `tool_use_id` |
-| 108,149 | `tool_use` | `id` `name` `input` `caller` |
-| 60,382 | `thinking` | `thinking` `signature` |
-| 22,458 | `text` | `text` |
-| 41 | `image` | `source{data, media_type}` |
-| 1 | `fallback` | `from{model}` `to{model}` |
-
-**Six types, four of which cover 99.99%.** This is the strongest evidence that a unified format is
-worth defining: the thing being unified is not a moving target. A message, a thought, a call, its
-result, and an attachment is the whole of what an agent does.
-
-### 4.3 What a `.sd` record carries
-
-One `.sd` per landed file, at the **same sequence number**. That is deliberate: an `.sf` node
-already addresses content as `{seq, row, block}`, and keeping the sequence and the row makes the
-existing coordinate address the normalised form unchanged. No second addressing scheme.
-
-```jsonl
-{"h":1,"schema":"sd/1","dialect":"claude-code/1","seq":1,"session":"…","stream":"main",
- "of":"transcript-20260901T003722Z-000001.jsonl","of_sha":"<digest of the landed file>"}
-{"row":1004,"id":"<record id>","parent":"<its containment parent>","call":"<provider call>",
- "run":"<the agent loop>","ts":"2026-08-31T16:14:42.170Z","trigger":"external",
- "parts":[{"k":"text","text":"Two answers, and the second is the one that needs specifying…"}]}
-{"row":1005,"id":"…","parts":[{"k":"call","id":"toolu_01…","name":"Bash",
- "input":{"command":"make check"}}]}
-{"row":1006,"id":"…","parts":[{"k":"result","of":"toolu_01…","failed":false,
- "text":"ok github.com/…","state":"truncated","bytes":1118873}]}
-```
-
-Two halves, and the split is the point:
-
-- **record level** — the identifiers the structure is built from, in ROLE names. `id`, `parent`,
- `call`, `run`, `trigger`, `ts`. These are what `internal/index` already stores; `.sd` is where
- the adapter writes them down instead of the index deriving them from a runtime's fields.
-- **part level** — what the content IS.
-
-**`.sd` says what the bytes are. `.sf` says what role they play.** A `part` of kind `text` might be a
-person's turn, an agent's answer, or an injected reminder — which of those it is comes from the
-structure, and `.sd` does not guess.
-
-### 4.4 Part kinds
-
-| kind | is | carries |
-| --- | --- | --- |
-| `text` | readable text | `text` |
-| `reasoning` | the model's own reasoning | `text` |
-| `call` | a request to run something | `id` `name` `input` |
-| `result` | what it returned | `of` `text` `data` `failed` |
-| `media` | an image or document | `media_type` `data` or a reference |
-| `unknown` | the dialect could not describe it | the raw location, `state:"unavailable"` |
-
-Every part also carries `state` (`available` · `truncated` · `redacted` · `unavailable`) and
-`bytes`, the size of the original. Those two are not decoration: one tool result reaches 1.1 MB and
-the tool-response share of a session runs from 17.3% to 83.5%, so a reader is routinely shown part
-of something and has to be told so.
-
-`unknown` is what keeps the format honest. A dialect meeting a shape it does not recognise records
-where the bytes are and says it could not describe them — the same rule the rest of the model
-follows, rather than a guess or a silent drop. The attachment list this project just removed failed
-exactly here: it dropped 635 records it did not recognise.
-
-### 4.5 The mapping, Claude Code to `.sd`
-
-Record level:
-
-| Claude Code | `.sd` | note |
-| --- | --- | --- |
-| `uuid` | `id` | |
-| `parentUuid` | `parent` | |
-| `message.id` | `call` | never `requestId` — a fabricated record can reuse a real one |
-| `promptId` | `run` | one loop the agent performed for one trigger |
-| `timestamp` | `ts` | |
-| `origin.kind`, else `attachment.origin.kind` | `trigger` | reading only the first loses 27.6% |
-| `logicalParentUuid` | `continues` | the record a new context resumes from |
-| `toolUseResult.toolUseId`, or a notification's tool id | `tool` | the tool use this record is about |
-| `toolUseResult.agentId`, a journal `agentId`, a notification's task id | `child` | the child stream this record names |
-| the `wf_…` directory, or `toolUseResult.runId` | `batch` | children started together; a join key, with nothing in the model for it |
-| `isCompactSummary`, `subtype`, `isMeta`, … | flags | as the index already maps them |
-| `requestId`, `agentId`, `sessionId`, `cwd`, `gitBranch`, `slug`, `version`, `userType` | — | not carried; identity is the file's, or unused |
-
-Part level:
-
-| Claude Code | `.sd` part | note |
-| --- | --- | --- |
-| `content[i].type == "text"` | `text` | |
-| `"thinking"` → `.thinking` | `reasoning` | **`.signature` is dropped** — 146.4 MB, 13.3% of the corpus, and the reasoning text it accompanies is 0.1 MB. It is material a provider verifies, not something a person reads. |
-| `"redacted_thinking"` | `reasoning`, `state:"redacted"` | |
-| `"tool_use"` / `"server_tool_use"` | `call` | `caller` is `{"type":"direct"}` on all 108,149; carried as `unknown` if it ever is not |
-| `"tool_result"` → `content`, `is_error` | `result` | absence of `is_error` does not mean success |
-| `"image"` → `source{data, media_type}` | `media` | |
-| `"fallback"` → `from`/`to` model | `unknown` | one occurrence; a control event, not content |
-| `message.content` as a bare string | one `text` part | 4,429 records |
-| `toolUseResult` object | merged into the `result` part | **49 key sets**; the readable output is `stdout`/`stderr`/`content`, the rest is per-tool structure that does not survive normalisation and is left in the landed record |
-| `toolUseResult` string | `result.text` | 7.2%; would break a struct decode |
-| `attachment.text` / `.content` | `text`, or `unknown` | 28 attachment types, heterogeneous |
-
-**An MCP call is a `call`.** Measured: 39 of 26,940 tool uses, structurally identical to any other —
-same block type, same keys, same `toolu_` id scheme, same result shape. The server is a prefix on
-the name, `mcp__<server>__<tool>`, which is an attribute and not a kind. The sample is thin, 0.1%,
-so this is stated as observed rather than as a rule.
-
-### 4.6 What else changes
-
-| | today | after |
-| --- | --- | --- |
-| `pkg/asb` | the round chain | **done** - `pkg/sessionflow`, schema `sf/1` |
-| `*.asb.jsonl` | round file | **done** - `*.sf` |
-| — | | `*.sd`, and a package that reads and writes it |
-| `internal/index` | built from raw payloads by the adapter | built from `.sd`, dialect-free |
-| the adapter | collect · index · (interpret content, missing) | **collect · convert to `.sd`.** Nothing else. |
-
-That last row is the whole point. Once `.sd` exists, the adapter's only remaining job is turning its
-runtime's bytes into the unified form. Everything above it — index, assembly, structure, rendering —
-reads `.sd` and has no dialect in it at all. **A second adapter is one converter.**
-
-Two things deliberately do NOT change:
-
-- **Landed files stay, byte-identical.** They are what `.sd` is derived from, what `asz verify`
- checks, and what makes a wrong normalisation recoverable.
-- **`.sf` stays immutable, time-free and digest-chained.** `.sd` is per landed file and therefore
- immutable by construction, so it needs no chain of its own.
-
-### 4.7 Both vocabularies, side by side
-
-The model strips runtime vocabulary, and that is right for storing a
-conversation and wrong for showing one. A person debugging their own session
-recognises `promptId`; telling them a "run" makes them translate in their head.
-A person comparing two agent products needs the opposite.
-
-So the adapter also declares a **glossary**: what its runtime calls each thing
-the model names. A reader chooses which words to see, and neither is presented
-as the other.
-
-```text
-asz conversation ID talk · run · llm.call · tool
-asz conversation -terms native ID talk · promptId · message.id · tool_use + tool_result
-asz conversation -terms both ID run (promptId)
-asz glossary the whole table
-```
-
-Measured on Claude Code: **59 terms, 36 named by the runtime and 23 derived
-here.** That ratio is the useful part. If everything had a native name the model
-would be a rename of one product rather than a model of any; if nothing did, it
-would have no evidence under it.
-
-Three rules keep the table honest, each enforced by a test:
-
-- **Every name the model can emit has an entry**, even if only to say the
- runtime has no word for it. The failure otherwise is silent: a reader asks for
- native terms, meets a name with no entry, and cannot tell "no native name" from
- "nobody wrote one down".
-- **An empty native name is an answer, not a gap.** A Talk, a Segment, a
- correlation quality - no agent product records them, and saying so beats a
- plausible word that sends a reader looking for a field that does not exist.
-- **A native name must say where to find it.** Being told `logicalParentUuid`
- without being told it sits on a `compact_boundary` record is half an answer.
-
-The glossary lives beside the extraction code, so a rename in the runtime
-changes both together. It is keyed by dialect (`claude-code/1`), not by adapter,
-because a push receiver for the same runtime would share every word of it.
-
-### 4.8 Open
-
-1. **Where the dialect is recorded.** `.sd` carries it in its own header, which settles it for
- reading. The landed envelope still conflates transport and dialect in one `adapter` string.
-2. **Whether the index survives.** If `.sd` carries the record-level identifiers, the index becomes
- a lookup structure over `.sd` rather than a separate extraction. It may still earn its place as a
- fixed-width binary form; that is a performance question, and `unmeasured`.
-3. **What `.sd` costs.** Dropping signatures and scaffolding should make it much smaller than the
- landed data, but the number is `unmeasured`. It decides whether `.sd` can simply replace landed
- files in an export.
-4. **Media.** 41 images, base64 inline. Inline in `.sd` is fine at that volume and would not be at
- another; no store is proposed yet.
-
----
-
-## 5. The viewer
-
-The reference is the v0.8 preview page, and the direction it sets is right: an
-**observability console**, not a chat log. A timeline across the top, a
-transcript beside it, and an inspector that answers *where did this come from*.
-That shape is what this model is for - a chat log would render the messages and
-throw away every relation, quality and unresolved reference the last two phases
-exist to produce.
-
-### 5.1 What the preview establishes
-
-| | |
-| --- | --- |
-| **Ground** | deep navy `#07111f`, surface `#0d1929`, lines `#263b55` |
-| **Text** | `#e9f0f8`, muted `#9cb0c7`, faint `#6f859f` |
-| **Accents** | blue `#58a8ff` · green `#35c980` · orange `#f4a641` · purple `#ac8cff` · teal `#31c7bd` · pink `#ef6eae` · red `#ff6f78` |
-| **Type** | Inter, with a system mono for identifiers |
-| **Layout** | toolbar · timeline canvas with a minimap and gap labels · transcript list · inspector |
-| **Inspector** | four tabs: Details · Relations · Evidence · Raw |
-
-Two details in it matter more than the styling.
-
-**It already shows qualification.** `unavailable` appears six times and
-`strong_inference` four. The preview was drawn before any of this was measured,
-and it was drawn honestly - a viewer that hides how a join was made would undo
-the phase that produced it.
-
-**It has a gap label on the timeline.** Which is the segment question rendered:
-a quiet period is drawn as a gap rather than silently closing one thing and
-opening another.
-
-### 5.2 What has changed under it
-
-The preview names things the model has since renamed, and asks for two it cannot
-have.
-
-| preview | now | note |
-| --- | --- | --- |
-| `agent_response` | `message.assistant` | |
-| `agent_instruction` | `message.external` | |
-| `tool_activity` | `tool` | |
-| `agent_call` · `agent.prompt` | `agent.call` | |
-| `agent.run` · `agent_run` | `run` | |
-| `control_trigger` | `trigger` on a Run | external or notification |
-| **`tool_execution`** | — | **does not exist.** The preview draws a tool as call → execution → result. Only two of those are recorded. Three tools report a real duration; for the rest it is `unavailable`, and a timestamp gap is not a substitute. |
-
-And three things it wanted are now available that were not when it was drawn:
-
-- **a title**, in 41 of 42 conversations
-- **a label on a child stream** - what that agent was asked to do
-- **usage per provider call**, and **`turn.duration`** measured by the runtime on
- 465 turns
-
-### 5.3 How it reads
-
-**Fold on demand. Cache nothing.** Measured on the largest real conversation:
-
-```text
-fold the whole chain 302 ms
-build and order the talk list 1 ms
-one talk's subtree, p99 13 µs
-```
-
-A design review set the threshold at 200 ms for building an index; the answer is
-thirteen microseconds. Every derived read artefact that four independent designs
-proposed - a bundle, a view directory, four projections, a locator - solves a
-problem that does not exist at this size.
-
-**Time comes from the index, not from the round.** A round is deliberately
-time-free so its bytes stay reproducible, and a timeline needs a time per node.
-Both hold: a node carries `{seq, row}`, the index carries the observed time for
-that position, and the viewer builds the lookup once when it loads a
-conversation. Nothing is added to a published round.
-
-### 5.4 The shape
-
-`asz view` - a local HTTP server on the standard library, no build step, no
-dependency, no database. It serves one page and a small JSON surface over the
-chain it already has.
-
-```text
-GET /conversations id, title, talks, span, size
-GET /c/{id} the folded view: talks, streams, segments
-GET /c/{id}/talk/{n} one talk's subtree, with times
-GET /c/{id}/record/{seq}/{row} one landed record, its parts
-```
-
-Four endpoints because the preview has four questions: which conversation, what
-happened in it, what happened in this turn, and what exactly did this step say.
-
-The inspector's four tabs come free from what is already stored:
-
-| tab | source |
-| --- | --- |
-| Details | the node's `kind`, `attrs`, and its resolved time |
-| Relations | every edge touching it, with `quality` and `via` |
-| Evidence | its `ref` and `refs`, and what they resolve to |
-| Raw | the `.sd` record: parts, flags, usage, and what was dropped |
-
-### 5.5 Order of work
-
-1. **The server and the transcript.** One conversation, its talks, one talk
- rendered with every step, quality badge and unresolved row visible. This is
- the slice that falsifies the most: whether a Talk is the unit a reader opens,
- whether quoting names one, and whether a wall of badges is readable.
-2. **The timeline.** Needs the time lookup and the gap labels, and answers
- whether segments as measured look right to a person.
-3. **The inspector.** Relations and Evidence are the phases 1 and 2 work made
- visible; Raw is the honesty check - a reader can always see what we stored.
-4. **Export.** Only after a viewer exists, because until then nobody knows what
- an export has to contain.
-
-### 5.6 What it must not do
-
-- **No chat-log rendering.** Dropping relations and quality would waste the model.
-- **No inferred timing.** A tool with no reported duration shows none.
-- **No hidden unresolved references.** They are the product of the honesty rule
- and belong on screen, not in a log.
-- **No cache before it is measured to be needed.** The numbers above say it is not.
-
-## 6. Open questions
-
-1. **Retention across two directories.** Carried over from Plan 02 and now sharper: a round
- references landed records, so pruning landed data while keeping rounds leaves a structure whose
- content cannot be read. A reader meeting that must be told the content is gone, not shown an
- empty one — which means content state has to reach the screen.
-2. **Reading a conversation that is still being written.** `unmeasured`. Collection and parse both
- run while someone could be reading.
-3. **Search.** Nothing indexes text today; the derived index holds identifiers only, deliberately.
-4. **Where the dialect is recorded.** A landed header carries one `adapter` string that names both
- transport and dialect. Interpretation needs the dialect alone. Deriving it from the name by
- convention works for the one adapter that exists and will not survive the second. Adding a field
- changes an envelope that is already on disk in immutable files, so the migration has to be
- thought through rather than assumed: the envelope is versioned (`h:1`), and landed files are
- never rewritten, so a reader would have to handle both. `unmeasured`: whether any existing
- landed data would need a compatibility path, or whether re-collection is acceptable.
-5. **Whether a runtime's own schema changes under us.** Claude Code's record shape is not a
- published contract and has already shifted during this project. Structure survives that - the
- adapter absorbs a rename - but content interpretation is a second surface with the same
- exposure, and nothing detects a shape change today. A dialect that meets a payload it does not
- recognise must report `unavailable` rather than guess, which is the same rule the rest of the
- model follows.
diff --git a/design-notes/README.md b/design-notes/README.md
deleted file mode 100644
index c93a6b1..0000000
--- a/design-notes/README.md
+++ /dev/null
@@ -1,10 +0,0 @@
-# Design Notes
-
-**These are working engineering notes, not official documentation.**
-
-They record measurements, dead ends, corrections, and open questions produced while designing
-SkyWalking AI Sessionizer against real agent-runtime data. They are deliberately unpolished and are not part
-of the published documentation set.
-
-Official documentation lives in [`../docs/`](../docs/) and is indexed by
-[`../docs/menu.yml`](../docs/menu.yml).
diff --git a/docs/en/adapters/claude-code-glossary.md b/docs/en/adapters/claude-code-glossary.md
index 8904aaa..5575eae 100644
--- a/docs/en/adapters/claude-code-glossary.md
+++ b/docs/en/adapters/claude-code-glossary.md
@@ -7,8 +7,10 @@ The dialect is **`claude-code/1`**. That names the vocabulary, not the adapter:
says how records were acquired, the dialect says how they are read. A push receiver for the same
runtime would be a different adapter and the same dialect.
-This table is generated from the code — `asz glossary` prints it — so it cannot drift from the
-mapping it describes. A test fails if the project's vocabulary grows a name with no entry here.
+The two tables below are copied from what `asz glossary` prints. The command reads the adapter's
+own declarations, so where this page and the command differ, the command is right. The
+[adapter contract](../concepts-and-designs/unified-conversation-model.md#adapter-contract) says
+what a test checks in those declarations.
```sh
asz glossary this table
@@ -17,16 +19,17 @@ asz conversation -terms native ID talk · promptId · message.id · tool_use +
asz conversation -terms both ID run (promptId)
```
-**59 terms: 36 that Claude Code names, 23 this project derives.** That ratio is worth watching. If
-everything had a native name the model would be a rename of one product rather than a model of any;
-if nothing did, it would have no evidence under it.
+**61 terms: 39 that Claude Code names, 22 this project derives.** That ratio is worth watching. If
+everything had a native name, the model would be a rename of one product rather than a model of
+any. If nothing did, it would have no evidence under it.
---
## Named by Claude Code
-Each row is a field the adapter actually reads. A rename in Claude Code changes this table and the
-extraction together, because they sit in the same file.
+Each row says where Claude Code records the thing. The adapter reads it in one of the places named,
+unless the note says it is not read. A rename in Claude Code changes this table and the extraction
+together, because they sit in the same package, `internal/adapters/claudecode`.
| model | Claude Code | where | note |
| --- | --- | --- | --- |
@@ -34,13 +37,14 @@ extraction together, because they sit in the same file.
| `agent.launch_ack` | `toolUseResult.status == "async_launched"` | a tool result | an acknowledgement, not a result |
| `batch` | `toolUseResult.runId` | a workflow launch result | also the name of the directory the batch's records are filed under |
| `call` | `message.id` | record | not requestId, which a client-fabricated record can reuse |
-| `cancels` | `toolUseResult.interrupted` | a tool result | |
+| `cancels` | `toolUseResult.interrupted` | a tool result | not read, so asz writes no cancels relation today |
| `child` | `toolUseResult.agentId` | a launch result, a journal record | |
| `context.injection` | `attachment` | an attachment record | |
| `continues` | `logicalParentUuid` | a compact_boundary record | the only link back; a boundary's own parentUuid is null |
| `control.command` | `system/local_command` | a system record | |
-| `control.interrupt` | `toolUseResult.interrupted` | a tool result | |
-| `control.permission` | `permission-mode` | a record type | |
+| `control.interrupt` | `toolUseResult.interrupted` | a tool result | not read, so asz emits no step of this kind today |
+| `control.notice` | `system/away_summary, bridge_status, informational, model_refusal_fallback` | a system record | anything the runtime says about the session itself, including every system subtype no other row names |
+| `control.permission` | `permission-mode` | a record type | not read, so asz emits no step of this kind today |
| `epoch.boundary` | `system/compact_boundary` | a system record | |
| `epoch.summary` | `isCompactSummary` | a user record | |
| `error.api` | `system/api_error` | a system record | |
@@ -51,22 +55,23 @@ extraction together, because they sit in the same file.
| `message.assistant` | `text block` | an assistant record | |
| `message.external` | `origin.kind == "human"` | a user record | also a queued_command attachment whose commandMode is "prompt" |
| `message.synthetic` | `message.model == "<synthetic>"` | an assistant record | the client wrote it; no model produced it |
-| `model` | `message.model` | an assistant record | <synthetic> on a record the client fabricated |
+| `model` | `message.model` | an assistant record | `<synthetic>` on a record the client fabricated |
| `notification` | `origin.kind == "task-notification"` | record | |
| `parent` | `parentUuid` | record | |
| `redacted` | `redacted_thinking` | a content block | |
| `reports` | `<task-notification>` | a user record's content | |
| `result_of` | `tool_use_id` | a tool_result block | also stated redundantly as sourceToolAssistantUUID |
-| `run` | `promptId` | record | one loop per distinct promptId |
+| `run` | `promptId` | record | one loop per distinct promptId in a stream. A model response carries none and reaches one through its parents |
| `runtime.notification` | `<task-notification>` | a user record's content | |
| `session` | `sessionId` | record, and the transcript filename | |
| `starts` | `toolUseResult.agentId, or a run journal` | a launch result | |
-| `stream` | `agentId` | a child record | the parent lineage is the session transcript itself |
+| `stream` | `agentId` | a child transcript's file name, `agent-<id>.jsonl` | the parent lineage is the session transcript itself |
| `summarizes` | `parentUuid` | an isCompactSummary record | |
| `thinking` | `thinking block` | an assistant record | |
| `time` | `timestamp` | record | |
| `tool` | `tool_use + tool_result` | two records | one step here, two blocks there |
| `trigger` | `origin.kind` | record, and attachment.origin.kind | |
+| `turn.duration` | `system/turn_duration` | a system record | the runtime measured it; it is not the gap between two record timestamps |
## Derived by this project
@@ -93,7 +98,6 @@ than a plausible word that would send a reader looking for a field that does not
| `strong_inference` | no identifier, but a well-founded structural argument |
| `talk` | one readable interaction; derived from where a run was triggered |
| `truncated` | part of the bytes, with the full length reported |
-| `turn.duration` | unavailable on the claude-vscode entrypoint |
| `unavailable` | the runtime never wrote it, or the source was pruned |
| `unknown` | no record of the run states a trigger |
| `unresolved` | no usable evidence |
@@ -111,7 +115,7 @@ The mapping runs both ways, and this direction matters for anyone going back to
| `agentId` | equals the stream name for a child stream; carrying it would store the same value twice. |
| `sessionId` | the landed file already says which session it belongs to. |
| `isSidechain` | absent on 28.7% of parent-lineage records, so it cannot say which stream a record belongs to. The file can. |
-| `cwd`, `gitBranch`, `slug`, `version`, `userType` | environment, not conversation. Still in the landed record for anyone who wants it. |
+| `cwd`, `gitBranch`, `slug`, `version`, `userType` | environment, not conversation. Not in the landed record. [A record keeps no source bytes](../formats/session-data.md#record), and no Session Data field holds these. The adapter reads `cwd` only to make an edited file's path relative. |
| `thinking.signature` | 13.3% of the corpus, wrapping 0.1 MB of reasoning text. A provider verifies it; a reader cannot read it. |
## What Claude Code does not supply at all
@@ -122,7 +126,7 @@ Reported `unavailable` rather than approximated. A second adapter might do bette
| --- | --- |
| tool execution timing | only `WebFetch.durationMs`, `WebSearch.durationSeconds` and `Agent.totalDurationMs`. Everything else has none, and a timestamp gap includes model and harness latency. |
| turn duration | absent on the `claude-vscode` entrypoint — 13 of 15 sessions sampled. |
-| a title or summary | for a session, a talk, or anything else. |
+| a title for a talk | the runtime names only the session, a child agent and a workflow. [Names](claude-code.md#names) says where. |
| retry identity | a retried attempt is not linked to the one it replaced. |
| the serialized provider request | never written locally, so model-context membership is a cross-check on the reconstruction rather than a statement about the wire. |
| its own record schema | not published, and it changed during this project. |
diff --git a/docs/en/adapters/claude-code.md b/docs/en/adapters/claude-code.md
index fb88b2c..40ebe57 100644
--- a/docs/en/adapters/claude-code.md
+++ b/docs/en/adapters/claude-code.md
@@ -7,6 +7,11 @@ cannot be supplied, said to be unavailable rather than approximated.
**Verified against** Claude Code 2.1.220 – 2.1.251. No earlier version was available for measurement,
so nothing here is confirmed below 2.1.220.
+**Measured on** one development machine's Claude Code history. It was live and grew while it was
+measured: about 2,920 files and 1.2 GB in the first pass, 2,970 files, 1.09 GB and 365,825 records
+when it was measured again, and 3,032 files in the last pass. A count is a snapshot. Where two
+passes disagree, or a count covers only part of the corpus, the page says which.
+
## Collection
The adapter reads local files. It requires **no configuration, no environment variables, and no
@@ -18,6 +23,7 @@ conversation history that predates its installation.
~/.claude/projects/<slugified-cwd>/<session-id>/subagents/agent-<agent-id>.jsonl
~/.claude/projects/<slugified-cwd>/<session-id>/subagents/agent-<agent-id>.meta.json
~/.claude/projects/<slugified-cwd>/<session-id>/subagents/workflows/<wf-id>/agent-<agent-id>.jsonl
+~/.claude/projects/<slugified-cwd>/<session-id>/subagents/workflows/<wf-id>/agent-<agent-id>.meta.json
~/.claude/projects/<slugified-cwd>/<session-id>/subagents/workflows/<wf-id>/journal.jsonl
~/.claude/projects/<slugified-cwd>/<session-id>/workflows/wf_<run-id>.json
~/.claude/projects/<slugified-cwd>/<session-id>/workflows/scripts/<label>-wf_<run-id>.js
@@ -26,7 +32,18 @@ conversation history that predates its installation.
A workflow script carries its run id in the filename, which is what makes it collectable
despite living outside its session's directory. Split on the **last** `-wf_`: a run id itself
contains hyphens (`wf_afa31e47-f6c`), so a greedy match starts at the first occurrence and
-swallows any later one.
+swallows any later one. On a real corpus, 172 of 172 script names parsed this way, and all 172 run
+ids matched a manifest in their own session.
+
+Transcripts are the primary channel because their content is complete. They store the person's
+prompt text, the assistant's prose, tool inputs and tool results in full. Claude Code's
+OpenTelemetry events do not. By default they redact `user_prompt` and `assistant_response`, cut
+content at about 60 KB, and omit tool inputs unless `OTEL_LOG_TOOL_DETAILS` is set. Those three
+defaults were read from the 2.1.245 binary. Events also redact reasoning in every case. Transcripts
+have two exceptions of their own. Reasoning text is usually empty, see
+[What this adapter cannot supply](#what-this-adapter-cannot-supply). And on the main stream, a long
+tool result's content block can be shorter than `toolUseResult.stdout`, see
+[Session Data mapping](#session-data-mapping).
OpenTelemetry is an optional second channel that adds measurement only — see
[OTLP](#otlp-optional-second-channel).
@@ -35,7 +52,14 @@ OpenTelemetry is an optional second channel that adds measurement only — see
A directory under `projects/` is named after the working directory Claude Code was launched in, with
`/`, `.` and space all collapsed to `-`. That transformation is **not reversible** — the real path is
-recoverable only from record content.
+recoverable only from record content. The rule was checked across all 25 source directories that
+held a transcript. Two did not fit a plain `/` rule, and they showed that `.` and space are replaced
+too: `/Users/…/Application Support/JetBrains/GoLand2026.1/…` became
+`-Users-…-Application-Support-JetBrains-GoLand2026-1-…`, and
+`/private/tmp/agentsessionizer-cc-v04.psixvf` became `-private-tmp-agentsessionizer-cc-v04-psixvf`.
+A [session filter](../setup/configuration.md#session-filters) that starts with `/` is converted
+forward the same way. Because the conversion loses information, `/private/tmp/**` matches the
+directory itself and any name that begins with its converted form followed by `-`.
More importantly, **a session's files are not confined to one such directory**. Claude Code files a
workflow run's *script* under whatever working directory the agent had at the time. Measured on a
@@ -52,26 +76,91 @@ Otherwise a directory containing nothing we collect could disqualify the whole s
exclude pattern — which is why filtering keys on the session's *primary* directory, the one
holding its main transcript.
+Child files are found at exactly two levels under a session directory: `subagents/` and
+`subagents/workflows/<run-id>/`. Nothing deeper is read, so a child file placed deeper would be
+missed. The second level is required. In the 2,970-file pass, a listing of `subagents/` alone found
+107 child files where the full walk found 2,910, a 96.3% miss, because most children live under
+`subagents/workflows/<run-id>/`.
+
+Every pass discovers the sources again rather than keeping a list, so a file that appears after one
+pass is collected by the next. A project directory that vanishes during a scan is skipped, because
+the corpus is live. One that cannot be read for any other reason is an error for the pass, so a
+pass that could not see every source does not report itself complete. That holds for project
+directories only. Inside a session, a directory that cannot be read is skipped with no error: the
+session directory itself, `subagents/`, `subagents/workflows/`, a run directory under it,
+`workflows/` and `workflows/scripts/`. The pass then reports itself complete without the files
+under it.
+
Two further rules:
- `memory/` is a sibling directory inside a project directory and is **not** a session. Match session
directories by identifier shape, not by "is a directory".
-- **Pruning is real and not atomic.** Claude Code deletes transcripts — the great majority of session
- ids known to its prompt history have none. A session directory whose main transcript has been
- deleted while its child streams survive is a normal state, not corruption.
+- **Pruning is real and not atomic.** Claude Code deletes transcripts. Of 345 session ids in its
+ prompt history, `history.jsonl`, 330 had no surviving transcript. In the first pass, 12 of 41
+ session directories had no main transcript while their whole `subagents/` tree survived. That is a
+ normal state, not corruption. When a source disappears, its cursor moves to `source_gone` and the
+ files already landed stay, so the storage root outlives the source. Separately, 32 of 61 main
+ transcripts had no subagent directory at all, which is a session that started no agent.
+
+### How each source is read
+
+| Source | How it changes | How it is read |
+| --- | --- | --- |
+| `<session-id>.jsonl`, `agent-<agent-id>.jsonl` | only grows | from a byte offset: each pass lands what was appended since the last one |
+| `journal.jsonl` | only grows | from a byte offset |
+| `wf_<run-id>.json` | rewritten as the run progresses | by content digest: a changed file lands again as a new version |
+| `<label>-wf_<run-id>.js` | may be rewritten | by content digest |
+| `agent-<agent-id>.meta.json` | written once, at spawn | by content digest, so in practice it lands once |
+
+Reading by offset is safe because Claude Code only appends to a transcript. Resuming a session
+appends to the same file, see [Conversation identity](#conversation-identity), and the block
+Claude Code re-writes before a context reset is also an append, see
+[Reconstruction rules](#reconstruction-rules). Across 60 main and 2,851 subagent files in the first
+pass, no parent id dangled and none pointed into another file. Truncation was never observed, but
+that is not proven. A cursor's size check and its tail digest catch it, and the collector then stops
+that source with a conflict rather than read from a position that no longer means what it did.
+
+A session is never finished. A transcript quiet for weeks can grow again when the session is
+resumed, so the adapter never marks a session done, and every pass checks every source again.
+Collection polls, as [collector](../setup/configuration.md#collector) describes. There is no file
+system watcher.
+
+Each child lands flat by its agent id, beside `main`, as the [storage root](../formats/storage-root.md)
+describes. That is safe because no agent id appears in two files, see
+[Execution streams](#execution-streams). Nesting children under their run or their parent would
+write a derived join into a write-once path. A join that later proved wrong would then mean moving
+files, and that matters most for the 31 depth-2 agents. Only a name of the shape
+`^a[0-9a-f]{16}$` counts as an agent id. If two source files would still land in one stream or run
+directory, the collector refuses the second and counts a conflict, rather than mix two files behind
+one cursor.
+
+### What is not read
+
+The adapter reads the conversation records and nothing else. The project's subject is conversation
+structure and provenance. The files below hold the runtime's working state or output spilled from a
+tool, not conversation records, so they are not read:
+
+- elsewhere under `~/.claude/` or beside it: the prompt history `history.jsonl`, `~/.claude.json`,
+ `file-history/`, the telemetry spool `telemetry/1p_failed_events.*`, the live process registry
+ `sessions/<pid>.json`, `shell-snapshots/`, `session-env/`, `backups/` and `paste-cache/`;
+- inside a project directory: `memory/`, `tool-results/` and `sessions-index.json`.
+
+Which files a tool call changed comes from the transcript's own patch and from the plugin, see
+[Workspace changes](#workspace-changes), never from `file-history/`. The prompt history appears on
+this page only as a measurement, under [Discovery is session-first](#discovery-is-session-first).
## Object mapping
| Model object | Claude Code | Notes |
| --- | --- | --- |
-| **Conversation** | `sessionId` *(default)* | overridable when an application supplies an explicit id |
-| **Segment** | derived | an activity window chosen by the assembler; not a runtime concept |
-| **Session** | `sessionId` | 1:1 with Conversation under the default mapping |
+| **Conversation** | `sessionId` | the only mapping implemented. See [Conversation identity](#conversation-identity). |
+| **Segment** | derived | an activity window chosen by the assembler, not a runtime concept |
+| **Session** | `sessionId` | 1:1 with Conversation |
| **ExecutionStream** `main` | `<session-id>.jsonl` | the only stream that compacts |
-| **ExecutionStream** child | `agent-<agent-id>.jsonl` | shares the parent `sessionId` |
+| **ExecutionStream** child | `agent-<agent-id>.jsonl` | named by the agent id in its file name. It shares the parent `sessionId`. |
| **Context epoch** | split by `system/compact_boundary` | main stream only |
-| **Talk** | derived from the input → output span | not a runtime concept |
-| **Run** | derived | not a runtime concept |
+| **Talk** | derived from the trigger of each prompt cycle | not a runtime concept. See [Talks and runs](#talks-and-runs). |
+| **Run** | `promptId`, within one stream | one run per distinct `promptId` in a stream |
### Conversation identity
@@ -85,8 +174,15 @@ file** as a fixed three-record chain: a root `user` record with `isMeta:true` ca
`<local-command-caveat>`, then a `user` record whose content is `<command-name>/clear</command-name>`,
then `system/local_command`. The successor carries no pointer back to its predecessor.
-Applications that consider several `/clear`-separated sessions to be one piece of work must supply an
-explicit conversation id.
+Resuming a session does not start a new one. Claude Code appends to the same file under the same
+`sessionId`, so a resumed session continues the same session and the same conversation. One
+transcript in the first pass held a 659-hour gap, across which `version` went from 2.1.220 to
+2.1.251, and its parent chain still linked across the gap.
+
+The session id is the only mapping implemented. `asz parse` always passes it as the conversation
+id, and no flag, configuration key or API takes another one today. So an application that considers
+several `/clear`-separated sessions one piece of work cannot yet say so. Supplying an explicit
+conversation id is planned.
### Execution streams
@@ -95,132 +191,283 @@ zero exceptions and zero files carrying more than one session id. It never gets
The partition is clean: no sidechain records in main transcripts, no top-level `agentId` in main
transcripts.
+**Stream membership comes from the file a record was written in, never from a field.** A child's
+stream id is the agent id in its file name, and the collector writes it into the landed header. The
+in-record `agentId` is not used to name a stream. That is safe. In a full scan of 2,686 agent files
+and 218,393 records in the first pass, the file name's agent id equalled the in-record `agentId` in
+2,686 of 2,686 files, no file carried more than one, and no two files shared one.
+
+`isSidechain` looks like the discriminator and is not. It is missing on 28.7% of main-transcript
+records in the 2,970-file pass, and inside a subagent file it is constant, so it says nothing about
+any one record. asz does not read it.
+
| | main | child |
| --- | --- | --- |
| `sessionId` | S | **S — identical** |
-| `isSidechain` | absent | `true` |
+| `isSidechain` | `false`, or missing (28.7% of records) | `true` |
| top-level `agentId` | absent | `^a[0-9a-f]{16}$` |
-Nesting stops at depth 2: of 2,686 agents, 2,654 are depth 1 and 31 are depth 2, each carrying
-`parentAgentId`. No depth ≥ 3 exists.
+Nesting stops at depth 2. In the 2,970-file pass, 2,683 sidecars were at depth 1, 31 at depth 2,
+and one had no depth field. No depth of 3 or more was seen. All 31 depth-2 agents come from one
+session, so a method that samples a few sessions will likely see none and conclude that the maximum
+is 1. The file layout carries no depth: every agent transcript sits flat in `subagents/` or in
+`subagents/workflows/<run-id>/`. Nesting is known only from a nested child's sidecar
+`parentAgentId`. asz does not read `spawnDepth`.
Child streams are structurally simpler than main: exactly one root per file, no duplicate record
-ids, **no compaction at all**, and no rewind forks. Their model context accumulates without
-interruption for the life of the stream.
+ids, **no compaction at all** (0 boundaries in 221,592 child records in the 2,970-file pass), and no
+rewind forks. Their model context accumulates without interruption for the life of the stream, so
+each child stream is one context epoch.
### Context epochs
`system/compact_boundary` is an explicit, machine-readable epoch boundary — the model never has to
-infer one. It carries `compactMetadata` plus **`logicalParentUuid`**, a pointer to the last
-pre-compaction message that appears on no other record type anywhere.
+infer one. asz detects it by the system record's `subtype`, never by the text `compact_boundary`,
+which appears far more often in message content than in real boundary records. An epoch is cut only
+at such a record, in landed order, and a stream with no boundary is one epoch.
+
+The boundary carries `compactMetadata` plus **`logicalParentUuid`**, a pointer to the last
+pre-compaction message. In the 2,970-file pass it was present on 27 of 27 boundaries and on no other
+record type. The boundary starts a new parent chain. Its own `parentUuid` is null on 27 of 27,
+so `logicalParentUuid` is the only link back, and a parent walk across a reset stops without it.
+- `logicalParentUuid` is never the line just before the boundary, so proximity cannot replace it.
- The boundary's `uuid` is the `parentUuid` of the immediately following `isCompactSummary:true`
- record, without exception.
-- The summary's timestamp is **always earlier** than its own boundary's, by 1 ms to 1,097 ms. Never
- order an epoch boundary by timestamp.
-- `preservedMessages.allUuids` may contain ids that exist nowhere on disk — records held in context
- but never written. Both those entries and `logicalParentUuid` must be typed unresolved-capable.
-- The preserved segment does not necessarily abut the boundary; resolve by id, never by proximity.
+ record, without exception. asz pairs the two through that parent.
+- The summary's timestamp is **always earlier** than its own boundary's, by 1 ms to 1,097 ms on
+ 26 of 26 boundaries in the first pass. Never order an epoch boundary by timestamp.
+- The summary carries a `promptId` and no `origin`. When no other record of its cycle states a
+ trigger, that cycle starts a Talk, under the rule in [Talks and runs](#talks-and-runs).
+- asz reads only `logicalParentUuid`. When it names a record that is not in the landed data, the
+ round carries an open `epoch_continuation` reference. `compactMetadata.preservedMessages` is not
+ read. One pointer is enough: `logicalParentUuid` equals the last entry of
+ `preservedMessages.allUuids` on 26 of 26 boundaries in the first pass. `allUuids` can name records
+ that exist nowhere on disk, 5 in that pass, which is why the pointer is checked rather than
+ trusted.
+
+### Talks and runs
+
+Claude Code records neither a Talk nor a Run. Both are built from two of its fields, `promptId` and
+`origin`, by an exact rule.
+
+- **A run is one prompt cycle**: one per distinct `promptId` within a stream. A `promptId` is not
+ unique across streams, because a child stream is written under a cycle id that also appears in its
+ parent, so the key is the stream and the `promptId` together. A model response carries no
+ `promptId`, and reaches one through its parents, as the
+ [pipeline](../concepts-and-designs/conversation-assembly.md#the-pipeline) describes. On this
+ machine on 2026-09-11, none of 192,743 assistant records carried a `promptId`. In a sample of 400
+ assistant records, whose pass was not recorded, all 400 reached a `promptId` through
+ `parentUuid`, in 1 to 7 hops, with a median of 2.
+- **A cycle's trigger is `origin.kind`**, read from the record and then from `attachment.origin`.
+ Reading only the record loses 27.6% of all trigger markers. `human` is external and
+ `task-notification` is a notification. The value is an open set: a third kind exists, so any
+ other value maps to `unknown`, and a switch on exactly two values would meet something it does
+ not expect.
+- **On the main stream, a Talk starts when a cycle has an external trigger, or when no record of
+ the cycle states any trigger.** A local slash command writes no `origin`, and it is a person
+ acting, so a rule that required an explicit marker would attach it to whatever came before. A
+ cycle whose only stated triggers are notifications or an unknown kind continues the Talk in
+ progress. A child stream is one Talk, created up front, and each of its cycles is a run in it.
+
+The count behind the rule: distinct `promptId` values equal human cycles, plus notification
+cycles, plus cycles with no `origin`. One sampled session had 65 cycles, 36 human and 29
+notification. On one real 59,495-record session, the main stream assembles to 307 Talks from 336
+cycles, which equals the number of `origin.kind:"human"` records counted independently on the raw
+file.
+
+The count holds only after duplicate records are removed. A re-emitted copy rewrites `origin`, so a
+Talk splitter that runs before removal over-counts human triggers by up to 15% in the 2,970-file
+pass.
## Step mapping
| Model kind | Claude Code source | Quality |
| --- | --- | --- |
-| `message.external` | `user` record, `origin.kind:"human"` | `exact_unique` |
+| `message.external` | `user` record with `origin.kind:"human"` and without `isMeta`; a `queued_command` attachment with `commandMode:"prompt"` | `exact_unique` |
| `message.assistant` | assistant `text` block | `exact_unique` |
| `message.synthetic` | `message.model == "<synthetic>"` | `exact_unique` |
-| `context.injection` | `attachment` records, 32 types | `exact_unique` |
-| `llm.call` | `(file, message.id)` group | `exact_unique` |
-| `thinking` | assistant `thinking` block — signature only | `unavailable` (content) |
+| `context.injection` | every other `attachment` record; a `user` record with `isMeta:true` | `exact_unique` |
+| `llm.call` | assistant records sharing one `message.id` in the session | `exact_unique` |
+| `thinking` | assistant `thinking` or `redacted_thinking` block | `exact_unique`. The text is usually `unavailable`. |
| `tool` | assistant `tool_use` block joined to its `tool_result` block; enriched by `toolUseResult` | `exact_unique` |
-| `agent.call` | `tool_use` named `Agent` / `Workflow` / Skill fork | `exact_unique` |
+| `agent.call` | a `tool_use` block that a child stream is joined to | `exact_unique` |
| `agent.launch_ack` | `toolUseResult.status:"async_launched"` | `exact_unique` |
-| `agent.output` | final assistant output of the child stream | `exact_unique` |
-| `runtime.notification` | `<task-notification>` block | `exact_unique` |
+| `agent.output` | the child stream's last non-synthetic response, plus the run journal's result record when there is one | `exact_unique` |
+| `runtime.notification` | a record whose `origin.kind` is `task-notification`, on the record or on `attachment.origin` | `exact_unique` |
| `epoch.boundary` | `system/compact_boundary` | `exact_unique` |
| `epoch.summary` | following `isCompactSummary:true` record | `exact_unique` |
-| `error.api` | `system/api_error` + `retryAttempt` | `exact_unique` |
-| `control.interrupt` | `[Request interrupted by user]` | `exact_unique` |
-| `control.permission` | denial strings, `permission-mode` records | `strong_inference` |
+| `error.api` | `system/api_error` | `exact_unique`. The retry state is `unavailable`. |
+| `control.interrupt` | `[Request interrupted by user]` | `unavailable`: not emitted today |
+| `control.permission` | denial strings, `permission-mode` records | `unavailable`: not emitted today |
| `control.command` | `system/local_command` | `exact_unique` |
| `turn.duration` | `system/turn_duration` | `exact_unique` |
-One tool use is one step. The `tool_use` block supplies `name` and `input` — for `Bash`, the command
-itself — and the `tool_result` block supplies the result. The join is exact and one-to-one: corpus-wide
-every `(file, tool_use_id)` maps to exactly one result once duplicate records are removed. A provider
-call to its tools is one-to-many, up to 16.
-
-A second, fully redundant join exists: every tool result carries a top-level
+**Every attachment is injected context by default.** There is no list of types, because a list has
+to be right about types that do not exist yet. An earlier list was wrong in both directions: 635
+records across 13 real attachment types fell through it and were dropped, while 5 types on it
+appear nowhere in the corpus. The first pass counted about 21,000 attachment records of 32 types. A
+`user` record with `isMeta:true` is text the harness wrote in the user role, so it is injected
+context too, not something a person said.
+
+**One attachment is a person's input.** A `queued_command` with `commandMode:"prompt"` is a message
+typed while the agent was working. It becomes `message.external`, and any other mode stays
+`context.injection`. These turns have no `.message` twin. In the first pass, 614 human-typed turns
+existed only as `queued_command` records, and in the 2,970-file pass a third of all human
+utterances lived only there, so a reader that follows messages alone loses them. The mode is needed,
+not only the type: of 1,558 unique queued records in the first pass, only 960 were human input, and
+reading the type alone would insert 598 machine-generated `<task-notification>` blobs as things the
+person typed. [asz.view](../formats/asz-view.md) takes such a turn's text from the envelope's
+`prompt[].text`. Most of these turns do not start a Talk. In the 2,970-file pass, 886 of 918
+joined a cycle that was already running. Such a record is placed in the enclosing Talk
+through its parents, and asz invents no cycle id for it. The scenario
+`tests/scenarios/attachment-input.yaml` checks both modes.
+
+One tool use is one step. The `tool_use` block supplies `name` and `input`. For `Bash`, the input
+is the command itself. The `tool_result` block supplies the result. The join key is the exact
+tool-use id, looked up across every stream of the session, and nothing matches on a prefix. Once
+duplicate records are removed the join is one-to-one. Across 2,924 files, 356,583 records and
+207,984 tool blocks in the first pass, each of 103,503 keys had exactly one result, and no key was
+shared across files. Several results for one id would make the step `exact_ambiguous`, with an open
+`tool_result` reference. A provider call to its tools is one-to-many, up to 16.
+
+The 2,970-file pass did find tool uses with no result, on child streams only: 7 of 79,418 tool uses
+there, each on the last line of its file. On the main stream it was 0 of 25,892. Such a step
+carries `result: unavailable` and an open `tool_result` reference.
+
+A second join exists in the data and is not used. Every tool result carries a top-level
`sourceToolAssistantUUID`, and it matched the record holding the corresponding `tool_use` in 105,409
-of 105,409 cases. It is a free integrity check on the id join.
+of 105,409 cases. asz does not read it and does not run that check.
+
+A tool use becomes `agent.call` only when the spawn join finds a child stream that came out of it.
+The kind comes from that join, not from the tool's name. A tool use whose child never landed stays
+`tool`, and the round carries an open reference, see [Agent spawn](#agent-spawn). The tool that
+starts an agent is named `Agent`. `Task` appears nowhere as a tool name in the corpus.
`toolUseResult` enriches the result with structure the content block lacks — `stdout`/`stderr` split,
`structuredPatch`, interruption flags — but almost only on the **main** stream: present on 100% of
main-transcript tool results and 2.1% of child-stream ones. Everything read from it, including the
-spawn joins below, is therefore a main-stream mechanism.
+spawn joins below, is therefore a main-stream mechanism. How it lands is in
+[Session Data mapping](#session-data-mapping).
**`toolUseResult` is not always an object.** On roughly 7% of results it is a bare string. Decoding it
into a struct fails, and because the failure covers the whole record, every identifier on that record
is lost with it — the tool join, the prompt cycle, the parent link. It must be read as raw JSON and
-shape-checked before any field access.
-
-`timing` is `unavailable`. Claude Code writes no separate execution record locally, and inferring a
-duration from the gap between the request and result records would report queueing and model latency
-as tool time. Three tools are the exception and do report a real duration — `WebFetch.durationMs`,
-`WebSearch.durationSeconds`, `Agent.totalDurationMs`. A configured timeout is not a measurement.
-OTLP's `claude_code.tool.execution` span supplies the rest, which is Phase 2.
-
-**Absence of `is_error` does not mean success** — it is present on 85.5% of result blocks.
+shape-checked before any field access. asz reads it raw and uses it only when it is an object.
+
+`timing` is `unavailable` on every tool step. Claude Code writes no separate execution record
+locally, and inferring a duration from the gap between the request and result records would report
+queueing and model latency as tool time. Three tools do report a duration the runtime measured,
+`WebFetch.durationMs`, `WebSearch.durationSeconds` and `Agent.totalDurationMs`. asz does not read
+them today, so those three are `unavailable` too. A configured timeout is not a measurement.
+[asz.view](../formats/asz-view.md#a-node-in-talks) reports the time from the request record to the
+result record where the join is exact. That is the interval between two records, not execution
+time. OTLP's `claude_code.tool.execution` span supplies the rest, which is Phase 2.
+
+**Absence of `is_error` does not mean success.** It is present on 85.5% of result blocks. asz sets
+`failed` only when the block carries it.
+
+`error.api` comes from `system/api_error` alone. asz reads no retry field, so every such step
+carries `retry_state: unavailable`, and no `retries` relation is written.
+
+`control.interrupt` and `control.permission` are not emitted today. An interrupt does not break the
+tool join, because the `[Request interrupted by user]` record is written after the tool result and
+the join does not depend on it. The next turn can re-parent to an earlier record, and it is still
+placed in its run through its own parents. No `cancels` relation is written, and the re-parent is
+not recorded as a stream discontinuity. The generated [glossary](claude-code-glossary.md) keeps the
+runtime's name for both kinds and for `cancels`, because the runtime has a word for each. Its note
+says asz does not read it yet.
### Provider calls
-**Group by `(file, message.id)` — never by `requestId`.** `message.id` is present on every assistant
-record; `requestId` is absent on some genuine calls, and a `<synthetic>` companion record can reuse a
-real call's `requestId`, which would put fabricated content inside a real provider call.
+**Group by `message.id` across the whole session. Never group by `requestId`.** `message.id` was
+present on 115,162 of 115,162 assistant records in the first pass, and it never appeared in more
+than one file, so the fragments of one call all sit in one stream. `requestId` is absent on some genuine
+calls, and a `<synthetic>` companion record can reuse a real call's `requestId`, which would put
+fabricated content inside a real provider call. A test on `requestId` is wrong in both directions:
+in the first pass, 24 of 97 synthetic records carried a non-null `requestId`, and 2 genuine calls
+carried none. The synthetic test is `message.model == "<synthetic>"`. A synthetic record can never
+join a real call: in the 2,970-file pass, all 122 synthetic records had a UUID-shaped message id,
+and all 184,330 real ones start with `msg_`.
A single response is split across several JSONL lines — 2.09 fragments per call corpus-wide, ranging
-1.59 to 2.51 per session. Line order is authoritative; timestamps are not, and ties occur. A call's
-fragments are **not contiguous**: on 24% of multi-fragment calls their own tool results sit between
-them, so grouping must scan the whole file by key rather than a run of lines.
+1.59 to 2.51 per session. **Line order is authoritative. Timestamps are not.** Fragments are sorted
+by landed file and row, which reproduces source order. In the first pass, 372 provider calls had
+fragments with tied timestamps, and at file level 2.76% of timestamped records were out of order
+after duplicates were removed. A call's fragments are **not contiguous**: on 24% of multi-fragment
+calls their own tool results sit between them, so grouping must scan by key rather than read a run
+of lines.
**Usage: take the last fragment in LINE ORDER. Never sum, and never use `stop_reason` to find it.**
`stop_reason` is not a terminal marker. A main transcript stamps the same value on every fragment of a
-call; a subagent transcript stamps it only on the last. So a `stop_reason` test picks the *first*
-fragment on main and the last on a child. Only line order works on both.
+call, in 23,324 of 23,329 calls in the 2,970-file pass. A subagent transcript stamps it only on the
+last. So a `stop_reason` test picks the *first* fragment on main and the last on a child. Only line
+order works on both.
Summing is wrong for a different reason on each side. Main repeats the final usage on every fragment,
so adding them multiplies the real number by the fragment count — 2.18× pooled, 9× worst case.
-Subagent partials climb `1, 2, 3 …`, where the last value is the whole count.
+Subagent partials climb `1, 2, 3 …`, where the last value is the whole count, and summing them adds
+only 2%.
-About 10% of calls never report a `stop_reason` (0.02% main, 14.5% subagent). Two consequences:
+About 10% of calls never report a `stop_reason` (0.02% main, 14.5% subagent, in the 2,970-file
+pass). Two consequences:
-- Their usage block is **present and wrong** — an output count of one to five, a streaming stub. Usage
- availability is gated on `stop_reason`, never on the field being present.
+- Their usage block is **present and wrong**, a streaming stub. 5,739 of 7,410 such calls have a last
+ output count of 5 or less, against 101 of 66,751 finished ones. Usage availability is gated on
+ `stop_reason`, never on the field being present.
- They are **not in flight**. Only 72 of 7,410 sit at end-of-file; the rest are mid-file with the
- conversation continuing past them, and their tools did run. Nothing waits for them.
+ conversation continuing past them, and 7,311 of them had their tools run. Nothing waits for them,
+ and nothing is marked open.
-`origin` is not in one place: it sits on the record and also on `attachment.origin`, and reading only
-the first loses 27.6% of all trigger markers. Its value is an open set — a third kind exists beyond
-`human` and `task-notification` — so a switch on exactly two values will meet something unexpected.
+Every `llm.call` node in a round carries `fragments` and `usage_from: last_fragment_in_line_order`. A
+finished call adds `usage: observed_replayable` and `usage_at`, the `{seq, row}` of the fragment the
+usage is read from. An unfinished call has `usage: unavailable` and `stop_reason: unavailable`.
+[asz.view](../formats/asz-view.md#a-node-in-talks) copies the token counts from the one record
+`usage_at` names.
### Agent spawn
-No single field spans the mechanisms, and they do very unequal amounts of work.
-
-| Mechanism | Parent → child join | Share |
-| --- | --- | --- |
-| **Workflow** | the `Workflow` result's `runId`, plus `journal.jsonl` `{type:"started", agentId}` naming each child | ~96% |
-| **Agent tool** | the parent's `toolUseResult.agentId` | |
-| **Skill fork** | the same field with `status:"forked"`; no separate pointer to the call, so the tool id comes from the result's own content block | |
-| **Nesting** | the child sidecar's `parentAgentId` | 31 children |
-| | *together* | ~4% |
-
-**`agent-<id>.meta.json` is not a general parent pointer.** It normally holds only
-`{agentType, spawnDepth}`. It names a parent for a *nested* child and nothing else, so it is a 3.3%
-edge case rather than a primary join. Without `parentAgentId` a child of a child is attributed to the
-main stream, flattening all 31 depth-2 spawns. `spawnDepth` is absent on one sidecar and must default
-to 1.
+No single field spans the mechanisms, and they do very unequal amounts of work. The share was
+measured in the 2,970-file pass.
+
+| Mechanism | Parent → child join | Quality | Share |
+| --- | --- | --- | --- |
+| **Workflow** | the `Workflow` result's `runId`, plus `journal.jsonl` `{type:"started", agentId}` naming each child | `strong_inference` | ~96% |
+| **Agent tool** | the parent's `toolUseResult.agentId` | `exact_unique` | |
+| **Skill fork** | the same field with `status:"forked"`; no separate pointer to the call, so the tool id comes from the result's own content block | `exact_unique` | |
+| **Nesting** | the child sidecar's `parentAgentId` | no `starts` relation, see below | 31 children |
+| | *together* | | ~4% |
+
+The quality is on the `starts` relation from the call to the child stream. A journal edge is
+`strong_inference` because the journal names no call. The call comes from the launch result that
+named the run: asz resolves the run to the tool use whose result names it, so the edge runs from
+that call to the child, with no node standing for the run. An edge from a parent result or from a
+completion notification is `exact_unique`.
+
+**Workflow membership comes from the journal, never from `agentType`.** A workflow step can ask for a
+named agent type, so `agentType` does not identify a workflow child, and asz does not read it.
+Discovery finds a workflow child under `subagents/workflows/<run-id>/`, and the run's journal names
+it. In the first pass, each `journal.jsonl` `{type:"started", agentId}` matched a child file, 2,583 of
+2,583.
+
+**`agent-<id>.meta.json` is not a general parent pointer.** It is written once, at spawn: 200 of 200
+sampled sidecars were older than their own transcript, none equal and none newer. Its content is
+spawn-time only, `agentType`, `description`, `spawnDepth`, `toolUseId`, `parentAgentId` and `model`,
+and never a status, a result or a duration. Most sidecars hold only `agentType` and `spawnDepth`.
+On this machine on 2026-09-11 that was 2,534 of 2,748 sidecars, and 2,517 of them belonged to
+workflow children. So most children carry no pointer to their parent in their own sidecar. There is
+no registry file. Each agent owns its sidecar. Pairing was exact: 2,697 sidecars to 2,697
+transcripts, with no orphan either way.
+
+Beyond which child it belongs to, asz reads two fields from a sidecar. `description` becomes the
+child stream's label. `parentAgentId` names the agent that started a nested child, and is the only
+evidence of nesting anywhere. asz reads no call id from a sidecar, so a sidecar never anchors a spawn
+on a call.
+
+A nested child named only by its sidecar is counted as joined and is not reported as an orphan. But
+no `starts` relation is written for it, because that edge has no call to start from. Like every
+child stream it sits under the session. A nested child whose completion notification names both the
+call and the child's agent id gets its `starts` relation through the notification.
**The journal answers one question:** which children belong to a run. It carries no timestamp, no
name, no parent pointer and no tool id. The launch itself comes from the parent's result, which names
@@ -231,57 +478,233 @@ is normal.
result. `forked` is a skill fork. `completed` is a **synchronous return that IS the result** and
carries `agentId`.
-Completion arrives as a `<task-notification>`, and its two ids are not equally good.
-**`<task-id>` is a generic task handle** — an `agentId` only about 8% of the time, otherwise a
-background-command id, a workflow task id or a monitor id. Joining on it unconditionally matches the
-wrong thing most of the time. `<tool-use-id>` is exact and is the primary key. A single notification
-can carry several `<task-id>` elements.
+**The child owns its output.** The child's messages and tools stay in the child stream. `agent.output`
+is a node in the child stream, with an `ends_with` relation from the stream to it. It points at the
+child's last non-synthetic response, and at the run journal's result record when there is one, and
+`returned_value` is then `observed_replayable`, otherwise `unavailable`. The journal's result is what
+the child returned, and it is not in the child's transcript: 2,353 of 2,436 journal results appear
+nowhere else, on a snapshot of the development corpus whose size was not recorded. asz does not drop
+the parent's copy. A synchronous return, `status:"completed"`, still appears as the result of the
+parent's `agent.call`.
+
+Completion arrives as a `<task-notification>`, and its two ids are not equally good. asz reads
+them only from a record whose message content is a plain string. The notification step comes from
+the record's trigger, not from this text, see [Step mapping](#step-mapping).
+**`<task-id>` is a generic task handle.** It is an `agentId` only about 8% of the time, and
+otherwise a background-command id, a workflow task id or a monitor id. Joining on it
+unconditionally matches the wrong thing most of the time. `<tool-use-id>` is exact, but it names
+the call, not the child, so asz tries the task id first. A single notification can carry several
+`<task-id>` elements. asz takes the first one that has an agent id's shape, and
+links the notification to that child with a `reports` relation, `exact_unique`. When that child's
+transcript has not landed, the round carries an open `notified_child` reference instead. With no
+such task id, asz resolves through the call named by `<tool-use-id>`, whose spawn edge names the
+child, and the `reports` relation is `strong_inference`.
Two cautions. The `<tool-use-id>` is **not always resolvable in the file it lands in** — a nested
child notifies the main session while the call that started it lives in an agent file — so resolution
-must search the session's whole file set. And `<status>` is **absent entirely** on a large class of
-notifications; requiring it will fail.
+searches the session's whole file set. And `<status>` is **absent entirely** on a large class of
+notifications, so asz does not read it.
The workflow journal **leads the filesystem**: a `started`, even a `result`, can be journalled before
the child transcript exists. That is normal for a live session, not corruption. Once the run is over
-it is instead evidence of a pruned or lost file.
+it is instead evidence of a pruned or lost file. Until the file appears, the round carries an open
+`child_stream` reference, and a later round resolves it. A batch whose launch call was never landed
+is carried as an open `spawn_call` reference.
-**A child with no parent at all exists** — two in the corpus. It is a state to record, not an
-assumption to assert.
+**A child with no parent at all exists.** In the 2,970-file pass, 99.93% of children resolved and two
+did not. An orphan stream stays under the session and in no parent's Talk, and the round carries an
+open `spawn_of_child` reference for it, because guessing a parent is worse than saying none was
+found.
## Reconstruction rules
Three rules, each required for correctness rather than efficiency.
-**1. Drop duplicate records by `(file, uuid)` before anything else.** Record ids are not unique within
-a file: 1,533 repeated ids among 318,199, in 11 of 62 main transcripts and **0 of 2,908 subagent
-transcripts**, always with multiplicity exactly 2 — so one pass is enough.
-
-Keep the **first** occurrence, and not only for position. The later copy is the worse one: its
-`promptId` is rewritten on 538 of 1,533 pairs, its `parentUuid` differs on 40, and where both carry
+**1. Remove duplicate records by record id, across the whole session, before anything else.** Record
+ids are not unique within a file. In the 2,970-file pass, 1,533 ids repeated among 318,199, in 11 of
+62 main transcripts and **0 of 2,908 subagent transcripts**, always with exactly two copies, so one
+pass is enough. Every repeat measured was inside one file. No measurement shows an id repeated across
+two files, and keying on the id across every file of the session relies on that. A record with no id
+is never treated as a copy. The same rule removes a record landed twice when a collector pass is
+interrupted between landing a file and saving its cursor. The scenario `tests/scenarios/replay.yaml`
+checks it.
+
+Keep the **first** occurrence, and take nothing from the later one. The later copy is the worse one:
+its `promptId` is rewritten on 538 of 1,533 pairs, its `parentUuid` differs on 40, and where both carry
`toolUseResult` the first is larger on 430 pairs and smaller on **zero** — 43.4% of captured tool
output would be discarded, with `stdout` blanked to `""` on 314 pairs. The `message` object itself is
-byte-identical on every pair, which is why this looks harmless and is not. Nothing distinguishes an
-original from a copy except line order.
-
-These duplicate blocks are not noise. Each is a pre-compaction re-emission of **exactly** the set of
-records lying off the following boundary's ancestry — which independently establishes that
-**context = chain ancestry ∪ off-chain siblings**, and distinguishes parallel-tool siblings (in
-context) from abandoned branches (dropped).
-
-**2. Assemble by `(message.id, line order)`, never by walking `parentUuid`.** The `parentUuid` graph is
-a **DAG, not a chain**: parallel tool dispatch forks it, so an ancestor walk traverses one branch and
-silently drops the other `tool_use` and its result.
-
-> Ancestor-walk reconstruction produces a message list containing a `tool_use` with no matching
-> `tool_result` — a body the Messages API would reject — on **39% of provider calls**. Line-order
-> reconstruction is balanced in 2,622 of 2,623 tool-bearing files.
-
-Every fork has exactly two children. Treat a fork as a topological hint, not a branch to choose.
+byte-identical on every pair, which is why this looks harmless and is not. The timestamps of the
+two copies are identical on 1,533 of 1,533 pairs in the 2,970-file pass, so time cannot say which
+copy came first. Nothing distinguishes an original from a copy except line order. The copies stay
+in the index as evidence and are counted, and every lookup leaves them out.
+
+Copies also make the tool join look ambiguous when it is not. In the first pass, before removal, 441
+tool-use ids joined ambiguously, 882 of 64,722 tool results (1.4%), only because of the re-emitted
+copies.
+
+These duplicate blocks are not noise. The first pass found 1,530 duplicated records in 10 main
+transcripts, and measured all 10. Each block is a re-emission just before a context reset: all 10
+end on the line immediately before a `compact_boundary`, and only 2 of 10 are a dense, contiguous
+replay of a prefix. The block is the set of records lying off the following boundary's ancestry. In
+10 of 10 files no duplicate lay outside that set, and in 8 of 10 no record off the ancestry was left
+out of the block. 0 of 1,530 duplicated records lie in the ancestry of the following boundary's
+`logicalParentUuid`. In the two exceptions, 330 and 37 records off the chain were not re-emitted,
+and they cluster in whole abandoned runs. That establishes **context = chain ancestry ∪ off-chain
+siblings**, and distinguishes parallel-tool siblings, which are re-emitted and in context, from
+abandoned branches, which are dropped.
+
+**2. Group a provider call by `message.id`, in line order. Never rebuild a call or a message list
+from `parentUuid` links.** The `parentUuid` graph is a **DAG, not a chain**, and two walks over it fail
+for two different reasons.
+
+- **Walking backward from a call to rebuild its message list** misses the sibling branch at a fork.
+ Parallel tool dispatch forks the graph, `tool_use#1 → {tool_use#2 (same message.id),
+ tool_result#1}`, so an ancestor walk traverses one branch and drops the other `tool_use` and its
+ result. In the first pass, 33,798 of 86,687 provider calls (39%) rebuilt this way carried a
+ `tool_use` with no matching `tool_result`, a body the Messages API would reject. Line-order
+ reconstruction was balanced in 2,622 of 2,623 tool-bearing files.
+- **Walking forward from a call's first fragment to collect its fragments** drops a tool use because
+ a call's fragments are interleaved with their own tool results, see
+ [Provider calls](#provider-calls). It is not a fork: no fragment has two children sharing its
+ message id. In the 2,970-file pass, the forward walk dropped a `tool_use` on 16,063 of 85,057
+ tool-bearing calls: 18.9% pooled, 2.1% on the main stream and 25.9% on child streams.
+
+Forks are more common where most calls are. The first pass found 4,073 forks, and every fork had
+exactly two children: a `tool_use` and a `tool_result` in 3,905, an assistant record and an
+`api_error` in 160. The fork rate was 0.49% on main and 1.69% on child streams, and child streams
+hold about 69% of provider calls, so a measurement on main transcripts alone understates the
+problem. Message-id grouping does not depend on the parent links at all: fragments of one message id
+never interleave with those of another, 0 of 86,716 calls.
+
+The ban is on rebuilding, not on the edge. asz does walk parent links backward, at most 64 hops, to
+find the run a record belongs to. That walk is single-valued, because a record has exactly one
+containment parent.
**3. Always iterate the full `content` array.** A line usually carries one block but sometimes carries
-several, mostly on `user` records — including `[image, text]`, where reading `content[0]` renders a
-base64 image as the user's prompt.
+several, mostly on `user` records. On this machine on 2026-09-11, 198 records in 2,832 transcript
+files carried more than one block: 186 `user` records and 12 `assistant` records. 55 of them were
+`[image, text]`, where reading `content[0]` renders a base64 image as the user's prompt. On 9 of the
+12 assistant records a `tool_use` block came after the first block, so reading only the first would
+break the tool join.
+
+## Session Data mapping
+
+The adapter converts each record once, while it is landed, into [Session Data](../formats/session-data.md).
+This section says what each Claude Code shape becomes. The roles a record plays are in the generated
+[glossary](claude-code-glossary.md).
+
+Every content block in the 3,032-file pass, 1.1 GB:
+
+| Claude Code block | Keys | Count | Part | State |
+| --- | --- | ---: | --- | --- |
+| `tool_result` | `content` `is_error` `tool_use_id` | 108,243 | `result` | `available` |
+| `tool_use` | `id` `name` `input` `caller` | 108,149 | `call`, the input in `data` | `available` |
+| `thinking` | `thinking` `signature` | 60,382 | `reasoning` | `available` with text, `unavailable` when empty |
+| `text` | `text` | 22,458 | `text` | `available` |
+| `image` | `source{data, media_type}` | 41 | `media`, the base64 inline in `data` | `available` |
+| `fallback` | `from{model}` `to{model}` | 1 | `unknown`, the bytes kept | `available` |
+
+Four of the six cover 99.99% of content blocks. Two more shapes are converted, though that pass
+counted none: `redacted_thinking` becomes `reasoning` marked `redacted`, and `server_tool_use` becomes
+a `call` keyed by its own id. Any other block type becomes `unknown` and keeps its bytes. Content given
+as a plain string, 4,429 records, becomes one `text` part. An attachment becomes `text` when it has a
+`text` field or a string `content`, and otherwise travels whole as a `data` part. This dialect never
+marks a part `truncated` or `omitted`.
+
+Where the bytes are, across 62 sessions and 1,100.1 MB of source records:
+
+| What | MB | Share |
+| --- | ---: | ---: |
+| record ids, usage blocks and the rest of the envelope | 370.0 | 33.6% |
+| tool results, the content block | 334.8 | 30.4% |
+| reasoning signatures | 146.4 | 13.3% |
+| tool results, the `toolUseResult` copy | 83.8 | 7.6% |
+| tool inputs | 75.0 | 6.8% |
+| injected context | 58.7 | 5.3% |
+| message text | 18.4 | 1.7% |
+| assistant text | 14.3 | 1.3% |
+| reasoning text | 0.1 | 0.0% |
+
+Tool results are 38.1% of the bytes, more than five times the tool inputs. Message text and
+assistant text, the text a person reads, are 3.0% together.
+
+**Reasoning signatures are not landed.** A thinking block's `signature` is listed in the record's
+`dropped` with its size and the reason, "a provider verifies it; a reader cannot read it". A provider
+checks the signature when a block is sent back to it, and no reader can use it. In the table above,
+signatures are 13.3% of the bytes and wrap 0.1 MB of reasoning text.
+
+**A tool result's content is not always a string.** Over 20,000 `tool_result` records: string 19,707,
+list of image 124, list of text 104, list of `tool_reference` 92. `tool_reference` is an undocumented
+block, `{"type":"tool_reference","tool_name":"Monitor"}`, 194 in the corpus. A `result` takes its
+`text` from the string, or from the text blocks joined. When there is no text, the part keeps the raw
+content as `data`, because the source can be pruned and the landed copy may be the only one. A
+reader that treats the content as a string loses the image and reference results.
+
+**A Claude Code result carries `text` and `data` together.** When the record carries a
+`toolUseResult` object, the whole object becomes the result part's `data`, beside the text. Both are
+kept on purpose. `toolUseResult` has a different shape for each tool, 49 distinct key sets across the
+corpus, which flattening would lose, and it has detail the block lacks: stdout and stderr apart, a
+patch, a status. It is also the only full copy of long output. Among 516 records whose output spilled
+to a file, the content block was shorter than `toolUseResult.stdout` by about 28 KB where it had been
+cut. The cost is the `toolUseResult` copy in the table above, 7.6% of the bytes. An `Edit` or
+`Write` result gains a second `data` part, a `changes/1` record, see
+[Workspace changes](#workspace-changes).
+
+**Four things are left out with no `dropped` entry**, which breaks the
+[Dropped](../formats/session-data.md#dropped) rule. They are known gaps:
+
+- a `tool_use` block's `caller` field, `{"type":"direct"}` on all 108,149 tool uses counted;
+- a `toolUseResult` that is a plain string, 7.2% of results;
+- the raw content of a result with no text, when the record also carries a `toolUseResult` object,
+ which takes `data` in its place;
+- the other fields of a record that becomes parts, which is a `user` or `assistant` message or an
+ attachment.
+
+Such a record lands with `id`, `parent`, `call`, `run`, `continues`, `tool`, `child`, `batch`,
+`started_by`, `label`, `time`, `trigger`, `model`, `usage`, `flags`, `from` and its parts, and
+nothing else. `requestId`, `isSidechain`, `cwd`, `gitBranch`, `slug`, `version`, `userType` and
+`entrypoint` do not land, and neither does the rest of an attachment that has text. They are gone
+once Claude Code prunes the source. The in-record `sessionId` and `agentId` do not land either, but
+the landed header names the same session and stream, see [Execution streams](#execution-streams).
+`cwd` is read only to make a changed file's path relative. A record with no message content and no
+attachment travels whole as one `data` part, so all its fields are kept. System records, sidecars,
+journal lines and manifests are of this kind.
+
+**An MCP tool use is an ordinary `call`.** Its server is only a prefix on the name,
+`mcp__<server>__<tool>`. This was observed on 39 of 26,940 tool uses (0.1%), too few to state as a
+rule.
+
+Five record fields come from places the glossary does not list:
+
+- `label`: `aiTitle` on the runtime's title record, else `description` on a child's sidecar, else
+ `workflowName` on a workflow manifest.
+- `started_by`: a nested child's sidecar `parentAgentId`.
+- `tool`: `toolUseResult.toolUseId` on a tool result, else the `<tool-use-id>` of a completion
+ notification. The glossary's `tool` row is the step kind of the same name, so it does not list
+ this role's source.
+- `from`: the record type. `assistant` is `agent`, `user` is `external`, and `system` and
+ `attachment` are `runtime`, as is every record of a sidecar, journal, manifest or script. So
+ `from: external` also covers every tool result and every `isMeta` record, which sit on `user`
+ records. The flags and the parts say which is which.
+- `usage`: `message.usage`, the input, output, cache read and cache write tokens.
+
+### Names
+
+Three runtime fields carry names, and the page adds a fourth source.
+
+- The conversation's title is `aiTitle`, on the runtime's title record. The runtime rewrites it as
+ the work goes on, so the session node takes the last one written, and
+ [asz.view](../formats/asz-view.md#top-level) shows it as `summary.title`.
+- A child stream's label is `description` in its `agent-<id>.meta.json`.
+- A workflow's name is `workflowName` in its manifest. It is shown on the call that launched the
+ workflow.
+- A workflow child with no label is named from its run journal's result row: its `surface`, else
+ its `summary`, else its `verdict` and the first refuted claim. `asz.view` marks such a stream
+ `named_by: journal`. A workflow child's own records never say what it was for. Over 178 children
+ of one conversation, 177 opened with byte-identical injected context.
+
+A title was present in 41 of 42 conversations, and `system/turn_duration` on 465 turns, in a sample
+whose selection was not recorded. `asz.view` shows a turn's `duration_ms` as the runtime measured it.
## Workspace changes
@@ -315,22 +738,43 @@ formats, and that the fold is unchanged.
| | Why |
| --- | --- |
-| **Reasoning text** | Claude Code sends `thinking.display:"omitted"`, so the provider returns only a signature. Raw-body capture redacts it again. `unavailable` — not recoverable by any local mechanism. |
-| **Serialized request** | system prompt, tool schemas and cache annotations are absent from transcripts. `input_manifest` is `truncated`, never `full`, with **inferred** message membership. |
+| **Reasoning text** | Claude Code asks the provider not to return it, so most thinking blocks carry only a signature. `unavailable` where the runtime wrote none. See below. |
+| **Serialized request** | system prompt, tool schemas and cache annotations are absent from transcripts: 0 files contain `"tools":[`, and `compactMetadata.preservedMessages.allUuids` names 5 ids that exist nowhere on disk. asz produces no input manifest and no `input_of` relation, and reports model-context coverage as `unavailable`, as the model's [adapter contract](../concepts-and-designs/unified-conversation-model.md#adapter-contract) requires. |
| **Injected preamble** | the instruction block prepended to the first user message has no transcript record. Its *data* survives as `attachment` records; its rendered form does not. |
| **Per-call duration and cost** | not written to transcripts. Available via OTLP. |
| **`tool.execution`** | no local record; the call and result are observable, the execution is not. |
-| **Retry attempt identity** | derivable only for failures that received an HTTP response; transport failures carry no request id. |
+| **Retry attempt identity** | derivable only for failures that received an HTTP response: in the first pass, 48 of 556 `api_error` records carried a real request id in `error.requestId`, and transport failures carry none. asz does not derive it yet. |
| **What a shell command changed** | not in the transcript, which holds the command and its output only. Available from the [plugin](../setup/claude-code-plugin.md). The same for an edit inside a subagent, whose transcript carries no patch. |
+**Reasoning text** is kept wherever the runtime wrote it, and whether it did depends on the model. In
+the first pass, 2,603 of 2,604 files with thinking blocks had empty text. The one exception was a
+subagent on `claude-haiku-4-5-20251001`, with 31 blocks of 61 to 1,742 characters. In the same
+session at 2.1.227, eleven sibling agents of the same type on `claude-opus-5` kept none. A live
+capture showed the cause: the request sends `thinking.display:"omitted"`. Raw-body capture on 2.1.245
+redacts the text again, and OTLP events always redact it, so it is not recoverable by any local
+mechanism. An empty block is marked `unavailable`, and a `redacted_thinking` block is marked
+`redacted`.
+
+The injected preamble was measured on captured request bodies from one development machine. It is
+one 8,338-character `<system-reminder>` block wrapping the instruction file, the user's email and the
+current date. A search of the transcripts' fields finds it on no record, and subagents receive it
+too. The attachment data behind it survives, 88.8% to 98.7% recoverable, but the rendered wrapper and
+its ordering do not. The tool list also changes between adjacent calls through deferred loading, from
+96 to 116 tools one turn apart, and only a `deferred_tools_delta` attachment records it.
+
The injected preamble is a **fixed** cost — roughly 8 KB without a project instruction file, up to
~40 KB with one — so its share of a conversation falls as the conversation grows. It is not a
-proportional loss.
-
-Because the mapping from injected records to request blocks is **beta-gated and varies between client
-versions**, and the active beta list is not recorded, a transcript-only reconstruction of the exact
-request message list is not possible. Verifying model-context continuity against what was actually
-sent requires captured provider bodies.
+proportional loss. In a real session holding 12,027,973 characters of conversation content it is
+about 0.3%. A 26 KB probe session would suggest 99%, and is not representative.
+
+The mapping from injected records to request blocks is **beta-gated and varies between client
+versions**. On one machine on one day, two schemes were seen. Without the
+`mid-conversation-system-2026-04-07` beta, attachments become separate reminder-wrapped user text
+blocks placed before the prompt. That is the reverse of the transcript, which writes the user record
+first with its attachments as descendants. With the beta, they merge into one 13,953-character
+`role:"system"` message placed after the user message. The active beta list is not recorded, so a
+transcript-only reconstruction of the exact request message list is not possible. Verifying
+model-context continuity against what was actually sent requires captured provider bodies.
## OTLP (optional second channel)
@@ -345,8 +789,7 @@ It contributes **no structure**. Events carry no agent identity — `query_sourc
lineage exists only on spans, which require an additional beta flag; several span names present in
the binary are unreachable in shipped builds.
-Content defaults are also restrictive: prompts and responses are redacted unless explicitly enabled,
-content is truncated, and reasoning is always redacted.
+Its content defaults are restrictive, as [Collection](#collection) describes.
### The same metrics from the transcripts
@@ -379,20 +822,35 @@ The runtime's events, with the latency and cost a transcript never carries, are
and the question there is whether they land as Session Data joined to `llm.call` by the request
id or travel untouched.
+The receiver speaks OTLP because Claude Code has exactly one outward protocol. Its exporter setting
+accepts `console`, `otlp` or `prometheus` for metrics, and `console` or `otlp` for logs and traces.
+Any other value throws. There is no webhook and no custom sink. This was read from the Claude Code
+binary. The version was not recorded.
+
## Local data hazards
Behaviours that will break a naive reader:
-- **Session ids are not reliably lowercase.** Joins must be byte-exact and never case-folded.
-- **Lines can be very large** — the longest measured is 979,632 bytes, and files reach 59 MB. A
- scanner with a 64 KB token limit fails silently.
+- **Session ids are not reliably lowercase.** 2 of 47 transcripts used an uppercase UUID as their
+ session id, for example `41090DAB-113C-…`. Discovery accepts either case, and joins must be
+ byte-exact and never case-folded.
+- **Lines can be very large.** The longest measured is 4,629,074 bytes, and 22 lines are over
+ 1 MiB, on the 60-session storage root that [Storage Root](../formats/storage-root.md#append-cursors)
+ measured on 2026-09-11. Files reach 59 MB. A scanner with a 64 KB token limit fails silently.
- **`cwd` and `gitBranch` are per-record**, not session constants; one session showed 52 distinct
`cwd` values. Only `sessionId`, `entrypoint` and `userType` are reliable per-file constants.
-- **A large share of lines carry no record id at all** — metadata-only types that are not
- context-bearing. A parser assuming every line has one will fail.
-- **Some files contain no conversation records at all**, holding only metadata types. Treating every
- `.jsonl` as a session produces empty conversations.
-- **`tool_use_id` is not always prefixed `toolu_`** — server-side tools use a different prefix.
+- **A large share of lines carry no record id at all.** In the first pass, 38,340 of 133,887
+ main-transcript lines (28.6%) had no `uuid`, across 14 metadata-only record types, none of them
+ context-bearing. A parser assuming every line has one will fail. asz lands such a record at its
+ position, and never treats two records without an id as copies of each other.
+- **Some files contain no conversation records at all.** In the first pass, 177 of 2,935 files held
+ no `user`, `assistant` or `system` record, only metadata types. asz treats every
+ `<session-id>.jsonl` as a session regardless: it lands and assembles the file, and the list shows
+ the result as a conversation with no Talks. How many of the 177 were main transcripts was not
+ measured.
+- **Never match a tool-use id on its prefix.** An id starting `srvtoolu_` appears in the corpus, but
+ the 2,970-file pass found it only inside a `WebSearch` result payload, one level below the tool
+ call, where nothing joins on it. The join key is the exact id.
- **The corpus is live.** Files appear and grow during a scan; counts are snapshots.
- **`toolUseResult` is a main-stream enrichment.** On workflow children only the error-string form
survives, and the key is *absent* rather than null.
diff --git a/docs/en/changes/changes-0.3.0.md b/docs/en/changes/changes-0.3.0.md
index 60168b7..7603351 100644
--- a/docs/en/changes/changes-0.3.0.md
+++ b/docs/en/changes/changes-0.3.0.md
@@ -116,3 +116,9 @@
- `export.otlp.logs` and `export.otlp.metrics` switch the two things a push sends, the landed
files and rounds as logs and the metrics spool as metrics, each on its own, both on unless set
off. `asz push` says which it is sending, and a configuration with both off is refused.
+
+## Documentation
+
+- `design-notes/` is removed. What its three notes described that has landed is now documented in
+ these pages, checked against the code rather than copied from the notes, which were older than
+ the code. The notes remain in the git history. The documentation is the one reference.
diff --git a/docs/en/concepts-and-designs/conversation-assembly.md b/docs/en/concepts-and-designs/conversation-assembly.md
index fd1de2c..de1d7b4 100644
--- a/docs/en/concepts-and-designs/conversation-assembly.md
+++ b/docs/en/concepts-and-designs/conversation-assembly.md
@@ -9,6 +9,20 @@ content: removing duplicate records needs record ids, grouping a provider call n
joining a tool needs tool-use ids, joining a spawn needs agent and run ids. Message text is read only
when a conversation is rendered, and only for the records that appear in it.
+**Every parse assembles the whole session from its index**, not only the records that arrived since
+the last round. So the same evidence always gives the same entities, and a round can be re-derived
+rather than only trusted. [The round chain](#the-round-chain) says how the result becomes a round.
+
+Where this page says *in the corpus*, it means one corpus of Claude Code history: 2,970 source
+files holding 365,825 records, 1.09 GB in all, from 25 project directories. The date of that
+measurement is unavailable.
+
+Assembling everything each time is cheap because the index is small. In the corpus, 365,554 index
+entries describing 1.0 GB of landed payload took 47 MB of index, about 12 times smaller. The largest
+single session's index is 16 MB, and it loads in 4.5 ms with every lookup map rebuilt. That is why
+the lookup maps are rebuilt on load and never stored. [Storage Root](../formats/storage-root.md#size)
+gives the index size on a later sample.
+
```sh
asz parse # assemble every session, append a round to each chain
asz parse SESSION # one session
@@ -22,16 +36,16 @@ Eight stages. The order is forced: each depends on the one before it.
| Stage | What it establishes |
| --- | --- |
-| 1. Remove duplicate records | nothing downstream is correct first; the first copy wins |
+| 1. Remove duplicate records | the first copy of each record id wins, and nothing downstream is correct before it |
| 2. Partition streams | one ordered lineage per agent, from the file it was written in |
| 3. Group provider calls | by message id, in line order |
| 4. Join tools | request and result become one step |
| 5. Join child agents | which call started which child stream |
| 6. Cut context epochs | only from an explicit reset record |
| 7. Build Talks and Runs | the conversation a person reads |
-| 8. Propose segments | activity windows that can be committed |
+| 8. Propose segments | activity windows, and whether each could be committed |
-Six rules in here are the opposite of the obvious choice, and each is the opposite because a
+Seven rules in here are the opposite of the obvious choice, and each is the opposite because a
measurement said so.
**A record type is not what a record is.** Every tool result in the corpus sits on a record whose
@@ -43,6 +57,18 @@ just before it resets model context. The later copy is the worse one: it has a r
cycle, and in many cases its captured tool output has been blanked. Keeping the last both moves
records away from their true position and destroys data.
+The key is the record id across the whole session, not within one file. First means first in
+landed order: the lower sequence, then the lower row. So the same rule also removes the copies that
+a crash between landing a file and committing its cursor leaves in a later file, as
+[Storage Root](../formats/storage-root.md#landed-files) describes. A record with no id is never a
+duplicate, because a key built on an empty id would collapse every such record into one. In the
+corpus, 44,585 of 365,825 records carry no id.
+
+A repeated copy is left out of every lookup, not only the lookup by id. In call grouping it would
+inflate a call's fragment count. In the tool join it would make an exact join look ambiguous. In a
+stream it would appear twice. The copy stays in the index as evidence of what the runtime wrote
+again.
+
**Group a provider call by message id, in line order.** Never by walking the parent pointer forward,
which drops a tool use on nearly one call in five; and never by request id, because a record the
client fabricated can carry a real call's request id.
@@ -57,9 +83,92 @@ said anything; it is the same interaction continuing. A locally typed command is
a person acting with no trigger recorded — so both are Talk starts and everything between them is
not.
+**A run id belongs to its stream.** A run id is not unique across a session. A child stream is
+written under a run id that also appears in the parent, and one id has been seen in three different
+child streams. Keyed on the id alone, a child's records land inside the parent's Talk, and the child
+gets no run of its own. So every Talk and Run is keyed by the stream and the run id together.
+
**Membership is resolved by walking backward.** A model response carries no cycle id but reaches one
through its containment parents, which is single-valued. Line proximity agrees in the simple case and
-disagrees exactly where it matters.
+disagrees exactly where it matters. The walk passes through records of every kind. An attachment
+carries a parent and is named as one, so a walk that skipped every record that is not a message
+would break. The walk follows at most 64 parent links, and the limit exists only so that corrupt
+data cannot loop. A small limit would be wrong: in the corpus, 140 of 44,973 records need more
+than 7.
+
+One rule is asserted rather than measured.
+
+**A child stream is always exactly one Talk.** Every run in it is a Run inside that Talk: the
+delegated prompt comes in, the final output goes out, and nobody outside the agent takes part. The
+Talk's trigger is `unknown`, because the parent wrote the prompt. The rule follows from what a child
+stream is. The main stream's rule cannot stand in for it, because that rule reads the trigger each
+run states, and in the corpus exactly one of 221,592 child records states a trigger. Applied to a
+child, it would start a new Talk on every run.
+
+### Segments
+
+Stage 8 proposes segments. It never commits one.
+
+Only the main stream is cut. Its Talks are taken in order of their first record time. A new window
+opens where the gap between one Talk's last record and the next Talk's first record is longer than
+the idle gap. Only records that resolve to a run count toward a Talk's time range. A window's id
+comes from where it opens, the landed position of its first Talk, and not from its place in the
+list. So late evidence that adds an earlier window cannot renumber a later one. The last window is
+`open`, and every earlier one is a `candidate`. A child stream's Talk joins the window whose time
+range holds its first record, as `strong_inference`, because delegated work belongs to the window
+of the Talk that asked for it.
+
+**A negative gap is no gap.** Record times can step backward. In the corpus, 0.315% of consecutive
+records step back, by up to 4.06 hours. Provider-error records are the main cause: their time is
+when the failed request began, not when the record was written. So a negative difference between
+two Talks is treated as no gap, never as a large one.
+
+**The idle gap is 10 minutes, and a gap only proposes.** A long gap often falls inside a turn rather
+than between two, so it cannot decide on its own. In the corpus:
+
+| Idle gap | Gaps that long | Share inside a run | Share inside a Talk |
+| --- | --- | --- | --- |
+| 10 minutes | 697 | 23.67% | 38.74% |
+| 15 minutes | 444 | unavailable | 30.41% |
+| 20 minutes | 302 | 9.60% | 21.52% |
+
+The Talk column is the one that matters. A window cut inside a Talk splits one interaction between
+two windows. Whether provider-error records count on this timeline moves the run column by five
+percentage points. In asz such a record counts only when its containment parents reach a run,
+because only those records set a Talk's time range.
+
+Twenty minutes would propose 57% fewer candidates. The default stays at 10 minutes because the gates
+below decide what could be committed, and a shorter gap only gives them more candidates to judge.
+How often the gates stop a bad candidate has not been measured.
+
+The gap is fixed in the build. No configuration key and no command flag changes it. It is part of
+the policy every round header carries, `v1+idle=10m0s`, so windows proposed under one gap never
+fold together with windows proposed under another. [Session Flow](../formats/session-flow.md#header)
+says how a chain refuses a change of policy.
+
+**Four gates decide whether a window could be committed.**
+
+| Gate | Passes when |
+| --- | --- |
+| `activity_boundary` | the window is not the last one |
+| `no_crossing_open_operation` | every tool use whose request time falls inside the window has exactly one result, also timed inside the window, and the session holds no open `child_stream` or `notified_child` entry. The second test covers the whole session, not the window. |
+| `lateness_watermark` | the window is not the last one, and the main stream's last record is more than one idle gap after the window ends |
+| `conversation_identity` | the conversation id is not empty. Assembly refuses to run without one, so this gate always passes. |
+
+A segment node carries `state`, `talks`, `gates_unmet` and `committable`. `gates_unmet` lists the
+gates that fail, and `committable` is true when none fail. [asz.view](../formats/asz-view.md) shows
+the same windows to a reader.
+
+`no_crossing_open_operation` is the gate that does real work. It does not exist for a request left
+with no result at the end of a file. In the corpus that is 0 of 25,892 tool requests on the main
+stream, because the runtime writes a result even for a failure. It exists for a result that arrives
+after the quiet period, which in the corpus happens on 85 of the 697 gaps longer than 10 minutes,
+12.2%. Committing that window would freeze the request in one window while its result lands in the
+next.
+
+**Nothing commits a segment.** Assembly proposes windows and reports `committable`, and no command
+acts on it. The [model](unified-conversation-model.md) calls a segment the commit unit, but the
+commit step is not implemented.
## What comes out
@@ -73,15 +182,25 @@ session
└ run
├ llm.call
│ ├ message.assistant · thinking
- │ └ tool
+ │ └ tool · agent.call
├ message.external · context.injection
- ├ agent.call → agent.launch_ack
+ ├ agent.launch_ack
└ runtime.notification
```
+A `tool` becomes an `agent.call` when a child stream is joined to it. Only its kind changes, so it
+stays under the provider call whose response asked for it. The `agent.launch_ack` is a separate
+record, and it sits in the run.
+
Everything else is a **typed relation** carrying its own correlation quality: `starts`, `reports`,
-`ends_with`, `result_of`, `follows`, `summarizes`, `in_segment`. Cross-stream flow is never containment,
-which is what stops a rendered conversation repeating every subagent's work inside its parent.
+`ends_with`, `follows`, `summarizes`, `in_segment`. Cross-stream flow is never containment, which is
+what stops a rendered conversation repeating every subagent's work inside its parent.
+
+A tool's request and result are not joined by a relation. The tool node references the request
+record and, once the join finds it, the result record. Those references are the join. Its quality
+is the node's `result_join` attribute. An edge would say the same thing again, and its far end would
+be a record with no node of its own. So assembly never writes the `result_of` relation that the
+[Glossary](glossary.md#relations) lists.
A Segment is a relation rather than a parent. It is a time window and a session outlives many of
them, so a session cannot sit under a segment in a tree where every node has one parent.
@@ -113,18 +232,60 @@ Three rules govern the fold, and each exists because its opposite loses informat
backwards; the chain cannot.
Rounds are linked by digest, not by filename: round N names the digest of round N−1. A round carries
-no wall-clock time, so the same landed evidence and the same parser version reproduce the same bytes
-and the same digest. Anything mutable or temporal lives in `conversation.state`, outside every
-digest.
+no wall-clock time, so the same landed evidence and the same parser and policy versions reproduce
+the same bytes and the same digest. Anything mutable or temporal lives in `conversation.state`,
+outside every digest.
+
+A parse builds a round in three steps. It fixes its watermark at the sequence the index has
+reached. It assembles the whole session up to that watermark. It then compares the result with the
+fold of the chain and writes only the difference:
+
+- An entity assembled unchanged is left out, because absence means unchanged.
+- A node or relation the fold holds and the assembly no longer produces gets a tombstone.
+- An open unresolved entry the assembly no longer produces is superseded as `resolved`.
+
+The comparison ignores an entity's revision and its frame type. The revision is the round number,
+and only the writer sets the frame type, so comparing either would make every entity look changed
+in every round. Because the whole session is assembled each time, duplicate removal always sees the
+first copy. A repeated record therefore cannot become a second node in a later round. A round over
+the byte budget is assembled again over a narrower window, as
+[`max_round_bytes`](../setup/configuration.md#parse) describes.
+
+**The watermark is fixed before assembly, and assembly reads nothing past it.** Deciding it
+afterwards would let a round hold nodes drawn from evidence that its own header and input digest do
+not cover. A collector writing at the same time makes that likely. A parse also refuses to run when
+the chain already covers more landed evidence than the index holds.
-A viewer folds the same chain, but not necessarily the same way: it may want the state as of an
-earlier round, or one entity's history across rounds, or to render rounds as they arrive. That is a
-later concern. What the chain guarantees is that the rounds hold enough to answer any of them.
+**A round is written whenever landed evidence advanced**, even when no entity changed. Such a round
+has no entity frames and only moves `through_seq`. Without it, the next parse would read the same
+evidence again and again, and "no round" would mean both "nothing new arrived" and "nothing new
+mattered". Records landed twice after a crash are one such case. The chain test
+`TestInterruptedPassRepeatsRatherThanLoses` checks that they move the watermark with no node,
+relation or tombstone. The scenario `replay` checks the same for a repeated block.
-`asz verify` checks four things. Three are digest properties — no gap in the round numbers, each
-round names its predecessor's digest, each round's digest matches its bytes. The fourth is that
-consumed landed sequences are contiguous across rounds, which catches skipped evidence that no digest
-would reveal.
+**The fold of rounds 1 to N equals one full assembly of the landed evidence through round N's
+`through_seq`.** The full assembly is the reference, because it is simple and plainly correct.
+Where the two disagree, the fold is wrong. Every [scenario](../guides/scenario.md#check) checks
+this in both formats, on every node's kind, parent and attributes and on the number of relations.
+It holds under any round budget, because the budget changes where the chain is cut, never what it
+says. A chain test builds one session as a single round and again under a one-byte budget, one round
+per landed file, and the two folds hold the same entities. The rule does not say that the fold of
+round N+1 extends the fold of round N. Later evidence replaces entities that earlier rounds wrote.
+
+**Rounds and segments are separate.** The round is the unit of appending. The segment is a proposed
+commit window. A segment is a node like any other, so when a later Talk turns the open window into a
+candidate, that change is an ordinary node revision in whatever round the parse writes. There is no
+separate mechanism for it, and nothing freezes a segment today, as [Segments](#segments) says.
+
+The page that `asz view` serves folds the same chain in the same way, every round in order.
+[asz.view](../formats/asz-view.md) says what it does with a broken chain, and when it folds again.
+A fold as of an earlier round, one entity's history across rounds, and a fold that takes rounds as
+they arrive are not implemented. The rounds hold enough to support each of them.
+
+`asz verify` checks every round against the round before it and against the landed files, as
+[Session Flow](../formats/session-flow.md#reading-it) lists. One of those checks is not about
+digests: the landed sequences the rounds consumed must be contiguous. It catches skipped evidence
+that no digest would reveal.
## Qualification
@@ -135,7 +296,35 @@ Every join carries a correlation quality: `exact_unique`, `exact_ambiguous`, `st
where several candidates share a key the relation stays `exact_ambiguous` and the assembler does not
choose one.
-What could not be resolved is carried as data, with a denominator. An assembler that drops what it
-could not resolve presents a partial conversation as a complete one. An entry is promoted to terminal
-only on terminal evidence — a pruned source, a finished run — never because it has stayed open for a
-number of rounds, which is a statement about how often we looked rather than about the data.
+What could not be resolved is carried as data. An assembler that drops what it could not resolve
+presents a partial conversation as a complete one.
+
+A count of what could not be resolved also needs the number it is out of. Without it, a reader
+cannot tell a small gap from a large one. The rounds do not carry that number. An unresolved entry
+records its kind, its reference, a reason and a state. Only [`asz parse`](../setup/command-line.md#parse)
+prints the number, and only for tool calls and child launches.
+
+The assembler writes seven kinds of unresolved entry:
+
+| Kind | Written when |
+| --- | --- |
+| `tool_result` | no result landed for a tool use, or several results carry its id |
+| `tool_use` | a result landed, and the request it answers never did |
+| `child_stream` | a child was announced, and its stream has not landed |
+| `notified_child` | a notification names a child whose stream has not landed |
+| `spawn_call` | a child, or a batch of children, names a launch call that never landed |
+| `spawn_of_child` | a child stream landed, and no landed record names the call that started it |
+| `epoch_continuation` | a reset names a record that is not in the landed data |
+
+Every entry is written `open`, and it resolves when the missing evidence lands. `child_stream` and
+`notified_child` are routine in a live session, because a child can be announced before its stream
+lands. While either is open, no segment is committable. A child in a batch that never reports a
+result is normal, and it is not recorded.
+
+The assembler makes one change of state. An open entry it no longer produces is superseded as
+`resolved`, and [the fold](../formats/session-flow.md#the-fold) keeps it. The scenario
+`unfinished-tool` checks a `tool_result` going from open to resolved. The format also defines
+`terminal`, for a producer with evidence that a reference will never resolve, such as a pruned
+source. The current assembler never writes it. Nothing counts rounds either. "Still open after N
+rounds" measures how often the chain was built, not the data, which is why the state has three
+values and not a counter.
diff --git a/docs/en/concepts-and-designs/glossary.md b/docs/en/concepts-and-designs/glossary.md
index 3eff1d6..13bb0ab 100644
--- a/docs/en/concepts-and-designs/glossary.md
+++ b/docs/en/concepts-and-designs/glossary.md
@@ -136,8 +136,15 @@ later agreeing observation.
## Storage
-**Landed record** — a source record copied into the storage zone, byte-for-byte. Write-once and
-read-only: temp file, fsync, rename, `chmod 0444`. Never appended to.
+**Landed record** — a source record converted once into
+[Session Data](../formats/session-data.md#record) and written to a `.sd` file. Its content becomes
+parts, and its identifiers take role names. It does not keep the source bytes. It keeps where they
+were and what they were: the line number counting from 1, the byte offset, their size, and a
+digest, the first 12 hexadecimal characters of their SHA-256. Content the dialect cannot describe
+keeps its bytes verbatim in an `unknown` part, so nothing is guessed or dropped. A reader is handed
+parts, so it never meets a runtime's shape. The digest still ties each record to its source.
+Write-once and read-only: temporary file, fsync, `chmod 0444`, rename. Never appended to.
+*Not:* a copy of the source record.
**Delta** — one file of landed records, covering a contiguous run of a source. A source becomes many
deltas over time.
diff --git a/docs/en/concepts-and-designs/overview.md b/docs/en/concepts-and-designs/overview.md
index 48888ef..7e695e5 100644
--- a/docs/en/concepts-and-designs/overview.md
+++ b/docs/en/concepts-and-designs/overview.md
@@ -40,11 +40,25 @@ Where a runtime cannot supply something, the adapter reports it as `unavailable`
approximating it. A missing model input, an unrecorded parent, or an unobservable causal link is
never manufactured to make a tree look complete.
-## Status
-
-Pre-alpha. The conversation model and the Claude Code data mapping are defined and evidence-backed.
-Collection, the derived index and assembly into a round chain are implemented. A local page that
-serves the assembled conversations is in progress. Static export and OTLP push are not started.
+## What is implemented
+
+The conversation model and the Claude Code data mapping are defined and evidence-backed. These
+parts are implemented:
+
+- **Collection, the derived index and assembly** into a round chain. `asz collect` runs them as one
+ pipeline. When `export.otlp.endpoint` is set, it also sends at the end of every pass.
+- **The local page.** [`asz view`](../setup/command-line.md#view) serves the assembled
+ conversations and only reads. [`asz server`](../setup/command-line.md#server) runs the pipeline
+ and the page in one process.
+- **Export over OpenTelemetry.** [`asz push`](../setup/export-otlp.md) sends the landed files and
+ the rounds to an OpenTelemetry logs receiver. It also sends the token metric when an adapter
+ produces it.
+- **One conversation as one document.** [asz.view](../formats/asz-view.md) is what the page draws
+ and what `asz conversation -json` prints.
+- **A storage root that travels on its own.** A copy of it can be verified, re-parsed and read on
+ another machine with no source files. See [What travels](../formats/storage-root.md#what-travels).
+
+A static export is not built.
## Non-goals
diff --git a/docs/en/concepts-and-designs/unified-conversation-model.md b/docs/en/concepts-and-designs/unified-conversation-model.md
index 5c810dd..3bb8aae 100644
--- a/docs/en/concepts-and-designs/unified-conversation-model.md
+++ b/docs/en/concepts-and-designs/unified-conversation-model.md
@@ -110,6 +110,7 @@ rather than synthesised.
| | `control.interrupt` | an externally initiated interruption |
| | `control.permission` | a permission decision or mode change |
| | `control.command` | a runtime command invoked in-line |
+| | `control.notice` | the runtime telling the requester something about the session itself, such as a lost connection, a fallback it took, or a summary of what happened while the requester was away |
| | `turn.duration` | a runtime-reported duration for a completed turn |
**A tool use is one step, not three.** It carries `name`, `input` (for a shell tool, the command
@@ -239,6 +240,37 @@ attempt identity.
An adapter that cannot expose ordered provider input must report model-context coverage as
`unavailable`. Timing and cache counters do not prove model-input membership.
+**A glossary for its dialect**
+
+The model uses its own names. A reader can ask for the runtime's words instead, or for both side by
+side, with `-terms` (see [Command Line](../setup/command-line.md#flags)). An adapter that lands
+Session Data makes this possible. Every landed header names the
+[dialect](../formats/session-data.md#header) its records were read as, and the adapter declares a
+[glossary](glossary.md#adapters) for that dialect. The glossary says what the runtime calls each
+name the model can emit, and where that word appears in the runtime's records.
+
+A glossary belongs to a dialect, not to an adapter. A local reader and a push receiver for one
+runtime read the same schema, so they share one glossary. Records of a different shape get a
+different dialect, even when the same runtime produced them.
+
+Three rules keep a glossary complete and checkable:
+
+1. **Every name the model can emit has an entry**, even when the runtime has no word for it. A
+ reader that asks for native words is shown the model's own name wherever there is no native
+ one. So a missing entry looks exactly like a thing the runtime never records. Without this rule,
+ the reader cannot tell "no native name exists" from "nobody wrote one down".
+2. **An empty native name is an answer.** It says the runtime has no word for the thing, because
+ the model derived it. The entry carries a note that says what the thing is. It names no
+ location, because there is no field in the runtime's records to point to.
+3. **A native name says where it appears** in the runtime's records. Without that, a reader learns
+ a word but not where to look for it.
+
+The model lists the names a glossary is checked against in `model.Vocabulary`. For the
+[built-in runtime adapter](../adapters/claude-code-glossary.md), a test fails when an entry breaks
+rule 2 or rule 3, or when a name in that list has no entry. The list leaves out one name the model
+emits, `control.notice`. So no test would fail if the entry for it were missing. The built-in
+glossary has that entry.
+
## Non-goals
- Grouping work by human or principal identity.
diff --git a/docs/en/formats/asz-view.md b/docs/en/formats/asz-view.md
index 295912d..c2ace11 100644
--- a/docs/en/formats/asz-view.md
+++ b/docs/en/formats/asz-view.md
@@ -7,8 +7,10 @@ steps with the text the referenced records carry, and the streams, the segments,
the rounds and the files sit beside them, each verified. The evidence is inside it once, so a
viewer never opens a `.sd` file; the `ref` on a node is a citation, not a pointer to fetch.
-It is never a file that the project writes. Package `pkg/sessionview` defines and owns the shape,
-and three things produce it from the same code:
+It is never a file that the project writes. A stored copy would have to be kept in step with the
+rounds and the landed files, and it would be the copy that falls out of date. Built from them each
+time, it always agrees with them. Package `pkg/sessionview` defines and owns the shape, and three
+things produce it from the same code:
| How | What you get |
| --- | --- |
@@ -110,7 +112,7 @@ usable round at all is an error, because there is nothing to show.
| `ref`, `refs` | the record it stands on and every record it covers, as `{seq, row, block}`, kept so a viewer can show the evidence |
| `text`, `state`, `bytes` | the part the node stands on: its readable text, clipped to the longest prefix of whole characters within 2,000 bytes, whether the content is `available`, and its full size. For a `data` part the text is the data as compact JSON. A reader wanting the whole record reads it by address. |
| `usage`, `flags`, `dropped` | what else the referenced record says, copied once: on an `llm.call`, the token counts `in`, `out`, `cache_read`, `cache_write` from the one record `usage_at` names, never a sum over fragments; the record's `flags`; and its `dropped` list, so a viewer can say what was left out and why |
-| a talk adds | `label`, `reply` (its last assistant message, clipped the same way), `runs`, `steps`, `tools`, `from`, `to`, `child`, `segment` |
+| a talk adds | `label` and `reply`, clipped the same way and described below, then `runs`, `steps`, `tools`, `from`, `to`, `child`, `segment` |
| a tool or agent call adds | `name`, `failed`, `result`, `result_state`, `result_bytes`, `request_to_result_ms` and `request_to_result_join`, the time from the request record to the result record where the assembler joined them exactly |
| a `turn.duration` step adds | `duration_ms`, `duration_measured_by` |
| `children` | containment, in record order: a talk holds runs, a run holds steps, a call holds what it produced |
@@ -119,6 +121,26 @@ usable round at all is an error, because there is nothing to show.
Keys a node has no value for are absent, not null. Nothing in a document is inferred beyond what
the fold and the records say. Where the fold says `unavailable`, the document says it too.
+**A talk's label and reply.** A talk's `label` is the text of the first `message.external` step in
+it. A talk can have none. A talk opened by a command typed locally is one example, because the
+runtime records no origin for such a command. The label is then the first text that is not empty
+among the talk's first three `context.injection` steps, in record order. A text that starts with
+`{"type":"deferred_tools_delta"` is skipped. It is Claude Code's record of a change in the tools
+available, and it says nothing about the work. So a label is not always something a person typed.
+A talk's `reply` is the last `message.assistant` or `agent.output` step in it, in record order. The
+earlier messages are what the agent said between tool calls, as the
+[Unified Conversation Model](../concepts-and-designs/unified-conversation-model.md#hierarchy)
+explains. Either key is absent when no step gives it a text.
+
+**Request to result is not tool time.** `request_to_result_ms` is the time between two records the
+runtime wrote, the request and its result, tied together by the tool-use id. It is not how long the
+tool ran. It can include waiting and other work between the two records. So `timing` in `attrs`
+stays `unavailable` beside it, for the reason the
+[Claude Code adapter](../adapters/claude-code.md#step-mapping) gives. Both keys are absent when the
+join is not `exact_unique`, when either record has no time, or when the result is earlier than the
+request. A gap under one millisecond keeps `request_to_result_join` and leaves out
+`request_to_result_ms`, because its value is zero.
+
## Workspace changes
`workspace_changes` lists which files each tool call changed, one entry per change record, in
@@ -152,7 +174,7 @@ checks the property `view_covers_the_session` at its end. This is how each view
| View | Read |
| --- | --- |
-| **Transcript** | `talks`, in order. Each talk's `label` is the person's input and its `reply` the last assistant message; its `children` are the runs, a run's children the steps, and a call's children what it produced: thinking, messages, tools. `text` is what to show for a step, `name` and `result` for a tool, `usage` on a call. `loose` holds whatever no talk contains, and is usually empty. |
+| **Transcript** | `talks`, in order. Each talk's `label` is usually the person's input and its `reply` the last thing the agent said in it, as [A node in `talks`](#a-node-in-talks) defines them; its `children` are the runs, a run's children the steps, and a call's children what it produced: thinking, messages, tools. `text` is what to show for a step, `name` and `result` for a tool, `usage` on a call. `loose` holds whatever no talk contains, and is usually empty. |
| **Flow timeline** | every node of every tree by `at`, with `kind` and `stream`; a node with `at` of `0` was never observed at a time and is placed by its position. |
| **Cross-stream flow** | `edges` on a node, and `relations` as the whole list: `starts` from an agent call to the child's `stream`, `reports` from the notification that resumed the parent, `ends_with` from a stream to the child's output, `follows` between epochs across a reset, `summarizes` from a summary to its boundary, `in_segment` from a talk to its window. Containment never crosses a stream; a child's work is under the child. |
| **Streams and segments** | `streams` with `role`, `label`, `parent` and `opened_by`, the step that started each; `segments` with the span of the talks placed in them. |
@@ -180,3 +202,22 @@ document with Horizon's conversation renderer, which asz embeds from a pinned Ho
conversation looks the same in `asz view` and in the SkyWalking UI.
The largest conversation measured, 357 talks and 16,121 steps, is 19 MB as one document and was
built in 0.7 s.
+
+A running `asz view` notices new rounds without being told. On every read it lists the
+conversation's rounds directory and compares the newest round there with the round it folded. When
+they differ, it folds again and builds a new document. So an `asz view` beside a separate
+`asz collect` stays current. The comparison is with the round the fold reached, not the newest one
+listed, so a round that was still being written at one read is picked up by the next.
+
+The page writes nothing derived to disk, such as a read index. It folds the chain when asked,
+because folding was measured to be fast enough. The sample was the largest session of a corpus of
+62 Claude Code sessions, 1,100.1 MB of landed records. It holds 53,106 nodes and 922 talks.
+
+| On that session | Took |
+| --- | ---: |
+| fold the whole chain | 302 ms |
+| build the ordered talk list | 1 ms |
+| one talk's subtree, at the 99th percentile | 13 microseconds |
+| walk every talk's subtree | 11 ms |
+
+A read index is added only when reading one talk takes more than 200 ms at the 99th percentile.
diff --git a/docs/en/formats/session-data.md b/docs/en/formats/session-data.md
index 901a59c..1935a47 100644
--- a/docs/en/formats/session-data.md
+++ b/docs/en/formats/session-data.md
@@ -2,8 +2,9 @@
Session Data is what was actually in a conversation, in one shape regardless of which agent
produced it. A landed file has the extension `.sd`. Its first line is a header carrying everything
-constant for the file. Every line after it is one source record, converted, with its content broken
-into parts named for what they are: a message, a thought, a call, its result.
+constant for the file, and its last line closes the file with a digest. Every line between them is
+one source record, converted, with its content broken into parts named for what they are: a
+message, a thought, a call, its result.
The conversion happens once, while the source is read, and never again. A runtime's vocabulary
stops at its adapter. A reader is handed parts, so there is no runtime shape for it to reach into.
@@ -13,6 +14,19 @@ rather than guessing or dropping them.
The format is public. Package `pkg/sessiondata` defines it, and a third-party adapter produces it.
The schema is `sd/1`.
+## One record per line
+
+Every record is one line, for two reasons.
+
+A row is a line. A round addresses a record by the file's sequence and the record's line in it,
+two integers rather than a path into nested structure. They stay valid because a landed file is
+never rewritten. See [Addressing a record](#addressing-a-record).
+
+A reader can decode one record at a time and never hold the whole file. A source record is landed
+whole however large it is (see [Append cursors](storage-root.md#append-cursors)), so a line has no
+fixed length. A line reader with a fixed limit, such as Go's `bufio.Scanner` at its 64 KB default,
+stops at the first longer line.
+
## Header
```json
@@ -27,12 +41,30 @@ The schema is `sd/1`.
| `schema` | `sd/1` |
| `seq` | the landed sequence number, monotonic per session |
| `at` | when the file was collected |
-| `kind` | what it was collected from: `transcript`, `agent_meta`, `journal`, `workflow_manifest`, `workflow_script`, `changes`, `provider_body`. `otlp_log` and `otlp_span` are reserved for a push transport. |
+| `kind` | what it was collected from: `transcript`, `agent_meta`, `journal`, `workflow_manifest`, `workflow_script` or `changes`. `otlp_log` and `otlp_span` are reserved for a push transport, and `provider_body` is reserved too. No adapter writes any of the three. |
| `adapter` | how the records were acquired, with its contract version |
| `dialect` | whose schema they were read as. A push receiver and a local reader for one runtime share a dialect and nothing else. |
-| `src` | the source, relative to the adapter's root |
+| `src` | the source, relative to the adapter's source root, with forward slashes on every platform |
| `session`, `stream`, `batch` | the session, the execution stream (`main` or an agent id), and the group of children a workflow run started |
+A reader refuses a header unless `h` is `1`, `schema` is `sd/1`, and `kind`, `session`, `src` and
+`dialect` are all present. The writer in `pkg/sessiondata` fills in `h` and `schema`, and refuses
+to start a file whose header fails the same check. `dialect` is required because without it nothing
+says whose vocabulary the parts were read in. A later reader could not tell a shape it should
+understand from one it should not.
+
+Because the header holds what is constant, no record repeats it, and a file can be read on its own,
+away from the directory it was landed in. It names its session, its stream or batch, its kind and
+its sequence.
+
+`src` is kept because the storage root does not follow the source's layout. The root files a
+session under its id and puts every child stream flat under `streams/`, keyed by agent id (see
+[Storage Root](storage-root.md)). For Claude Code, that drops two things the source path holds: the
+project directory a source file sits in, and the workflow directory a child's transcript sits in.
+`src` is the only place both survive. It uses forward slashes on every platform, so a root reads
+the same on the machine that collected it and on any other. With a record's `off` and `sha`, it
+lets a reader check the record against its source for as long as the source exists.
+
## File kinds
One `.sd` file holds one cut of one source. The header's `kind` says which, and the records of
@@ -47,6 +79,11 @@ each kind carry a characteristic set of fields.
| `workflow_script` | the program a workflow ran | one record whose part is `unknown`: the source is a program, not data | nothing; kept because it is part of the session |
| `changes` | the files the asz Claude Code plugin saw a tool call change, one line per call | one record per observed call, with `id`, `tool` and `time` lifted from it and the line whole as one `data` part, a `changes/1` record; no `from` and no flags, so assembly emits no node for it | nothing in a round; the view joins each to its step by `tool`. A transcript's `Edit` and `Write` results carry the same record as a second `data` part, from the runtime's own patch. See the [Claude Code [...]
+A source that is one document, not a stream of lines, lands as one record with `ord` 1 and `off` 0.
+Its `sha` and `bytes` cover the whole document. The Claude Code adapter's `agent_meta`,
+`workflow_manifest` and `workflow_script` files are of this kind, and a new version of the document
+lands as a new file (see [Snapshot cursors](storage-root.md#snapshot-cursors)).
+
Files of every kind are bound by the rounds' `input_digest`, so a session travels or archives as
all of them. A session's `agent_meta`, `workflow_manifest` and `workflow_script` records carry
no time, so those files have no record time range.
@@ -67,12 +104,12 @@ role names. The part level carries what the content is.
| Field | Meaning |
| --- | --- |
-| `ord`, `off`, `sha`, `bytes` | where the record was in the source: line number, byte offset, digest of the source bytes and their size. The bytes themselves are not kept; the digest stays so provenance is provable, and a record that claims a source it did not come from is detectable. |
+| `ord`, `off`, `sha`, `bytes` | where the record was in the source: its line number counting from 1, the byte offset its line starts at, the first twelve hexadecimal characters of the SHA-256 of its bytes, and their size. The bytes themselves are not kept. The digest stays, so provenance is provable and a record that claims a source it did not come from is detectable. |
| `id`, `parent` | the record's own identity and its containment parent |
| `call` | the provider call this record is a fragment of |
| `run` | the agent loop it belongs to, one per trigger |
| `continues` | on a reset boundary only, the record the new context resumes from |
-| `tool`, `child`, `batch`, `started_by` | joins the runtime states outside the content: the call a sidecar answers, the child stream a record names, the group it belongs to, the stream that started this one |
+| `tool`, `child`, `batch`, `started_by` | joins the runtime states outside the content: the call a record refers to (a notification names the call it completes), the child stream a record names, the group it belongs to, the stream that started this one |
| `label` | a name the runtime gave something. The only naming evidence in the data. |
| `from` | who produced it: `agent`, `external`, `runtime`. A record's type is not what a record is; most records that look like a person are a tool answering. |
| `time`, `trigger`, `flags` | when, what started the loop, and states such as `finished` |
@@ -84,6 +121,47 @@ role names. The part level carries what the content is.
Empty identifiers are common and mean the runtime supplied none. Nothing is inferred to fill
them.
+`child_result` is not only on a `journal` record. On a `transcript` record it marks a result that
+is a child's synchronous return, which is the parent's copy of the child's output. Assembly reads
+the flag only on a `journal` record. Nothing removes the parent's copy, so it stays as the result
+of the call that started the child.
+
+`asz verify` checks `ord`, `off` and `bytes` without the source. In each stream or run, across its
+files of one kind, `ord` must run 1, 2, 3 with no gap, and each record must start at the byte after
+the previous record's line ends. A skipped line would otherwise show only as a shorter
+conversation. A repeated record is not a gap (see [Landed files](storage-root.md#landed-files)).
+
+### Addressing a record
+
+A round points at a record with `{seq, row}`, and at one of its parts with `{seq, row, block}`
+(see [Session Flow](session-flow.md#entities)). In Session Data terms:
+
+| Field | Is |
+| --- | --- |
+| `seq` | the `seq` in the file's header |
+| `row` | the record's line after the header, counting from 1 |
+| `block` | the part's position in `parts`, counting from 0 |
+
+The index numbers every part, including parts it has no use for, so a number names the same part
+to the index, to a round and to a reader. A dialect writes one part for each piece of content in
+the source record, in source order. Content it cannot describe goes in as an `unknown` part in the
+same place, never left out. A part the adapter adds beside the content, such as the `changes/1`
+record on an edit result, comes after them.
+
+## Closing line
+
+```json
+{"t":"end","records":118,"digest":"c41e…"}
+```
+
+Every file ends with this line. `records` is the number of record lines, and `digest` is the
+SHA-256, in hexadecimal, of every byte before this line: the header and every record, each with its
+newline. The writer computes it as it writes, so a file can be checked on its own.
+
+A reader refuses a file with no closing line, or one whose digest or count does not match. A file
+cut short would otherwise read as a shorter conversation, with nothing saying so. `asz verify`
+reads every landed file this way.
+
## Parts
| Kind | Is | Carries |
@@ -91,22 +169,46 @@ them.
| `text` | readable text | `text` |
| `reasoning` | the model's own reasoning | `text` when the runtime kept it |
| `call` | a request to run something | `id`, `name`, `data` |
-| `result` | what a call returned | `of`, `text` or `data`, `failed` |
+| `result` | what a call returned | `of`, `failed`, and `text`, `data` or both |
| `media` | an image or a document | `media`, `data` |
| `data` | structure that is not prose: a record the runtime keeps for itself, a manifest | `data` |
-| `unknown` | content the dialect could not describe | the raw bytes in `data` |
+| `unknown` | content the dialect could not describe | the raw bytes as one JSON string in `data`, and the reason in `text` |
Every part carries `state` and `bytes`. `state` is one of `available`, `truncated`, `redacted`,
`omitted` or `unavailable`, and `bytes` is the size of the original even when the part holds less.
-A reader is always told how much of the original it has. One tool result in the measured corpus
-was 1.1 MB, and the share of a session that is tool output ran from 17% to 84%, so a reader is
-routinely shown part of something and has to know it.
+A reader is always told how much of the original it has.
+
+No fixed rule could decide how much of a part to keep or show, because tool output varies too much.
+The measured corpus is 62 Claude Code sessions: 3,032 files and 1,100.1 MB of source records. One
+tool result in it was 1.1 MB. Over its 28 sessions larger than 1 MB, the share of a session that is
+tool output ran from 17.3% to 83.5%, with a median of 35.1%. A rule tuned to the median is wrong by
+a factor of two at both ends. So every part states its own size and state, and a reader decides.
+No built-in adapter writes `truncated` or `omitted`, so no part they write is clipped. Clipping is
+left to a reader, such as the [asz.view](asz-view.md#a-node-in-talks) document, which clips a
+node's text and gives the full size beside it. A part is still not always all the runtime gave.
+The Claude Code dialect leaves some content out with no `dropped` entry. Its
+[Session Data mapping](../adapters/claude-code.md#session-data-mapping) lists these as known gaps.
+
+An `unknown` part's `text` is not content. It is the dialect's reason for not describing the bytes,
+such as `the record is not valid JSON` or `block type "x" is not one this dialect describes`. Its
+`state` is `available`, because every byte is kept, and `bytes` is their size. A reader that shows
+`text` as the message would show the reason in its place.
+
+Media is kept inline. A `media` part's `data` is the base64 text the runtime wrote, as one JSON
+string. `media` is its media type, and `bytes` is the length of the base64 text. There is no
+separate store for media. The measured corpus held 41 image blocks, so inline costs little. A
+runtime that sends images routinely should be measured again.
+
+A `result` can carry `text` and `data` together: the readable output, and a structured form of the
+same result that the runtime gave beside it. A reader that reads only one of them loses the other.
`failed` is absent, not false, when the runtime said nothing. Most results carry no error flag,
and reading that as success would report something the runtime never claimed.
The set of kinds is small because the thing described is small. Measured across a corpus of 3,032
-Claude Code files, six content shapes exist and four cover 99.99% of records.
+Claude Code files, six content-block shapes exist and four cover 99.99% of content blocks. The
+Claude Code [Session Data mapping](../adapters/claude-code.md#session-data-mapping) gives the
+counts.
## Dropped
@@ -117,3 +219,39 @@ Claude Code files, six content shapes exist and four cover 99.99% of records.
`dropped` exists so a loss is stated rather than silent. It lists only what the dialect understood
and chose not to carry. Everything it did not understand travels as an `unknown` part, so a later
version of the dialect can interpret it without collecting again.
+
+The source bytes are not kept, so a `.sd` file is the only landed copy of what it holds, and
+everything above the adapter treats it as the authority. A conversion error on content the dialect
+understood, a message, a thought, a call or a result, can be corrected only by collecting again
+from the runtime's source, and only while that source exists. Claude Code deletes its own
+transcripts, and a storage root outlives them (see [Storage Root](storage-root.md#size)). Only an
+`unknown` part can be read again from the store alone, because only it carries its original bytes.
+
+## Producing Session Data
+
+There is no Go interface to implement. An adapter is whatever lands Session Data in a storage root.
+Nothing above it reads a runtime field, so another runtime needs no code on the assembling side
+(see [What travels](storage-root.md#what-travels)).
+
+An adapter writes:
+
+- A header on every file, with `adapter`, its name and contract version, and `dialect`, beside the
+ other fields a reader requires. See [Header](#header).
+- Records with role-named identifiers and parts, one part for each piece of content, in source
+ order. See [Addressing a record](#addressing-a-record).
+- The closing line. The writer in `pkg/sessiondata` adds it on close.
+- Files in the layout and under the names [Storage Root](storage-root.md) shows. Each is written
+ once and made read-only, numbered one above the highest sequence already in the session, while
+ the session's lock, `<session-id>/.lock`, is held.
+
+The built-in adapters are wired into `cmd/asz` by name, so a third-party adapter is a separate
+program that writes the storage root. `asz parse` finds sessions by listing the root, and `asz view`
+serves the chains the root holds, so such a root is assembled and shown like any other. The index is
+not part of the contract. `asz parse` builds a session's index from its landed files when the
+session has none, and never reads past the index it has (see [The index](storage-root.md#the-index)).
+So a program that adds files to a session that already has an index removes its `index/` directory.
+That costs only a rebuild.
+
+A dialect also has a [glossary](../concepts-and-designs/glossary.md#adapters), keyed by the
+dialect: what its runtime calls each name the model uses. `cmd/asz` gives the page and
+`asz glossary` the Claude Code glossary, whatever dialect a conversation was read in.
diff --git a/docs/en/formats/session-flow.md b/docs/en/formats/session-flow.md
index e221654..6ae450e 100644
--- a/docs/en/formats/session-flow.md
+++ b/docs/en/formats/session-flow.md
@@ -23,8 +23,8 @@ one `commit`.
"session_from_time":"2026-08-06T16:20:45.531Z","session_through_time":"2026-08-12T02:36:31.749Z",
"title":"Check otel-rules support for meter mal","talks":357,"steps":16121,"streams":132,"segments":50,"unresolved":0}
{"t":"node","id":"ack/1/1338","revision":1,"kind":"agent.launch_ack","parent":"run/…","ref":{…}}
-{"t":"relation","id":"…","revision":1,"type":"starts","from":"…","to":"…","quality":"exact","evidence":[…]}
-{"t":"unresolved","id":"…","revision":1,"kind":"tool","ref":"toolu_…","reason":"…","state":"open"}
+{"t":"relation","id":"rel/starts/tool_toolu_01Ax…/stream_a308…","revision":1,"type":"starts","from":"tool/toolu_01Ax…","to":"stream/a308…","quality":"exact_unique","via":"child sidecar","evidence":[…]}
+{"t":"unresolved","id":"unres/tool_result/toolu_01Bq…","revision":1,"kind":"tool_result","ref":"toolu_01Bq…","reason":"no result landed for this tool use","state":"open"}
{"t":"commit","digest":"d829…","counts":{"nodes":1338,"relations":52,"unresolved":0}}
```
@@ -36,18 +36,45 @@ one `commit`.
| `round` | counts from 1 |
| `previous` | the digest of round N-1. Empty only for round 1. This is a dependency, not audit metadata: a round is built on the fold of the round it names. |
| `from_seq`, `through_seq` | the landed sequence range this round consumed |
-| `input_digest` | binds the round to the landed evidence it read. It is chained rather than recomputed, the previous input digest hashed with the digests of the newly landed files, so producing it stays proportional to new data. |
-| `parser`, `policy` | the interpretation versions. A change to either that alters meaning starts a new chain. |
+| `input_digest` | binds the round to the landed evidence it read. It is chained rather than recomputed, so producing it stays proportional to new data. How it is computed follows the table. |
+| `parser`, `policy` | the interpretation versions. `policy` includes the idle gap, the quiet period after which a segment can be closed. asz writes `v1+idle=10m0s`, and a round made under another gap has another policy. |
| `from_time`, `through_time` | the earliest and the latest record time among the landed files this round consumed, as the runtime wrote them, in UTC. Absent when no record in the window carries a time. |
| `session_from_time`, `session_through_time` | the session's own range as of this round: when it began, and its last activity so far. The `session` node carries the same pair as `from_time` and `through_time` in its attributes; the header repeats it so a reader of the header alone learns it without folding. |
| `title`, `talks`, `steps`, `streams`, `segments`, `unresolved` | what a list of conversations shows, as of this round: the session's title and the fold's counts of talks, steps, streams, segments and open unresolved references. A receiver lists conversations off the newest round's header, or off the attributes a sender copies from it, and never folds. |
+| `changes`, `lines_added`, `lines_removed` | the workspace change records the landed files carry up to `through_seq`. `changes` counts the distinct records: one for each tool call a producer watched for file changes, whether or not the call changed a file. The other two are the lines those records' diffs add and remove. |
+| `llm_calls`, `subagents`, `bash_runs` | the fold's `llm.call` nodes, its streams whose role is `child`, and its `tool` and `agent.call` nodes named `Bash`. A call that started a child agent becomes an `agent.call` and keeps its name, so `bash_runs` does not fall when the child's transcript lands. |
+
+Each of the six counts in the last two rows is absent from a round written before that count
+existed. A reader takes an absent count as unknown, never as zero. The list that
+[`asz view`](../setup/command-line.md#view) serves shows a dash for it.
+
+The input digest is a SHA-256 in lowercase hex:
+
+```text
+input_digest = hex( SHA-256( previous || 0x00 || d1 || 0x00 || d2 || ... ) )
+```
+
+- `previous` is the previous round's `input_digest` as hex text. It is empty for round 1.
+- `d1`, `d2` and the rest are the SHA-256 of each whole landed file with a sequence from `from_seq`
+ through `through_seq`, in lowercase hex. They are sorted as strings first, because directory
+ listing order is not stable.
+
+A chain holds one interpretation, so its parser and policy never change. The parser refuses to
+extend a chain built by another parser or policy, and [the fold](#the-fold) refuses to mix them. To
+move a conversation to a new parser or policy, remove `_conversations/<id>/` and parse again. The
+landed files rebuild it, as [Storage Root](storage-root.md#retention) says.
A header carries no wall-clock time. A round's bytes must be reproducible from its inputs, so the
-same landed range, the same previous digest and the same parser version yield the same digest.
-When a round was produced lives outside every digest, in `conversation.state`. Record times are
-different: the runtime wrote them into the landed files, so they are evidence, and the two time
+same landed range, the same previous digest, and the same parser and policy versions yield the same
+digest. When a round was produced lives outside every digest, in `conversation.state`. Record times
+are different: the runtime wrote them into the landed files, so they are evidence, and the time
fields above reproduce with the round.
+The order of the frames is part of the bytes. asz writes the header, then every node, then every
+relation, then every unresolved entry, each group sorted by id, and then the commit. Without the
+sort, two runs over the same input could order the frames differently and produce different
+digests.
+
### Entities
Nodes, relations and unresolved references share one envelope: an `id`, a `revision` and an
@@ -58,21 +85,72 @@ so an id cannot shift when an earlier source is backfilled later.
| --- | --- |
| `node` | `kind`, `parent` for containment, `stream`, `ref` or `refs` pointing at the landed records that back it, and `attrs` |
| `relation` | `type`, `from`, `to`, `quality`, `via`, and `evidence`. A typed edge that is not containment. Cross-stream flow is a relation, never containment: a child's work stays under the child. |
-| `unresolved` | `kind`, `ref`, `reason`, and `state`: `open`, `resolved` or `terminal` |
+| `unresolved` | `kind`, `ref`, `reason`, and `state`: `open`, `resolved` or `terminal`. [Qualification](../concepts-and-designs/conversation-assembly.md#qualification) lists the kinds asz writes and when it writes each. |
+
+An id joins its parts with `/`. An empty part is left out, and a `/` inside a part becomes `_`, so
+the run in talk `talk/main/8d53…` is `run/talk_main_8d53…/8d53…`. The parts are joined rather than
+hashed, so an id stays readable in a round file and in an error message.
+
+| Entity | Id |
+| --- | --- |
+| a node with an identity from the runtime | a prefix and that identity, such as `tool/<tool use id>` or `stream/<stream name>`. Where the identity is not unique across streams, the stream is a part as well, as in `talk/<stream>/<prompt cycle id>`. A chain holds one session, so no other id needs the session as a part. |
+| a node with no identity from the runtime | a prefix and the landed location of its evidence: `<prefix>/<seq>/<row>`, with `:<block>` added for one content block, such as `ack/1/1338`. A landed record never moves, so neither does the id. |
+| a relation | `rel/<type>/<from>/<to>`. It is keyed by what it joins, not by the evidence that showed it, so one edge observed twice is one relation. The assembler merges the evidence and keeps the stronger [quality](../concepts-and-designs/conversation-assembly.md#qualification). Once either observation says `conflict`, the relation stays `conflict`, because more agreement does not make a disagreement untrue. |
+| an unresolved reference | `unres/<kind>/<ref>`. It is keyed by what is missing, so the revision that resolves it has the same id. Keyed by the round that noticed it, every round would open a new one. |
+
+A node's prefix is short, and it is not always the node's `kind`. An `llm.call` is `call/<id>`. A
+tool call that started a child agent becomes an `agent.call` and keeps its `tool/<tool use id>`.
+These are the prefixes asz writes:
+
+| Kind | Id |
+| --- | --- |
+| `session` | `session/<session id>` |
+| `stream` | `stream/<stream name>` |
+| `epoch` | `epoch/<stream>/<id of the reset record>`. A stream's first epoch is `epoch/<stream>/0`. A reset record with no `id` is named by its location, `at_<seq>_<row>`. |
+| `segment` | `segment/at_<seq>_<row>`, the location of its first talk's first record |
+| `talk` | `talk/<stream>/<prompt cycle id>`, from its first run. A child stream is one talk, so its talk is `talk/<stream>`. |
+| `run` | `run/<talk id>/<prompt cycle id>` |
+| `llm.call` | `call/<provider call id>`, the `call` its fragments share |
+| `tool`, `agent.call` | `tool/<tool use id>` |
+| `agent.output` | `output/<stream>` |
+| `message.assistant`, `message.synthetic` | `msg/<seq>/<row>:<block>` |
+| `thinking` | `think/<seq>/<row>:<block>` |
+| `message.external` | `input/<seq>/<row>` |
+| `context.injection` | `inject/<seq>/<row>` |
+| `agent.launch_ack` | `ack/<seq>/<row>` |
+| `runtime.notification` | `notify/<seq>/<row>` |
+| `epoch.boundary` | `boundary/<seq>/<row>` |
+| `epoch.summary` | `summary/<seq>/<row>` |
+| `error.api` | `error/<seq>/<row>` |
+| `turn.duration` | `turn/<seq>/<row>` |
+| `control.command` | `command/<seq>/<row>` |
+| `control.notice` | `notice/<seq>/<row>` |
A reference into the landed data is a landed sequence, a row, and optionally a content block. That
-is the whole address; a reader turns it into a moment through the index and into content through
-the `.sd` record.
+is the whole address. A reader takes both the content and the time from the `.sd` record it names.
+The record's `time` field is when the runtime says it happened. The page that `asz view` serves
+never uses the index for either, as [Storage Root](storage-root.md#the-index) says, so a root
+without its index still shows its times. Apart from the `session` node's range, a node carries no time of its own.
+
+Content is referenced, never inlined. A provider call's input is everything before it in its
+context. Copying each call's input into its node would store every message again in each later
+call, so storage would grow with the depth of the conversation rather than with its content. With
+references only, the rounds of a 48-session root measured 105 MB, 8.9% of the source records, on
+2026-09-03, as [Storage Root](storage-root.md#size) shows.
### Commit
`digest` covers every preceding line of the file, so a round verifies itself, and it is the value
-the next round names as `previous`. `counts` lets a reader detect a truncated round without folding
-it.
+the next round names as `previous`. It is the SHA-256 of those lines, each ending in its newline, in
+lowercase hex. `counts` lets a reader detect a truncated round without folding it.
## The fold
-The conversation as of round N is the fold of rounds 1 through N, applied in order.
+The conversation as of round N is the fold of rounds 1 through N, applied in order. A fold takes a
+round only if it carries the next round number, names the digest of the round before as
+`previous`, and carries the chain's `conversation` and round 1's `session`, `parser` and `policy`.
+It refuses any other round, so a reader detects a mix of interpretations instead of folding through
+it.
- A frame with a higher `revision` supersedes the earlier one with the same id.
- Absence in a later round means unchanged, never deleted. Removal is an explicit tombstone, which
@@ -98,7 +176,41 @@ listing the directory and the previous input digest is taken from the last round
## Reading it
-`asz conversation ID` folds a chain and prints what it holds. `asz verify` recomputes every commit
-digest. The node kinds and relation types are the vocabulary of the
+`asz conversation ID` folds a chain and prints what it holds. The node kinds and relation types are
+the vocabulary of the
[Unified Conversation Model](../concepts-and-designs/unified-conversation-model.md), and the
[glossary](../concepts-and-designs/glossary.md) fixes each word.
+
+A reader rejects a round, rather than folding it, when any of these is true. A server that mirrors
+the format must reject the same rounds. The rules are strict because nothing looks at a round again
+once it is folded. A reader that accepts what a correct producer never writes can disagree with
+another reader about what the round said.
+
+- The first line is not a header, a second header appears, or a line follows the commit.
+- A line does not decode, or its frame type is unknown.
+- The header breaks one of its own rules. Its schema is not `sf/1`. It lacks `conversation`,
+ `session`, `parser`, `policy` or `input_digest`. Its `round` is 0. Its `previous` is missing
+ after round 1, or present on round 1. Its `from_seq` is 0, or its `through_seq` is below
+ `from_seq` minus one. A `through_seq` of exactly `from_seq` minus one is valid: that round
+ consumed no new evidence.
+- An entity has no id, or an id appears twice in one round.
+- A `revision` differs from the header's `round`.
+- A reference is sequence 0 row 0, or names a sequence past `through_seq`, which the round's input
+ digest does not cover.
+- A relation that is not a tombstone lacks `type`, `from` or `to`.
+- An unresolved entry that is not a tombstone has a state other than `open`, `resolved` or
+ `terminal`.
+- The commit is missing, so the round is truncated, or its `digest` or `counts` do not match what
+ was read.
+
+[`asz verify`](../setup/command-line.md#verify) checks six things for every round:
+
+1. No round is missing before it.
+2. Its `previous` is the digest of the round before.
+3. Its own digest matches its bytes, and it passes every rule above.
+4. Its `from_seq` is the previous round's `through_seq` plus one.
+5. Every landed sequence it consumed is still on disk.
+6. The input digest recomputed from those files equals its header's.
+
+Check 4 is the one no digest can make. The digests prove that the rounds were not changed. Only
+contiguous `from_seq` and `through_seq` prove that no landed evidence was skipped.
diff --git a/docs/en/formats/storage-root.md b/docs/en/formats/storage-root.md
index 8a9ffd8..95e6c7d 100644
--- a/docs/en/formats/storage-root.md
+++ b/docs/en/formats/storage-root.md
@@ -2,11 +2,13 @@
The storage root is a directory. Everything the project keeps is under it, in two halves: what was
collected, one directory per session, and what was assembled, one directory per conversation.
+Beside them sit the metrics spool and the export state, which belong to the root rather than to a
+session.
```text
<root>/
<session-id>/
- session.state next_seq, liveness, last scan
+ session.state next_seq, liveness (always unknown), last scan
.lock one collector per session
streams/
main/
@@ -26,26 +28,46 @@ collected, one directory per session, and what was assembled, one directory per
script.cursor · script-…sd
index/
entries.bin the derived index
- index.state
+ index.state how far the index reaches
_conversations/
<conversation-id>/
conversation.state the head pointer, outside every digest
- .lock
+ .lock one parser per chain
rounds/
r000001-<digest>.sf
r000002-<digest>.sf
+ _metrics/ the metrics spool, when an adapter produces metrics
+ spool.state the next number for a received request
+ metrics.state what the local derivation has counted
+ .lock
+ metrics-<received-at>-<seq>-otlp.pb one request the receiver adapter was sent
+ metrics-<session-id>-<seq>-local.pb the points derived from the session's landed file <seq>
+ push.state what asz push has sent, with each file's digest
+ _export/
+ .lock one pusher per root
```
Child streams are flat siblings of `main`, keyed by agent id. The storage path deliberately does
not mirror the source tree: a path must not encode a relationship the pipeline is supposed to
-derive.
+derive. [Export over OpenTelemetry](../setup/export-otlp.md) says how the metrics spool fills and
+what `push.state` records.
+
+`liveness` in `session.state` is always `unknown`. asz has no check for whether a session is
+still running.
## Landed files
A landed file is [Session Data](session-data.md): one header line and then one record per source
-record. It is written once, to a temporary name, synced, renamed into place and made read-only.
-It is never appended to and never rewritten. A source that keeps growing produces more files, each
-starting where the previous cursor stopped.
+record. It is written once to a temporary name, synced, made read-only, and only then renamed into
+place. So a file under its final name is read-only from the moment it appears. It is never appended
+to and never rewritten. A source that keeps growing produces more files, each starting where the
+previous cursor stopped.
+
+A landed file is named `<kind>-<collected-at>-<seq>.sd`. `<collected-at>` is the UTC time the
+session's pass started, with nanoseconds and no separators, such as `20260831T091204.481523000Z`.
+Names in this form sort in time order as plain text. They hold no `:`, which Windows does not allow
+in a file name. Every file that one pass lands for a session carries the same stamp. `<seq>` is
+zero-padded to at least six digits. A reader orders files by the sequence, never by the stamp.
The sequence number in the name is issued per session and is monotonic across every stream in it.
That is what lets assembly track progress with one watermark instead of one entry per stream, and
@@ -54,7 +76,11 @@ the counter: on start, a collector raises its counter above the highest sequence
so a crash between writing a file and saving state cannot reissue a number.
A file lands before its cursor is committed. A crash between the two lands the same records twice
-in the next pass, and assembly removes the duplicate. The reverse order would lose data.
+in the next pass. The reverse order would lose data. The cursor would already point past records
+that never landed, and no later pass would read them. The index keeps the first copy of each
+record id and marks later copies as duplicates, which assembly skips. A record with no id has
+nothing to compare, so both of its copies stay in the index. `asz verify` counts records whose
+source line had already landed as re-landed, and does not fail on them.
A file is cut once. The collector cuts at `max_delta_bytes`, 2 MiB by default, and a round then
addresses every record by file and line and binds itself to the file digests, so re-cutting a file
@@ -63,6 +89,122 @@ only. To bring an existing root under a new budget, `asz repack DEST` re-cuts ev
new root, keeping each record's bytes and order, carries the cursors over, and builds the chains
again on the new files, which describe the same conversation at new positions.
+## Cursors
+
+Each source has one cursor, beside the files it tracks, named `<kind>.cursor`. The dot matters.
+Landed files are named `<kind>-<collected-at>-<seq>.sd`, so a cursor named with a hyphen would share
+their prefix, and a scan for one kind's files would take the cursor for data.
+
+A cursor belongs to one source rather than to the session, because a session can hold hundreds of
+sources. Measured on 2026-09-11 on one machine's storage root of 60 Claude Code sessions, the
+largest session has 616 streams and 1,309 cursors, 614,788 bytes in all. One file per session would
+mean rewriting all of that to record that one source moved. A cursor is rewritten only when its own
+source changed. The other numbers in this section come from the same root.
+
+A cursor is plain text, one key and value per line, replaced atomically. There are two kinds.
+
+### Append cursors
+
+Sources that only grow are tracked by position: transcripts, workflow journals, and the plugin's
+change records.
+
+```text
+schema 1
+kind append
+source -Users-me-github-project/221ac729-….jsonl
+dev 16777232
+ino 287890584
+offset 62230163
+ord 17108
+last_uuid 573afdf5-…
+tail_sha256 98ebccc0…
+size 62230163
+mtime 1787283075
+last_seq 35
+updated_at 2026-09-06T15:02:49.496762Z
+state active
+```
+
+| Field | What it is for |
+| --- | --- |
+| `source` | the source path, relative to the adapter's root, with forward slashes |
+| `dev`, `ino` | the device and the inode the file had when it was read, or on Windows its volume serial number and file index. Only `ino` is compared, and it is what this page calls the file's identity. A change means the file was replaced, or is being reached another way. The bytes decide which, see below. |
+| `offset` | the bytes consumed, always at the end of a line, and where the next read starts. Compared with the file's size, it says whether anything is new. A size below it means the file was cut short. |
+| `ord` | the last source line consumed, so line numbers continue without counting again from byte 0 |
+| `last_uuid` | the id of the last record that had one. For diagnosis only: many lines carry no id, and nothing compares it. |
+| `tail_sha256` | the SHA-256 of the 1 MiB before `offset`. A mismatch means bytes already read were rewritten. |
+| `size`, `mtime` | the size and modification time last seen. Recorded, never compared. |
+| `last_seq` | the sequence of the last file this source landed |
+| `state` | `active`, `source_gone` or `conflict` |
+
+Every pass takes each append source through the same steps:
+
+1. The file is gone. The cursor becomes `source_gone`, and the landed files stay. This is normal:
+ Claude Code prunes its transcripts, and the landed files outlive them.
+2. The file is shorter than `offset`. It was cut short, and the source is a `conflict`.
+3. The size equals `offset` and the identity is unchanged. Nothing is new, and nothing more is
+ read. A change is judged by the size, never by `mtime`, so clock skew and timestamp granularity
+ cannot hide a change or invent one.
+4. Otherwise the 1 MiB before `offset` is digested. If it differs from `tail_sha256`, the source is
+ a `conflict`.
+5. The read starts at `offset`, takes at most
+ [`max_delta_bytes`](../setup/configuration.md#collector), and stops at the last complete
+ newline. An unterminated last line is a write in progress. It is left for a later pass, so a
+ landed record is always a whole source line. A single line longer than the budget is read to
+ its end, so the source cannot get stuck behind it.
+6. Each line is converted to a record. A line that is not valid JSON does not stop collection. It
+ lands as an `unknown` part that keeps its bytes.
+7. The file lands. Then the cursor is saved with the new `offset`, `ord` and `tail_sha256`.
+8. While complete lines remain, the pass reads the next window, up to 65 windows from one source.
+ A source that still has data after that counts as pending. A watching collector carries on at
+ its next pass, and a single pass of `asz collect` exits non-zero. The limit stops one source
+ that grows faster than it is read from keeping every other source waiting.
+
+A changed `ino` is not a conflict by itself. The same file reached through another file system
+has a new inode: a bind mount into a container, a restored backup, a copied source tree. So the
+bytes decide. If the 1 MiB before `offset` still matches, collection continues and the cursor
+takes the new `dev` and `ino`. If it does not match, the source is a `conflict`. A cursor with no
+tail digest cannot be checked this way, so for it a new `ino` is a conflict. The rule comes from a
+failure: when the inode alone decided, every source collected on a host was a conflict when the
+same root was collected inside a container.
+
+A change of `dev` alone is never noticed. `dev` is saved with the cursor and never read back.
+When nothing lands and `ino` is unchanged, the cursor is not saved, so the `dev` it holds can be
+out of date.
+
+The window is 1 MiB rather than the whole consumed prefix, because the whole prefix would be read
+again on every pass that finds new data. In the same root, the largest transcript has 62,230,163
+bytes consumed. The window catches a change to any byte inside it. A rewrite further back than
+1 MiB is not caught. A record can be longer than the window, so the window can cover only part of
+the last record: 22 of the 404,956 records from growing sources are over 1 MiB, the largest
+4,629,074 bytes.
+
+A conflict is kept. The cursor is saved with `state conflict`, every later pass skips the source,
+and nothing in asz clears it. Collection stops because an `offset` into bytes that changed no
+longer means what it meant.
+
+The pass that finds the conflict names the file and the reason on standard error. Every later pass
+reports only how many sources are in conflict, not which ones. A watching collector prints that
+count after each pass, and a single pass of `asz collect` exits non-zero. To find the sources, look
+for cursors whose `state` is `conflict`.
+
+All of this rests on transcripts being append-only, and that is observed, not proven. None of the
+6,080 cursors in the same root is in conflict.
+
+### Snapshot cursors
+
+Sources that are rewritten whole are tracked by a digest of their content: child-agent sidecars,
+workflow manifests and workflow scripts. A snapshot cursor carries `content_sha256` in place of the
+position fields. Every pass reads the whole file, trims trailing newlines and digests it. When the
+digest differs from `content_sha256`, the whole file lands as a new file of one record, and the
+cursor takes the new digest. An empty file lands nothing.
+
+So a change to such a source becomes a new immutable version, never an edit, and a run directory
+can hold several manifest files, one per version. This kind exists for the workflow manifest. It is
+rewritten as its run goes on, and its final version carries the run's `status`, `durationMs`,
+`totalTokens` and `result`. A sidecar uses the same cursor. In the same root, each of 2,782 child
+streams holds one sidecar version, and each of 153 runs holds one manifest version.
+
## The index
`index/` holds identifiers and roles, never text: which record is in which stream, which call a
@@ -71,6 +213,13 @@ this and never the payloads; the page does not read it at all, and takes every t
record itself. It is derived and disposable. Delete it and the next collect or parse
rebuilds it from the landed files; a schema change discards it rather than migrating it.
+`index/index.state` records `indexed_seq`, the landed sequence the index covers, with its schema
+and its counts of entries, blocks and strings. Cursors commit per source, but the index is written
+once per session, at the end of its pass. A crash between the two leaves the index behind what
+landed, and nothing would land again to fill the gap. So the next collect compares `indexed_seq`
+with the highest sequence on disk and indexes the landed files above it again. Parse never reads
+past `indexed_seq`, so a round cannot cite a record the index does not hold.
+
## The chain
`_conversations/<id>/rounds/` holds [Session Flow](session-flow.md): an append-only chain of
@@ -79,14 +228,27 @@ name carries the round number and the first twelve characters of the round's dig
`conversation.state` is a cache of the head. It is rebuilt by listing the directory, because a
crash between publishing a round and saving state must not lose the round.
+Publishing a round reads the head and then writes the next round. `.lock` lets one parser do that
+at a time. Without it, two parsers could both read round N and both write a round N+1. The digest
+is part of the file name, so the two files would not collide, and the chain would fork with no
+error. A second parser gets a lock error and writes nothing. Under the lock, a round that does not
+follow the head is refused, and a round file is created only when no file of that name exists, then
+made read-only. A test in `tests/chain` runs four parsers at once on one session and requires
+exactly one round.
+
## What travels
A storage root is complete on its own. A copy of it can be listed, verified, re-parsed and read on
-another machine with no source files and no collector: the landed files name the adapter that
-wrote them, so the index rebuilds from them alone, and the chain head is recovered from the
+another machine with no source files and no collector. Session Data is already converted: its
+records name identifiers by role, not by any runtime's fields. So the index rebuilds from the
+landed files with no adapter and no runtime-specific code, and the chain head is recovered from the
rounds directory. `asz verify` checks every landed file against its digest and every round against
its commit digest without touching the source.
+For the same reason, the index, assembly and the chain hold no code for any one runtime. Supporting
+another runtime needs an adapter that writes Session Data, and nothing on the assembling side. The
+mock dialect the tests use is one: it writes Session Data directly, and the same code assembles it.
+
## Size
Measured on 2026-09-03 on one machine's storage root of 48 Claude Code sessions:
@@ -109,5 +271,12 @@ The store had also outlived its source: it held 48 sessions where the source dir
The session directory is the unit of retention: removing it purges everything collected for that
session. Removing `index/` costs a rebuild. Removing a conversation's rounds loses the structure
only; a new parse rebuilds it from the landed files, and because a round carries no wall-clock
-time, the same landed range and the same parser version reproduce the same bytes and the same
-digest.
+time, the same landed range and the same parser and policy versions reproduce the same bytes and
+the same digest.
+
+Remove a session directory whole, never only some of its landed files. A round names records by
+landed sequence and row, so a round whose files are gone points at records nobody can read.
+`asz verify` reports each such round with each missing sequence, and exits non-zero. The
+[asz.view](asz-view.md) document is still served. Its `summary.state` is `incomplete`, and
+`summary.problems` has one line for each missing file in each round. A node that stood on a missing
+record has no text and no content state.
diff --git a/docs/en/guides/contributing.md b/docs/en/guides/contributing.md
index 799b964..d74fdef 100644
--- a/docs/en/guides/contributing.md
+++ b/docs/en/guides/contributing.md
@@ -54,8 +54,7 @@ rather than approximating. The same rule holds in the code. The model never inve
sessions, parentage or causal links; a reference that cannot be resolved is carried as data and
shown as such.
-`design-notes/` holds the working notes: measurements, corrections and open questions, unpolished
-by intent. `docs/` is the official documentation and is published on the SkyWalking website.
+`docs/` is the official documentation and is published on the SkyWalking website.
## Invariants
diff --git a/docs/en/setup/claude-code-plugin.md b/docs/en/setup/claude-code-plugin.md
index f2f71c6..f16c69c 100644
--- a/docs/en/setup/claude-code-plugin.md
+++ b/docs/en/setup/claude-code-plugin.md
@@ -62,15 +62,36 @@ ${CLAUDE_PLUGIN_DATA}/output/<session-id>/main.jsonl
${CLAUDE_PLUGIN_DATA}/output/<session-id>/<agent-id>.jsonl
```
+A record goes to the file of the stream its hook event came from. An event inside a subagent
+carries `agent_id`, and the file takes that name. No event on the main stream carries `agent_id`,
+so an event without one goes to `main.jsonl`. The same absence decides who records an edit: the
+plugin records one only when the event carries `agent_id`. See
+[What was verified](#what-was-verified).
+
Each line is a `changes/1` record: the tool-use id it belongs to as its id, `captured_by:
asz-plugin`, the session, the stream, the time, how it was observed, the root, the policy it ran
under, and for each changed file its path, the operation, size and hash before and after, and the
hunks as a unified diff with line numbers. `pkg/changes` in the repository defines the shape.
+Hook input carries neither a message id nor a request id. So a record names its tool call and
+never the provider call that made it, and the view joins it to its step by the tool-use id alone.
+See [Workspace changes](../adapters/claude-code.md#workspace-changes).
+
+Two hooks can run at the same time. So each record is written whole, in one write to a file
+opened for appending, and synced to disk before the hook returns. asz lands a file only up to its
+last complete newline, and a line still being written waits for the next pass. Appending with
+`cat >>` to one shared file from concurrent subagents was seen to corrupt lines above about 64 KB.
+The sample it was seen on is unavailable.
asz's `claude-code-changes` adapter, on by default, finds these files beside Claude Code's own,
tails them, and lands each line as a record of kind `changes` under the stream the tool ran in.
See [Configuration](configuration.md#the-changes-adapter).
+Each file it lands names the dialect `asz-changes/1` in its
+[header](../formats/session-data.md#header), not `claude-code/1`. The plugin writes its records in
+the model's own words, not in Claude Code's shape. Records of a different shape get a different
+dialect, even when the same runtime produced them, as the
+[adapter contract](../concepts-and-designs/unified-conversation-model.md#adapter-contract) says.
+
## Settings
Optional, in `${CLAUDE_PLUGIN_DATA}/settings.yaml`. Every value has a default, and the plugin runs
@@ -181,7 +202,8 @@ go version env list doc
`git add` is on the list because it changes only `.git/`, which is excluded scope. `git commit`
is not, because a pre-commit hook can rewrite files. A miss is not a loss: the next scan compares
against the last manifest, so a write that slipped through lands as an unattributed change. On
-the corpus above, 69.3% of shell commands are read-only under this list. The fixture
+the corpus above, 69.3% of shell commands are read-only under this list, so most shell commands
+wait for no scan. The fixture
`plugins/claude-code/internal/readonly/testdata/commands.txt` holds 93 real commands with the
outcome each must get, and the classifier's test reads it.
@@ -213,21 +235,44 @@ only one window spans is that window's alone; one changed in a step both span is
each record names the other under `overlaps`. Nothing claims who wrote a byte: a person or an
editor can write inside any window. The record says which tool windows could have.
-A scan stops at `scan_timeout` and the record says so, with `coverage: partial`. A hook that
-fails, for any reason, exits 0 and writes to `${CLAUDE_PLUGIN_DATA}/log/plugin.log`; it never
-stops the tool.
+Claude Code runs a hook synchronously, so the tool call waits until the hook returns. That is why
+a scan has a cap. A scan stops at `scan_timeout` and the record says so, with `coverage: partial`.
+The cap is for one scan, and a hook scans each root in turn. The default, 30 seconds, is half the
+60 seconds `hooks/hooks.json` gives each tool event. A hook still running at its timeout is killed
+and leaves no record for its call, so a `scan_timeout` near 60 seconds loses records instead of
+marking them partial. The scans of one root take turns under a lock, and a hook waiting for that
+lock is bounded only by its own timeout. The session's start and end run no scan, and get 30
+seconds.
+
+On 5 live headless sessions with a hook on every event, a hook invocation took about 5.6 ms.
+Those hooks ran no scan. How long a scan takes has not been measured.
+
+A hook that fails, for any reason, exits 0 and writes to `${CLAUDE_PLUGIN_DATA}/log/plugin.log`.
+It never stops the tool.
## What was verified
Each of these was read from a run of Claude Code 2.1.260 with a logging plugin, not from
-documentation: `hooks/hooks.json` with `${CLAUDE_PLUGIN_ROOT}` loads and fires; every tool event
-carries `session_id`, `tool_use_id`, `tool_input` and `cwd`; every event inside a subagent
-carries `agent_id`, and it is the id in the subagent transcript's file name; a failed shell
-command fires `PostToolUseFailure` with the exit code in `error`; the `Edit` response carries the
-patch and the original content, and so does a `Write` over an existing file, whose patch ends with
-the git marker line for a missing final newline; the `NotebookEdit` response names its file as
-`notebook_path` with the whole file before and after and no patch, so the plugin diffs the two;
-`CLAUDE_PLUGIN_DATA` exists before the first hook runs; a hook past its `timeout` is killed and the
-tool proceeds. The plugin itself was run inside Claude Code on macOS with a shell command, an
-edit and a subagent, and asz collected and showed the result. Windows is a build target; its hook
-command line has not been exercised.
+documentation:
+
+- `hooks/hooks.json` with `${CLAUDE_PLUGIN_ROOT}` loads and fires.
+- Every tool event carries `session_id`, `tool_use_id`, `tool_input` and `cwd`.
+- Every event inside a subagent carries `agent_id`. It is the id in the subagent transcript's file
+ name.
+- No event on the main stream carries `agent_id`.
+- A failed shell command fires `PostToolUseFailure`, with the exit code in `error`.
+- The `Edit` response carries the patch and the original content.
+- So does a `Write` over an existing file. Its patch ends with the git marker line for a missing
+ final newline.
+- The `NotebookEdit` response names its file as `notebook_path`. It carries the whole file before
+ and after and no patch, so the plugin computes the hunks from the two.
+- `CLAUDE_PLUGIN_DATA` exists before the first hook runs.
+- A hook past its `timeout` is killed, and the tool proceeds.
+
+A second sample, 5 live headless sessions with a hook on every event, agrees. `agent_id` is on
+every event inside a subagent and on no event of the main stream. No event in that sample carries
+a message id or a request id.
+
+The plugin itself was run inside Claude Code on macOS with a shell command, an edit and a
+subagent. asz collected and showed the result. Windows is a build target. Its hook command line
+has not been exercised.
diff --git a/docs/en/setup/command-line.md b/docs/en/setup/command-line.md
index daf4bff..e6c44c7 100644
--- a/docs/en/setup/command-line.md
+++ b/docs/en/setup/command-line.md
@@ -27,13 +27,21 @@ asz version print the version
| --- | --- |
| `-config FILE` | Configuration file. Default: `./asz.yaml` when present, else the compiled defaults. |
| `-once` | One pipeline pass rather than the collector's interval. Overrides `collector.mode`. `collect` then exits; `server` goes on serving what that pass produced, because a server that exits serves nothing. |
-| `-terms MODE` | Name things in the model's words (`unified`, the default), the runtime's (`native`), or `both`. |
+| `-terms MODE` | Name things in the model's words (`unified`, the default), the runtime's (`native`), or `both`. Every command accepts it, but only the plain output of `asz conversation` uses it. The asz.view document and the page have no such switch. |
## sources
-Discovers sessions under the source root and counts their files without landing anything. One
-row per session: directories, execution streams, child-agent sidecars, workflow journals and
-manifests. Sessions removed by the include and exclude filters are counted on the `filtered` line.
+Discovers sessions under the source root and counts their files without landing anything. The
+first line names the source root it resolved. Sessions removed by the include and exclude filters
+are counted on the `filtered` line. Then comes one row per session: directories, execution streams,
+child-agent sidecars, workflow journals and manifests. A line with the total of sessions and streams
+follows the table.
+
+After the total, a list names every session whose files span more than one source directory, with
+its number of directories. It shows that discovery groups files by session and not by directory, as
+[Discovery is session-first](../adapters/claude-code.md#discovery-is-session-first) explains. The
+list is absent when no session spans more than one.
+
With the `claude-code-changes` adapter enabled, a second table lists the sessions the plugin has
written change records for, with their streams and workspace.
@@ -65,12 +73,20 @@ One line per pass, and a quiet pass prints nothing:
| `landed` | source files that produced new data |
| `records` | source records landed |
| `rounds` | rounds written by the parse step |
+| `busy` | how many times another process on the same root held a session, a conversation's chain or the export state this pass needed, absent when 0 |
| `pushed` | files and rounds sent, absent when no endpoint is named |
| `metrics` | spooled metrics requests sent |
| `errors` | listed on standard error after the line |
-Anything the pass could not do is listed on standard error under the line, and the next pass tries
-it again: a source that was busy, a session that failed to parse, a request the receiver refused.
+`busy` is not an error. Running two pipelines on one root is supported, and the process that holds
+a lock is doing the same work. A session whose chain was held is parsed on a later pass. A single
+pass has no later pass, so there that session is listed as an error. So is an export state that
+another process still holds after two minutes of waiting, because then nothing was sent.
+
+Anything else the pass could not do is listed on standard error under the line, one `error:` line
+each: a session that failed to parse, a request the receiver refused. A watching collector goes on
+to the next pass. A single pass exits non-zero, so an unattended backfill can be read by its exit
+status.
## index
@@ -94,10 +110,25 @@ SESSION ROUND SEQ NODES RELS UNRES TALKS RU
0438c73b-2367-4ed5-9de3-13ef9a17ed01 1 305 17249 703 0 357 445 11473 5620/5620 131/131
```
-`ROUND` is the round written, or `-` when nothing changed. `SEQ` is the landed sequence the
-conversation now covers. `UNRES` counts references that could not be resolved. `TOOLS` and
-`CHILDREN` are joined out of total. The command exits non-zero when any session failed to parse,
-so a loop or a CI step can read the status instead of the log.
+`ROUND` is the round written, or `-` when nothing changed. A round is cut at the byte budget, so
+one pass may write several, and the row then shows the last. `SEQ` is the landed sequence the
+conversation now covers.
+
+`NODES`, `RELS` and `UNRES` count one round: the frames that round wrote. A round writes only what
+is new or changed, so after the first round these numbers are small, and they are 0 when no round
+is written. `NODES` and `RELS` include tombstones, the entities the round removes. `UNRES` includes
+the entries the round marks `resolved`, so it is not the number still open.
+
+`TALKS`, `RUNS` and `STEPS` count the whole conversation through `SEQ`. `TOOLS` and `CHILDREN` also
+count the whole conversation, and carry the denominator that
+[Qualification](../concepts-and-designs/conversation-assembly.md#qualification) asks for: how many
+tool calls and child-agent launches were joined, out of how many exist. The number of unresolved
+references still open has no denominator. It is the round header's `unresolved`, the
+`summary.unresolved` of the [asz.view](../formats/asz-view.md) document, and the `still open` count
+of `asz conversation`.
+
+The command exits non-zero when any session failed to parse, so a loop or a CI step can read the
+status instead of the log.
## repack
@@ -155,8 +186,7 @@ backfill does not look like a hung command.
Serves the conversations as a page on `ADDR`, `127.0.0.1:8787` by default, and **only reads**:
nothing in this process collects, parses or sends. It is the web host for a root that asz collect
filled, or one copied from another machine, or one a receiver wrote. The list is at `/` and
-one conversation at `/c/{id}`. The page reads the folded chain on demand and caches nothing beyond
-the process.
+one conversation at `/c/{id}`.
A conversation is drawn by Horizon's conversation renderer, the same one the SkyWalking UI uses
for the conversations the OAP holds, so the two draw a conversation identically. asz embeds the
@@ -169,10 +199,72 @@ files.
A root with no conversations in it is refused rather than served empty: run `asz collect` first,
or `asz server` to collect and serve together.
-The list shows what the head round's header counted — talks, model calls, subagents, Bash runs,
-changes with the lines they added and removed, and open unresolved references — so it never folds a
-conversation to draw a row. A count the head round does not carry shows a dash rather than a zero:
-a round cut before that count existed does not know the answer.
+The list folds every conversation to draw its rows. Talks, steps, open unresolved references and
+the time span are read from the fold. Model calls, subagents, Bash runs, and changes with the lines
+they added and removed are what the head round's header counted. A header count the head round does
+not carry shows a dash rather than a zero: a round cut before that count existed does not know the
+answer. A conversation whose chain has no round that can be folded is left out of the list.
+
+The page reads everything it shows from a JSON API on the same address:
+
+| Path | Returns |
+| --- | --- |
+| `/api/status` | the mode and the last refresh. See [server](#server) and below. |
+| `/api/conversations` | one row per conversation, for the list |
+| `/api/c/{id}/view` | the whole [asz.view](../formats/asz-view.md) document, which the renderer draws alone |
+| `/api/c/{id}/record/{seq}/{row}` | one landed record, whole. This is what the Evidence tab shows. |
+| `/api/glossary` | what the runtime calls each name the model uses, as [glossary](#glossary) prints it |
+
+There is no endpoint per talk. Beyond the document, a conversation page asks the API for two
+things: `/api/glossary`, once as it loads, and a landed record, only when the reader opens one in
+the Evidence tab.
+
+A row of `/api/conversations` carries `id`, `title`, `talks`, `steps`, `streams`, `segments`,
+`rounds`, `from`, `to` and `unresolved`, the references still open, then `changes`, `lines_added`,
+`lines_removed`, `llm_calls`, `subagents` and `bash_runs`. A null in the last six means the head
+round was written before that count existed. `from` and `to` are unix milliseconds: when the
+session began and its last activity.
+
+`/api/c/{id}/record/{seq}/{row}` returns the [Session Data record](../formats/session-data.md#record)
+as it was landed, with its `dropped` list. `seq` names the landed file and `row` the record in it,
+counting from 1. The record is read from disk on every request and never cached, because it is
+wanted only when someone opens it. The answer is 404 when the conversation, the sequence or the row
+does not exist.
+
+`/api/glossary` returns three keys. `dialect` says whose vocabulary the terms are in. `terms` is
+keyed by the model's name, and each entry carries the runtime's `native` name, `where` the runtime
+records it, and a `note`. `fields` describes the fields Session Data defines itself, such as `ord`,
+`parts` and `usage`, in one line each. A field that names something the model names, such as
+`stream`, `call` or `run`, is described under `terms` instead. The keys of a landed file's last
+line, `t`, `records` and `digest`, are in neither.
+
+The page uses the glossary to explain the landed record the Evidence tab shows. Every key found in
+`terms` or `fields` carries a `?`, which opens what the key means. For a name the model uses, that
+is the runtime's word or the fact that the runtime has none, where to look in the runtime's records,
+the note, and the dialect. For a field Session Data defines, it is the field's line from `fields`.
+The page has no switch to the runtime's words.
+
+The page folds a conversation when it is first asked for, and keeps the fold in memory until the
+head round on disk moves. The asz.view document is built once per fold. Nothing is written to disk.
+A read cache on disk would cost more than it saves. Measured on the largest session of a corpus of
+62 real sessions, 53,106 nodes and 922 talks, folding the whole chain took 302 milliseconds,
+building the talk list 1 millisecond, and walking one talk's subtree 13 microseconds at the 99th
+percentile.
+[asz.view](../formats/asz-view.md#reading-it) gives the size and build time of a whole document.
+
+Another process can write the root while the page reads it: an `asz collect` running beside it, or
+a receiver. Nothing needs a restart. Every request for a conversation lists its rounds directory,
+and folds again when the head round has moved. A landed file that arrives after the fold is found
+by a new scan of the session directory. That scan runs at most once a second, so a request for a
+sequence that does not exist does not walk the directory every time. The root itself is checked on
+the collector interval, 5 seconds by default. `/api/status` reports as `last_refresh` the last time
+a head round or the size of its file changed, and the list page reloads when that time moves.
+
+A round file is created under its final name and written in place, so a listing can show a round
+before all its bytes are there. The fold then stops before that round. For that one read, the
+document names the round in `summary.problems`, and its `summary.state` is `mismatch`, the same
+state a failed digest gives. The next read folds it. The cost of checking the head on every read,
+and of folding again while a conversation is still being written, has not been measured.
## scenario
@@ -222,8 +314,8 @@ failed are not recorded as sent and go again on the next pass. See
Prints every name the model uses, what the runtime calls it, where in the runtime's records it is
found, and a note. A `—` in the runtime column means the model derives the concept and the runtime
-has no word for it. Every command that prints names accepts `-terms native` to use the runtime's
-words instead.
+has no word for it. With `-terms native`, the plain output of `asz conversation` uses the
+runtime's words instead.
## verify
@@ -239,6 +331,21 @@ checked 44 conversation chain(s), 44 round(s)
all landed data is contiguous and matches its digests
```
+Contiguous means that, in every stream, the landed records continue the source's line numbers and
+byte offsets with no gap. Each record carries the line it came from, `ord`, the byte it started at,
+`off`, and its size, `bytes`. The next new record must be on the next line and must start at
+`off + bytes + 1`, the one being the newline the source had. That proves every source byte up to
+the cursor is accounted for, without the source. A skipped line is listed as an `ord gap`, and
+bytes no record accounts for as a `byte gap`, each with its file and row. The check then goes on.
+
+A landed file's last line carries a digest of every line before it, so reading the file catches
+one that was edited or cut short. Such a file ends the check with an error that names it.
+
+A record that repeats a range already landed is counted, not reported as a problem. verify prints
+`N record(s) landed more than once by an interrupted pass; assembly removes them`. A file lands
+before its cursor is committed, so an interrupted pass repeats data rather than losing it.
+[Storage Root](../formats/storage-root.md#landed-files) explains the order.
+
## version
Prints the version the binary was built with, the Go version, and the platform. A release build
diff --git a/docs/en/setup/configuration.md b/docs/en/setup/configuration.md
index bc4db7d..3ddbd2b 100644
--- a/docs/en/setup/configuration.md
+++ b/docs/en/setup/configuration.md
@@ -34,9 +34,9 @@ transcripts.
## adapters
A list. `claude-code-local` reads Claude Code's files from this machine. `claude-code-changes`
-reads the change records the asz Claude Code plugin writes beside them; see
+reads the change records the asz Claude Code plugin writes beside them. See
[the changes adapter](#the-changes-adapter). `claude-code-otlp` receives the runtime's own
-exporter; see [the receiver adapter](#the-receiver-adapter). Every command runs once per enabled
+exporter. See [the receiver adapter](#the-receiver-adapter). Every command runs once per enabled
adapter.
| Key | Default | Meaning |
@@ -48,7 +48,7 @@ adapter.
| `exclude` | `/private/tmp/**` | Session filters, see below. |
| `metrics` | `false` | Derive the runtime's own metric family from the landed files, `claude_code.token.usage` in phase one, name for name with the runtime's exporter. See [Metrics](export-otlp.md#metrics). |
| `listen` | none | On `claude-code-otlp` only: the address the runtime's exporter is pointed at, such as `127.0.0.1:4317`, serving gRPC and HTTP with protobuf on the one port. |
-| `metrics_lookback` | `24h` | How far back the first derivation over a root reaches. A duration such as `24h`, or a number of days such as `7d`; `0` or `none` derives everything. Later passes derive every new file whole. |
+| `metrics_lookback` | `24h` | How far back the first derivation over a root reaches. A duration such as `24h`, or a number of days such as `7d`. `0` or `none` derives everything. Later passes derive every new file whole. |
### Session filters
@@ -70,9 +70,30 @@ are the tool's, not yours, which is why they are excluded by default.
| Key | Default | Meaning |
| --- | --- | --- |
-| `mode` | `watch` | `watch` polls the source continuously. `once` makes a single pass and exits, which is the backfill path over history that already exists. `-once` on the command line overrides the file. |
+| `mode` | `watch` | `watch` polls the source continuously. `once` makes a single pass, which is the backfill path over history that already exists. `asz collect` then exits, and `asz server` goes on serving what that pass produced. `-once` on the command line overrides the file. |
| `interval` | `5s` | How long the pipeline sleeps between passes in watch mode. It is the whole period: `asz collect` and `asz server` land, parse and send once per interval. |
-| `max_delta_bytes` | `2097152` | The largest `.sd` file the collector writes, 2 MiB. A large catch-up is split into several files, and a single record larger than this is landed whole. A file travels whole as one log record, so this is also the largest record a receiver has to accept. A change applies to new files only; `asz repack` brings an existing root under a new budget. |
+| `max_delta_bytes` | `2097152` | How much of a growing source the collector lands in one `.sd` file, 2 MiB. A large catch-up is split into several files. When the next source line is longer than the budget, the collector reads to the end of the source instead. That file then holds the long line and every complete line after it. A file travels whole as one log record, so the largest record a receiver has to accept is the largest file, not the budget. A change applies to new files only. ` [...]
+
+### Several local adapters
+
+`collector` is set on each adapter, but `asz collect` and `asz server` run one pipeline for all
+the enabled local adapters, `claude-code-local` and `claude-code-changes`. One pass reads them
+all. See [collect](command-line.md#collect). One pipeline has one mode and one period, so the
+settings combine:
+
+- It watches when any of them has `mode: watch`. It makes a single pass only when every one of
+ them has `mode: once`, or with `-once`.
+- It runs at the shortest `interval` among them. A source that asks for 2 seconds is still read
+ every 2 seconds when another asks for a minute.
+- `max_delta_bytes` does not combine. Each adapter cuts its own files at its own budget.
+
+A disabled adapter does not count. The receiver, `claude-code-otlp`, takes no `collector` block,
+so it does not count either. When it is the only adapter enabled, `asz collect` sends what it
+lands to `export.otlp.endpoint` every 5 seconds, the compiled default.
+
+Both commands print the result when they start, on their `source` line: `(every 5s)` when the
+pipeline watches, `(once)` when it does not. `asz view` collects nothing, but it checks the root
+for new rounds at the same shortest interval.
## parse
@@ -137,10 +158,10 @@ adapters:
`claude-code-changes` lands the records the [asz Claude Code plugin](claude-code-plugin.md)
writes: which files each shell command changed, and each edit made inside a subagent, as
git-style hunks. Empty `source_root` resolves `plugins/data` under the same directory
-`claude-code-local` resolves, and reads every plugin directory named `asz-changes-*` under it;
-set it to collect from a copy. The session filters are the ones above, judged by the workspace
-each session's records name. It is on by default because it costs nothing when the plugin is not
-installed: there is nothing to discover. It takes no `metrics`.
+`claude-code-local` resolves, and reads every plugin directory named `asz-changes-*` under it.
+Set `source_root` to collect from a copy. The session filters are the ones above, judged by the
+workspace each session's records name. It is on by default because it costs nothing when the
+plugin is not installed: there is nothing to discover. It takes no `metrics`.
## The receiver adapter
@@ -155,7 +176,7 @@ adapters:
`claude-code-otlp` receives what Claude Code's own OpenTelemetry exporter sends, with the
runtime configured as its documentation says: `CLAUDE_CODE_ENABLE_TELEMETRY=1`,
`OTEL_METRICS_EXPORTER=otlp`, `OTEL_EXPORTER_OTLP_ENDPOINT` at `listen`, over gRPC or
-`http/protobuf`. Phase one lands its metrics in the storage root's spool for `asz push`; logs and
+`http/protobuf`. Phase one lands its metrics in the storage root's spool for `asz push`. Logs and
traces are accepted and dropped. It runs while `asz collect` or `asz server` runs, beside the
local adapter, and not with `-once`. `metrics` may be on here or on `claude-code-local`, never
on both: the configuration refuses to load, since the two would count the same tokens twice.
diff --git a/docs/en/setup/export-otlp.md b/docs/en/setup/export-otlp.md
index f3eeb3b..8e4fa15 100644
--- a/docs/en/setup/export-otlp.md
+++ b/docs/en/setup/export-otlp.md
@@ -155,14 +155,26 @@ account for. `go run ./tools/otlpdump -redact FILE.pb` prints any spooled reques
With `metrics: true` on the `claude-code-local` adapter, the collector derives the points from
the landed files by the assembler's own rule: the usage of a call is its last fragment's in line
order, never a sum, and only a call that finished counts. A main transcript repeats the final usage
-on every fragment; a child's carries streaming partials on all but the last, and a call with no
-terminal fragment, 7% of a real corpus's child calls, has no usage at all. A call cut at a landed
-file boundary is read on into the next file of its stream, and a file ending mid-call waits for
-that file for a short grace before it is derived with what it has, under a watching collector; a
-single pass, the backfill over history that already exists, derives with what is there and never
-waits. A call is counted once however
-many files its records reach, and a record the runtime re-emitted before a context reset is the
-same call again.
+on every fragment. A child's carries streaming partials on all but the last. A call has finished
+when one of its fragments reports a stop reason. A call that never does is left out, because its
+usage block is a streaming stub, not a count. That is about 10% of all calls in a corpus of 2,970
+files and 365,825 records written by Claude Code 2.1.220 to 2.1.251. It is 14.5% of the calls on
+child streams and 0.02% of those on main transcripts, as
+[Provider calls](../adapters/claude-code.md#provider-calls) reports.
+
+A call cut at a landed file boundary is read on into the next file of its stream only when the
+file's last record is a fragment of that call. The reading stops at the first record of the next
+file that is not a fragment of the call. A tool result carries no call id, and in the same corpus a
+call's own tool results sit between its fragments on 24% of multi-fragment calls. So a call cut
+next to one of its own tool results is read short. On a child stream only the last fragment says
+the call finished, so such a call is left out of the metric. On a main transcript every fragment
+repeats the final usage, so the call still counts in full, in the minute of the last fragment read.
+
+Under a watching collector, a file whose last record is a fragment of a call not yet counted waits
+a short grace for the next file of its stream, then is derived with what it has. A file that ends
+with a tool result does not wait. A single pass, the backfill over history that already exists,
+derives with what is there and never waits. A call is never counted twice, however many files its
+records reach, and a record the runtime re-emitted before a context reset is the same call again.
Points are summed per minute and attribute set, and the windows of one series never overlap: a
point takes its minute unless the series already has a point at or past it, as when two children
@@ -176,8 +188,9 @@ again to the same bytes and nothing is counted twice.
The other source of the same family is the runtime's exporter itself: the `claude-code-otlp`
adapter receives what Claude Code sends and lands each metrics request in the same spool, bytes as
-received. One root sends one source: `metrics` may be on for the local adapter or for the
-receiver, and the configuration refuses both.
+received. The receiver does not check whether it has landed a request before. A request the
+exporter sends twice lands twice, and `asz push` sends both copies. One root sends one source:
+`metrics` may be on for the local adapter or for the receiver, and the configuration refuses both.
The points wait in the storage root's `_metrics/` spool, one write-once file per landed file
with points or per request received, and go out in order under the same budget and the same
diff --git a/internal/adapters/claudecode/glossary.go b/internal/adapters/claudecode/glossary.go
index 18a58d4..dfb6f8f 100644
--- a/internal/adapters/claudecode/glossary.go
+++ b/internal/adapters/claudecode/glossary.go
@@ -28,10 +28,16 @@ const Dialect = "claude-code/1"
// glossary is what Claude Code calls the things the model names.
//
-// It lives beside the extraction code on purpose. Every entry with a Native
-// value corresponds to a field the converter reads, so a rename in the
-// runtime changes both together, and a test checks that the model's whole
-// vocabulary is accounted for here.
+// It lives in the same package as the extraction code on purpose, so a rename
+// in the runtime is made to both in one change. A Native value says where the
+// runtime records something, and the adapter reads it there unless the Note
+// says it is not read. Such an entry keeps its Native value anyway, because an
+// empty one would say the runtime has no word for the thing. A test checks
+// that the model's whole vocabulary is accounted for here.
+//
+// Each name has one entry. NewGlossary keeps only the last entry for a name,
+// so a second one silently hides the first. Run, stream and tool are each both
+// a role and a node kind, and their entries are with the kinds.
//
// An empty Native is not a gap. A Talk, a Segment, a correlation quality - the
// runtime records none of them, and saying so is more useful than a plausible
@@ -42,31 +48,28 @@ var glossary = model.NewGlossary(Dialect,
model.Term{Unified: model.RoleParent, Native: "parentUuid", Where: "record"},
model.Term{Unified: model.RoleCall, Native: "message.id", Where: "record",
Note: "not requestId, which a client-fabricated record can reuse"},
- model.Term{Unified: model.RoleRun, Native: "promptId", Where: "record",
- Note: "on user records only; a model response reaches one through its parents"},
model.Term{Unified: model.RoleBatch, Native: "toolUseResult.runId", Where: "a workflow launch result",
Note: "also the name of the directory the batch's records are filed under"},
- model.Term{Unified: model.RoleStream, Native: "agentId", Where: "a child record, and its filename",
- Note: "main has no name of its own; it is the session's own transcript"},
model.Term{Unified: model.RoleContinues, Native: "logicalParentUuid", Where: "a compact_boundary record",
Note: "the only link back; a boundary's own parentUuid is null"},
- model.Term{Unified: model.RoleTool, Native: "tool_use_id", Where: "a tool_result block, a notification, a sidecar"},
model.Term{Unified: model.RoleChild, Native: "toolUseResult.agentId", Where: "a launch result, a journal record"},
model.Term{Unified: model.RoleTime, Native: "timestamp", Where: "record"},
model.Term{Unified: model.RoleModel, Native: "message.model", Where: "an assistant record",
Note: "<synthetic> on a record the client fabricated"},
model.Term{Unified: model.RoleTrigger, Native: "origin.kind", Where: "record, and attachment.origin.kind"},
- // ---- structure the model derived; the runtime records none of it ----
+ // ---- structure; the runtime names a session, a stream and a run, and the model derives the rest ----
model.Term{Unified: model.KindConversation, Note: "supplied, never inferred; defaults to the session id"},
model.Term{Unified: model.KindSegment, Note: "an activity window chosen for commit"},
model.Term{Unified: model.KindSession, Native: "sessionId", Where: "record, and the transcript filename"},
- model.Term{Unified: model.KindStream, Native: "agentId", Where: "a child record",
+ // The file name, not the record: the adapter reads the in-record agentId
+ // only on a sidecar or a journal record.
+ model.Term{Unified: model.KindStream, Native: "agentId", Where: "a child transcript's file name, agent-<id>.jsonl",
Note: "the parent lineage is the session transcript itself"},
model.Term{Unified: model.KindEpoch, Note: "the span between two compact_boundary records"},
model.Term{Unified: model.KindTalk, Note: "one readable interaction; derived from where a run was triggered"},
model.Term{Unified: model.KindRun, Native: "promptId", Where: "record",
- Note: "one loop per distinct promptId"},
+ Note: "one loop per distinct promptId in a stream. A model response carries none and reaches one through its parents"},
// ---- steps ----
model.Term{Unified: model.KindMessageExternal, Native: "origin.kind == \"human\"", Where: "a user record",
@@ -88,13 +91,17 @@ var glossary = model.NewGlossary(Dialect,
model.Term{Unified: model.KindEpochBoundary, Native: "system/compact_boundary", Where: "a system record"},
model.Term{Unified: model.KindEpochSummary, Native: "isCompactSummary", Where: "a user record"},
model.Term{Unified: model.KindErrorAPI, Native: "system/api_error", Where: "a system record"},
- model.Term{Unified: model.KindControlInterrupt, Native: "toolUseResult.interrupted", Where: "a tool result"},
- model.Term{Unified: model.KindControlPermission, Native: "permission-mode", Where: "a record type"},
+ // The runtime has a word for these two, so the Native value stays, but
+ // nothing reads it yet.
+ model.Term{Unified: model.KindControlInterrupt, Native: "toolUseResult.interrupted", Where: "a tool result",
+ Note: "not read, so asz emits no step of this kind today"},
+ model.Term{Unified: model.KindControlPermission, Native: "permission-mode", Where: "a record type",
+ Note: "not read, so asz emits no step of this kind today"},
model.Term{Unified: model.KindControlCommand, Native: "system/local_command", Where: "a system record"},
model.Term{Unified: model.KindTurnDuration, Native: "system/turn_duration", Where: "a system record",
Note: "the runtime measured it; it is not the gap between two record timestamps"},
model.Term{Unified: model.KindControlNotice, Native: "system/away_summary, bridge_status, informational, model_refusal_fallback",
- Where: "a system record", Note: "anything the runtime says about the session itself"},
+ Where: "a system record", Note: "anything the runtime says about the session itself, including every system subtype no other row names"},
// ---- relations ----
model.Term{Unified: model.RelStarts, Native: "toolUseResult.agentId, or a run journal", Where: "a launch result"},
@@ -106,7 +113,8 @@ var glossary = model.NewGlossary(Dialect,
model.Term{Unified: model.RelSummarizes, Native: "parentUuid", Where: "an isCompactSummary record"},
model.Term{Unified: model.RelInSegment, Note: "an activity window chosen for commit"},
model.Term{Unified: model.RelRetries, Note: "unavailable; retry identity is not recorded"},
- model.Term{Unified: model.RelCancels, Native: "toolUseResult.interrupted", Where: "a tool result"},
+ model.Term{Unified: model.RelCancels, Native: "toolUseResult.interrupted", Where: "a tool result",
+ Note: "not read, so asz writes no cancels relation today"},
model.Term{Unified: model.RelInputOf, Note: "unavailable; the serialized provider request is not written locally"},
// ---- qualification; all of it is this project's judgement ----