You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2026/09/04 13:33:19 UTC
(skywalking-ai-sessionizer) branch main updated: Hold the document to the whole session across rounds, and check it in three stages
This is an automated email from the ASF dual-hosted git repository.
wu-sheng pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/skywalking-ai-sessionizer.git
The following commit(s) were added to refs/heads/main by this push:
new 8ccb5c5 Hold the document to the whole session across rounds, and check it in three stages
8ccb5c5 is described below
commit 8ccb5c52a70283ba5df0ecdd38e1903e9ea5a21a
Author: Wu Sheng <wu...@foxmail.com>
AuthorDate: Fri Sep 4 21:33:12 2026 +0800
Hold the document to the whole session across rounds, and check it in three stages
A scenario with two checkpoints already landed and parsed in three
stages and wrote three rounds over landed files cut at each stage. Now a
scenario says so, three-rounds, with expectations at every stage, and a
property of every scenario, view_covers_the_session, holds the final
asz.view document to the whole session: every round, verified; every
landed file with its digest as on disk; every talk, run and step of the
fold in a tree; the session's own range; and a verified state.
For that to hold in general the document gained loose: the runs and
steps no talk contains, as trees from their highest such ancestor, such
as a child's output the fold parents to the session when the child's
stream opened no talk. It is empty for every scenario and for most
conversations, and it exists so the document holds every step the fold
holds. The example on the format page is regenerated.
---
docs/en/changes/changes-0.2.0.md | 5 +-
docs/en/formats/asz-view-example.yaml | 39 +++++++++++
docs/en/formats/asz-view.md | 9 ++-
docs/en/guides/scenario.md | 9 +++
internal/scenario/expect/expect.go | 114 +++++++++++++++++++++++++++++++
internal/scenario/run/run.go | 1 +
internal/view/conversationview.go | 42 +++++++++++-
pkg/sessionview/sessionview.go | 16 +++--
tests/scenarios/three-rounds.expect.yaml | 38 +++++++++++
tests/scenarios/three-rounds.yaml | 37 ++++++++++
10 files changed, 301 insertions(+), 9 deletions(-)
diff --git a/docs/en/changes/changes-0.2.0.md b/docs/en/changes/changes-0.2.0.md
index e657878..fbb5eca 100644
--- a/docs/en/changes/changes-0.2.0.md
+++ b/docs/en/changes/changes-0.2.0.md
@@ -68,7 +68,10 @@
`pkg/sessionview` defines and owns the shape and `asz view` serves it at `/api/c/{id}/view`;
`asz conversation -json` or `-yaml` prints it. It is never a file. The format page says how each
view of a conversation is drawn from the document, and carries a complete example generated from
- the fixture scenario, which a test keeps current. A server holding the same files, such as the SkyWalking OAP, builds the same
+ the fixture scenario, which a test keeps current. The document holds every run and step of the
+ fold: what no talk contains sits under `loose`, and a property of every scenario holds the
+ document to the whole session, every round and every landed file included, however many rounds
+ the session was parsed in. A server holding the same files, such as the SkyWalking OAP, builds the same
document. See the format page.
- The page reads Session Data and Session Flow and nothing else. It took every record's time from
diff --git a/docs/en/formats/asz-view-example.yaml b/docs/en/formats/asz-view-example.yaml
index 480b84c..05c92a1 100644
--- a/docs/en/formats/asz-view-example.yaml
+++ b/docs/en/formats/asz-view-example.yaml
@@ -657,6 +657,45 @@ talks:
dir: out
quality: strong_inference
via: inside the window of the talk that delegated it
+loose:
+ - id: boundary/1/15
+ kind: epoch.boundary
+ parent: epoch/main/s8-boundary
+ stream: main
+ at: 1767225611100
+ ref:
+ seq: 1
+ row: 15
+ text: '{"compactMetadata":{"preservedMessages":{"allUuids":["s7-synthetic"]},"trigger":"auto"},"logicalParentUuid":"s7-synthetic","subtype":"compact_boundary","type":"system"}'
+ state: available
+ bytes: 168
+ flags:
+ - context_reset
+ edges:
+ - type: summarizes
+ other: summary/1/16
+ dir: in
+ quality: exact_unique
+ via: containment parent
+ - id: summary/1/16
+ kind: epoch.summary
+ parent: epoch/main/s8-boundary
+ stream: main
+ at: 1767225610700
+ ref:
+ seq: 1
+ row: 16
+ text: 'Summary: the build was run and checked.'
+ state: available
+ bytes: 39
+ flags:
+ - reset_summary
+ edges:
+ - type: summarizes
+ other: boundary/1/15
+ dir: out
+ quality: exact_unique
+ via: containment parent
relations:
- id: rel/ends_with/stream_a0a10ef0666c4dc7e/output_a0a10ef0666c4dc7e
type: ends_with
diff --git a/docs/en/formats/asz-view.md b/docs/en/formats/asz-view.md
index 512874b..2788aa6 100644
--- a/docs/en/formats/asz-view.md
+++ b/docs/en/formats/asz-view.md
@@ -76,6 +76,7 @@ fails when the committed file no longer matches what the code produces.
| `streams` | one per execution stream: `id`, `name`, `role` (`main` or `child`), `label`, `parent`, `records`, `steps`, `talk`, `named_by`, and `opened_by`, every step the assembler could tie to the start of the stream as `{step, stream, talk, quality}`; several means it did not choose, and neither does a view |
| `segments` | one per activity window: `id`, `state`, `committable`, `talks`, `from`, `to` |
| `talks` | one tree per talk, in time order. See the node below |
+| `loose` | the runs and steps no talk contains, as trees from their highest such ancestor: a child's output the fold parented to the session because the child's stream opened no talk, for instance. Empty for most conversations. With `talks`, it holds every run and step of the fold, so the document covers the whole session |
| `relations` | one per relation of the fold: `id`, `type`, `from`, `to`, `quality`, `via`, `evidence` |
| `unresolved` | one per reference the assembler could not resolve, open or since resolved: `id`, `kind`, `ref`, `reason`, `state` |
@@ -116,11 +117,15 @@ the fold and the records say. Where the fold says `unavailable`, the document sa
## Rendering the whole conversation
The document is complete: a viewer draws every view of a conversation from it and fetches
-nothing else. This is how each view reads it.
+nothing else, however many rounds the session was parsed in. A session landed and parsed in three
+stages has three rounds and its landed files cut at each stage, and the document built from the
+head holds every round, every file, and every talk, run and step, exactly as one built from a
+single parse would; the scenario `three-rounds` checks that at each stage, and every scenario
+checks the property `view_covers_the_session` at its end. This is how each view reads it.
| View | Read |
| --- | --- |
-| **Transcript** | `talks`, in order. Each talk's `label` is the person's input and its `reply` the last assistant message; its `children` are the runs, a run's children the steps, and a call's children what it produced: thinking, messages, tools. `text` is what to show for a step, `name` and `result` for a tool, `usage` on a call. |
+| **Transcript** | `talks`, in order. Each talk's `label` is the person's input and its `reply` the last assistant message; its `children` are the runs, a run's children the steps, and a call's children what it produced: thinking, messages, tools. `text` is what to show for a step, `name` and `result` for a tool, `usage` on a call. `loose` holds whatever no talk contains, and is usually empty. |
| **Flow timeline** | every node of every tree by `at`, with `kind` and `stream`; a node with `at` of `0` was never observed at a time and is placed by its position. |
| **Cross-stream flow** | `edges` on a node, and `relations` as the whole list: `starts` from an agent call to the child's `stream`, `reports` from the notification that resumed the parent, `ends_with` from a stream to the child's output, `follows` between epochs across a reset, `summarizes` from a summary to its boundary, `in_segment` from a talk to its window. Containment never crosses a stream; a child's work is under the child. |
| **Streams and segments** | `streams` with `role`, `label`, `parent` and `opened_by`, the step that started each; `segments` with the span of the talks placed in them. |
diff --git a/docs/en/guides/scenario.md b/docs/en/guides/scenario.md
index d0ecdfc..fbebfbd 100644
--- a/docs/en/guides/scenario.md
+++ b/docs/en/guides/scenario.md
@@ -147,6 +147,7 @@ properties: # all on unless set false
cross_format: true
records_match: true
push_follows_the_wire: true
+ view_covers_the_session: true
parse:
max_round_bytes: 0 # a parse setting, when the scenario needs one
push:
@@ -164,6 +165,14 @@ writer plants beside the session. Across formats, the folds must agree, and so m
records themselves, field by field: the runtime's adapter and the sd writer must land the same
evidence from the same scenario, which is what makes a scenario a conformance test for an adapter.
+The document is checked too: at the end of every scenario, `view_covers_the_session` holds the
+`asz.view` document to the whole session: every round, verified; every landed file with its digest
+as on disk; every talk, run and step of the fold in a tree; the session's own range; and a verified
+state. A scenario with checkpoints is the multi-round case: `three-rounds` lands and parses in
+three stages, from the start to the first checkpoint, from there to the second, and from there to
+the end, so three rounds sit over landed files cut at each stage, and the final document must
+cover the session as one parse would.
+
The push is checked too. Every scenario, in both formats, is pushed to a receiver in the test,
one file per request, and every request is compared with the tables of
[Export over OpenTelemetry](../setup/export-otlp.md): the resource and the scope, one record per
diff --git a/internal/scenario/expect/expect.go b/internal/scenario/expect/expect.go
index 76bd0db..89de655 100644
--- a/internal/scenario/expect/expect.go
+++ b/internal/scenario/expect/expect.go
@@ -40,6 +40,7 @@ import (
"github.com/apache/skywalking-ai-sessionizer/pkg/model"
"github.com/apache/skywalking-ai-sessionizer/pkg/sessiondata"
"github.com/apache/skywalking-ai-sessionizer/pkg/sessionflow"
+ "github.com/apache/skywalking-ai-sessionizer/pkg/sessionview"
)
// File is an expectation file: one block per checkpoint, "final" for the
@@ -79,6 +80,7 @@ type Properties struct {
DiscoveryIgnoresNoise *bool `yaml:"discovery_ignores_noise"`
RepackKeepsStructure *bool `yaml:"repack_keeps_structure"`
PushFollowsTheWire *bool `yaml:"push_follows_the_wire"`
+ ViewCoversTheSession *bool `yaml:"view_covers_the_session"`
}
// On reports whether a property is enabled.
@@ -826,3 +828,115 @@ func RecordsWellFormed(root, session string) ([]string, error) {
}
return out, nil
}
+
+// ViewCoversTheSession checks the asz.view document holds the whole
+// session: every round of the chain, verified; every landed file, with its
+// digest as on disk; every talk, run and step of the fold in a tree, under
+// a talk or under loose; the session's own range; and a verified state.
+func ViewCoversTheSession(root, session string) ([]string, error) {
+ srv := view.New(storage.NewZone(root), nil)
+ c, err := srv.Load(session)
+ if err != nil {
+ return nil, err
+ }
+ doc, err := c.Build()
+ if err != nil {
+ return nil, err
+ }
+ v := c.View
+ var out []string
+ bad := func(format string, a ...any) {
+ out = append(out, "view_covers_the_session: "+fmt.Sprintf(format, a...))
+ }
+ if doc.Summary.State != sessionview.StateVerified {
+ bad("state is %s: %v", doc.Summary.State, doc.Summary.Problems)
+ }
+ // Every round, verified, and the head.
+ chain := sessionflow.OpenChain(root, session)
+ files, err := chain.List()
+ if err != nil {
+ return nil, err
+ }
+ if len(doc.Rounds) != len(files) || doc.Head.Round != v.Round || doc.Head.Digest != v.Digest {
+ bad("%d rounds in the document, %d on disk; head %d/%s against %d/%s", len(doc.Rounds), len(files), doc.Head.Round, firstN(doc.Head.Digest, 12), v.Round, firstN(v.Digest, 12))
+ }
+ for _, r := range doc.Rounds {
+ if !r.Verified {
+ bad("round %d is not verified", r.Round)
+ }
+ }
+ // Every landed file, as on disk.
+ landed, err := storage.LandedFiles(storage.NewZone(root), session)
+ if err != nil {
+ return nil, err
+ }
+ byFile := map[string]sessionview.File{}
+ for _, f := range doc.Files {
+ byFile[f.File] = f
+ }
+ for _, lf := range landed {
+ rel, _ := filepath.Rel(root, lf.Path)
+ rel = filepath.ToSlash(rel)
+ f, ok := byFile[rel]
+ if !ok {
+ bad("landed file %s is not in the document", rel)
+ continue
+ }
+ d, err := storage.FileDigest(lf.Path)
+ if err != nil {
+ return nil, err
+ }
+ if f.Digest != d || f.Seq == nil || *f.Seq != lf.Seq {
+ bad("landed file %s: digest or sequence differ from disk", rel)
+ }
+ }
+ if len(doc.Files) != len(landed)+len(files) {
+ bad("%d files in the document, %d landed and %d rounds on disk", len(doc.Files), len(landed), len(files))
+ }
+ // Every talk, run and step, in a tree.
+ inTrees := map[string]bool{}
+ var walk func(n *sessionview.Node)
+ walk = func(n *sessionview.Node) {
+ inTrees[n.ID] = true
+ for i := range n.Children {
+ walk(&n.Children[i])
+ }
+ }
+ for i := range doc.Talks {
+ walk(&doc.Talks[i])
+ }
+ for i := range doc.Loose {
+ walk(&doc.Loose[i])
+ }
+ for id, n := range v.Nodes {
+ switch n.Kind {
+ case model.KindConversation, model.KindSession, model.KindStream, model.KindEpoch, model.KindSegment:
+ continue
+ }
+ if !inTrees[id] {
+ bad("%s (%s) is in the fold but in no tree of the document", id, n.Kind)
+ }
+ }
+ if len(doc.Talks) != len(v.NodesByKind(model.KindTalk)) || doc.Summary.Talks != len(doc.Talks) {
+ bad("%d talks in the document, %d in the fold, summary says %d", len(doc.Talks), len(v.NodesByKind(model.KindTalk)), doc.Summary.Talks)
+ }
+ // The session's own range.
+ if sn := v.Nodes[sessionflow.NodeID("session", session)]; sn != nil {
+ var attrs struct {
+ From string `json:"from_time"`
+ Through string `json:"through_time"`
+ }
+ _ = json.Unmarshal(sn.Attrs, &attrs)
+ if doc.Summary.From != stamp(attrs.From).UnixMilli() || doc.Summary.To != stamp(attrs.Through).UnixMilli() {
+ bad("summary range %d..%d, the session node says %s..%s", doc.Summary.From, doc.Summary.To, attrs.From, attrs.Through)
+ }
+ }
+ return out, nil
+}
+
+func firstN(s string, n int) string {
+ if len(s) <= n {
+ return s
+ }
+ return s[:n]
+}
diff --git a/internal/scenario/run/run.go b/internal/scenario/run/run.go
index d187c0d..6a01b89 100644
--- a/internal/scenario/run/run.go
+++ b/internal/scenario/run/run.go
@@ -227,6 +227,7 @@ func checkFormat(sc *scenario.Scenario, ex *expect.File, f scenario.Format, out
{"records_well_formed", ex.Properties.RecordsWellFormed, func() ([]string, error) { return expect.RecordsWellFormed(out, session) }},
{"repack_keeps_structure", ex.Properties.RepackKeepsStructure, func() ([]string, error) { return repackKeepsStructure(out, session, ex.Parse.MaxRoundBytes) }},
{"push_follows_the_wire", ex.Properties.PushFollowsTheWire, func() ([]string, error) { return pushFollowsTheWire(out, session, ex.Push) }},
+ {"view_covers_the_session", ex.Properties.ViewCoversTheSession, func() ([]string, error) { return expect.ViewCoversTheSession(out, session) }},
{"reproducible", ex.Properties.Reproducible, func() ([]string, error) {
// Two parses of identical landed evidence must produce identical
// rounds. The chain in out was cut at checkpoints, so it is not
diff --git a/internal/view/conversationview.go b/internal/view/conversationview.go
index 65458f3..907e8aa 100644
--- a/internal/view/conversationview.go
+++ b/internal/view/conversationview.go
@@ -72,7 +72,7 @@ func (c *Conversation) Build() (*sessionview.Conversation, error) {
Kinds: o.kinds, RelationTypes: o.rels, Quality: o.quality,
},
Streams: o.streams, Segments: o.segments,
- Rounds: []sessionview.Round{}, Files: []sessionview.File{}, Talks: []sessionview.Node{},
+ Rounds: []sessionview.Round{}, Files: []sessionview.File{}, Talks: []sessionview.Node{}, Loose: []sessionview.Node{},
Relations: []sessionview.Relation{}, Unresolved: []sessionview.Unresolved{},
}
if sn := c.View.Nodes[sessionflow.NodeID("session", c.Session)]; sn != nil {
@@ -93,10 +93,14 @@ func (c *Conversation) Build() (*sessionview.Conversation, error) {
v.Summary.State, v.Summary.Problems = state, problems
talks := c.Talks()
+ loose := c.looseRoots()
var refs []*sessionflow.Ref
for _, t := range talks {
refs = append(refs, c.refsUnder(t)...)
}
+ for _, n := range loose {
+ refs = append(refs, c.refsUnder(n)...)
+ }
recs := c.records(refs)
rows := map[string]talkRow{}
for _, row := range o.talks {
@@ -120,6 +124,9 @@ func (c *Conversation) Build() (*sessionview.Conversation, error) {
}
}
+ for _, n := range loose {
+ v.Loose = append(v.Loose, c.step(n, 0, recs))
+ }
for _, r := range c.View.Relations {
v.Relations = append(v.Relations, sessionview.Relation{
ID: r.ID, Type: r.Type, From: r.From, To: r.To, Quality: r.Quality, Via: r.Via, Evidence: r.Evidence,
@@ -137,6 +144,39 @@ func (c *Conversation) Build() (*sessionview.Conversation, error) {
return v, nil
}
+// looseRoots finds the runs and steps no talk contains, and returns the
+// highest such ancestor of each, in order, so the document can hold them
+// as trees. A step is contained by a talk when a talk is above it; one
+// whose ancestors are only structure, the session, a stream, an epoch or a
+// segment, is loose.
+func (c *Conversation) looseRoots() []*sessionflow.Node {
+ roots := map[string]*sessionflow.Node{}
+ for _, n := range c.View.Nodes {
+ if n.Kind != model.KindRun && !isStep(n.Kind) {
+ continue
+ }
+ top := n
+ covered := false
+ for cur := n; cur != nil; cur = c.View.Nodes[cur.Parent] {
+ if cur.Kind == model.KindTalk {
+ covered = true
+ break
+ }
+ if cur.Kind == model.KindRun || isStep(cur.Kind) {
+ top = cur
+ }
+ }
+ if !covered {
+ roots[top.ID] = top
+ }
+ }
+ var out []*sessionflow.Node
+ for _, n := range roots {
+ out = append(out, n)
+ }
+ return sessionflow.InOrder(out)
+}
+
// sessions lists the sessions the fold holds, the conversation's own first.
func (c *Conversation) sessions() []string {
out := []string{c.Session}
diff --git a/pkg/sessionview/sessionview.go b/pkg/sessionview/sessionview.go
index 1e861ed..8c453d1 100644
--- a/pkg/sessionview/sessionview.go
+++ b/pkg/sessionview/sessionview.go
@@ -62,11 +62,17 @@ type Conversation struct {
Summary Summary `json:"summary"`
- Rounds []Round `json:"rounds"`
- Files []File `json:"files"`
- Streams []Stream `json:"streams"`
- Segments []Segment `json:"segments"`
- Talks []Node `json:"talks"`
+ Rounds []Round `json:"rounds"`
+ Files []File `json:"files"`
+ Streams []Stream `json:"streams"`
+ Segments []Segment `json:"segments"`
+ Talks []Node `json:"talks"`
+ // Loose holds the runs and steps no talk contains, as trees from their
+ // highest such ancestor: a child's output the fold parented to the
+ // session because the child's stream opened no talk, for instance. It
+ // is empty for most conversations, and it exists so the document holds
+ // every step the fold holds.
+ Loose []Node `json:"loose"`
Relations []Relation `json:"relations"`
Unresolved []Unresolved `json:"unresolved"`
}
diff --git a/tests/scenarios/three-rounds.expect.yaml b/tests/scenarios/three-rounds.expect.yaml
new file mode 100644
index 0000000..0f749d1
--- /dev/null
+++ b/tests/scenarios/three-rounds.expect.yaml
@@ -0,0 +1,38 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+checkpoints:
+ first:
+ rounds: 1
+ written: true
+ talks_on: {main: 1}
+ view: {state: verified, rounds: 1, talks: 1}
+ second:
+ rounds: 2
+ written: true
+ delta: true
+ talks_on: {main: 2, checker: 1}
+ relations: {starts: 1, reports: 1, ends_with: 1}
+ view: {state: verified, rounds: 2, talks: 3}
+ final:
+ rounds: 3
+ written: true
+ delta: true
+ kinds: {talk: 6, stream: 2, epoch: 3, epoch.boundary: 1, tool: 2} # the reset's summary opens a talk of its own
+ talks_on: {main: 5, checker: 1}
+ relations: {starts: 1, reports: 1, ends_with: 1, follows: 1, summarizes: 1}
+ view: {state: verified, rounds: 3, talks: 6}
diff --git a/tests/scenarios/three-rounds.yaml b/tests/scenarios/three-rounds.yaml
new file mode 100644
index 0000000..cc87836
--- /dev/null
+++ b/tests/scenarios/three-rounds.yaml
@@ -0,0 +1,37 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# One session, landed and parsed in three stages: from the start to the
+# first checkpoint, from there to the second, and from there to the end.
+# Three rounds come out, over landed files cut at each stage, and the final
+# document must cover the whole session as if it had been parsed at once.
+title: three rounds
+steps:
+ - input: start the work
+ - call: {text: Starting, tool: {name: Bash, input: {command: make build}, result: built}}
+ - call: {text: Build done.}
+ checkpoint: first
+ - input: now check it
+ - call:
+ agent: {name: checker, prompt: check the build, steps: [{call: {text: checks pass}}], notify: true}
+ - call: {text: Checks done.}
+ checkpoint: second
+ - input: wrap up
+ - call: {text: Wrapping up, tool: {name: Bash, input: {command: git status}, result: clean}}
+ - reset: {summary: "Summary: built, checked, wrapped up."}
+ - input: one more thing
+ - call: {text: All done.}