You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2026/10/08 14:20:57 UTC
(skywalking-ai-sessionizer) 01/01: Give a run that supplies no thread the thread of the run it ran inside
This is an automated email from the ASF dual-hosted git repository.
wu-sheng pushed a commit to branch langsmith-inherit-thread
in repository https://gitbox.apache.org/repos/asf/skywalking-ai-sessionizer.git
commit a1b7bea819e1c181d42da803ba19e9f00abc85af
Author: Wu Sheng <wu...@foxmail.com>
AuthorDate: Thu Oct 8 22:20:46 2026 +0800
Give a run that supplies no thread the thread of the run it ran inside
LangSmith JS passes a decorated function's metadata to none of the runs
inside it. A model call made inside a traceable function therefore named
no thread, and landed under its trace in a conversation of its own: the
answer was there, and the conversation of the question had none. The
Python client passes the metadata down itself, so its runs were whole.
A run that supplies no key now takes the conversation of the nearest run
it ran inside whose conversation was supplied. The trace and the parent
on the wire say where it ran, and the thread was supplied there. A run
that supplies its own key keeps it, and a trace that supplied none
anywhere still lands unassigned.
The conversation is decided once the run's ancestry has arrived, which
the receiver already waits for. While a request waited, the unassigned
name of such a run was remembered, and kept when the request was tried
again after its root arrived; it is no longer remembered until the run
lands. A run that landed unassigned after waiting as long as it may stays
there, and its later arrivals follow it, so one run is never split. A run
above that landed that way supplied nothing, and is passed over.
The JS traceable capture now lands as one conversation and assembles
with the question, the model call and its answer. Six tests cover the
rule; breaking each part of it fails one.
---
docs/en/adapters/langsmith.md | 25 +++-
docs/en/changes/changes.md | 13 +-
docs/en/setup/langchain.md | 12 ++
internal/adapters/langsmith/collect.go | 45 ++++--
internal/adapters/langsmith/inherit_test.go | 210 ++++++++++++++++++++++++++++
internal/adapters/langsmith/pending.go | 18 +++
tests/langchain_test.go | 5 +
tests/langchainjs_test.go | 32 ++---
8 files changed, 321 insertions(+), 39 deletions(-)
diff --git a/docs/en/adapters/langsmith.md b/docs/en/adapters/langsmith.md
index 7ae8d74..2abf133 100644
--- a/docs/en/adapters/langsmith.md
+++ b/docs/en/adapters/langsmith.md
@@ -51,9 +51,10 @@ lands as the Python one does: the person's question opens the turn, a tool's res
tool said, and a decorated function keeps its arguments. Read as the Python client writes them,
every JS run was refused for its end time, and, with that read, no question opened a LangGraph JS
conversation and a tool's result landed as the serialized message. Because the metadata is not
-passed down, a model call made inside a decorated function supplies no thread of its own, and
-lands under its trace as any run that supplied none does; a LangGraph JS graph puts the thread on
-every run, so its conversation is whole.
+passed down, a model call made inside a decorated function supplies no thread of its own. It takes
+the function's, as [Ownership is a namespace](#ownership-is-a-namespace) says a run that supplies
+none does; landed under its trace, the answer was a conversation of its own and the question's had
+none. A LangGraph JS graph puts the thread on every run.
An update carries only what changed. `update_run(id, outputs=...)` sends the id and the outputs,
with the trace, the project, the kind of run and the dotted order all null — read on its own,
@@ -304,8 +305,22 @@ and `会话-1` as real thread keys, so the session directory is derived from the
a digest of the whole namespace, under an `ls-` prefix that keeps it off the leading underscore
enumeration skips and away from every Windows device name.
-A trace with no supplied key is not given one. It lands under a name that says the identity was not
-supplied, and nothing merges it with anything else.
+A run that supplies no key takes the conversation of the nearest run it ran inside whose
+conversation was supplied. That run supplied the thread, and the trace and the parent on the wire
+say this one ran inside it, so nothing is guessed. A run that supplies its own key keeps it. The
+Python client passes a decorated function's metadata down to the runs inside it, and LangSmith JS
+does not, so without this a model call made inside a JS function landed apart from the question it
+answered.
+
+The order runs arrive in does not decide it. A run waits for its ancestry, as
+[A nested agent is its own stream](#a-nested-agent-is-its-own-stream) says, and its conversation is
+decided when the ancestry has arrived. A run whose ancestry has not arrived when it has waited as
+long as it may lands as one that supplied none, and stays there: a later arrival of the same run
+lands where its first one did, so one run is never split across two conversations. A run above it
+that landed that way supplied nothing, and is passed over.
+
+A trace with no supplied key anywhere is not given one. It lands under a name that says the
+identity was not supplied, and nothing merges it with anything else.
## What the wire cannot supply
diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md
index 6e94272..71d5cff 100644
--- a/docs/en/changes/changes.md
+++ b/docs/en/changes/changes.md
@@ -19,10 +19,19 @@
of these now reads the message by its class. A decorated function carries no `ls_method` from it,
and is known by the library that made the run, which both clients write. A model call inside
a decorated function supplies no thread, because LangSmith JS passes the function's metadata to
- none of the runs inside it, and lands under its trace. Measured with LangSmith JS 0.10.8 on a
- LangGraph JS agent and a traced function, the two applications under `tests/apps/langchain-js`.
+ none of the runs inside it, and takes the function's, as the next entry says. Measured with
+ LangSmith JS 0.10.8 on a LangGraph JS agent and a traced function, the two applications under
+ `tests/apps/langchain-js`.
Their captures sit beside the Python ones under `tests/apps/langchain/testdata`, and the recorder
reads the chunked body that client sends.
+- A run that supplies no thread takes the conversation of the nearest run it ran inside whose
+ conversation was supplied. The trace and the parent on the wire say where it ran, and the thread
+ was supplied there. A model call inside a LangSmith JS `traceable` supplies none, so it landed in
+ a conversation of its own under its trace, with its answer, while the question's conversation had
+ no answer. A run that supplies its own key keeps it, and a trace that supplied none anywhere still
+ lands unassigned. The conversation is decided once the run's ancestry has arrived, so the order
+ runs arrive in does not change it. A run landed unassigned after waiting as long as it may stays
+ there, and its later arrivals follow it.
- A summary `SummarizationMiddleware` wrote now resets the context, as a Claude Code compaction
does. The middleware marks its summary message with `additional_kwargs.lc_source` set to
`summarization`. A model call sent that message starts a new epoch in its stream, with an
diff --git a/docs/en/setup/langchain.md b/docs/en/setup/langchain.md
index 5d5ceb4..e1c9b08 100644
--- a/docs/en/setup/langchain.md
+++ b/docs/en/setup/langchain.md
@@ -70,6 +70,18 @@ metadata instead:
def answer(question): ...
```
+In LangChain JS, the same goes on the client's `traceable`:
+
+```js
+const answer = traceable(async (question) => { /* ... */ }, {
+ name: "answer",
+ metadata: { thread_id: "the-conversation" },
+});
+```
+
+A run inside it that names no thread, such as a model call, is part of the same
+conversation.
+
## What owns a conversation
A thread key alone is not an identity. Two applications can both use the
diff --git a/internal/adapters/langsmith/collect.go b/internal/adapters/langsmith/collect.go
index a6c3b3d..682acd8 100644
--- a/internal/adapters/langsmith/collect.go
+++ b/internal/adapters/langsmith/collect.go
@@ -306,7 +306,7 @@ func (c *Collector) convertWith(request Waiting, open *pending, wait bool) (Land
runs := make([]Run, len(operations))
sessions := make([]string, len(operations))
methods := make([]string, len(operations))
- unassigned := 0
+ unsupplied := make([]bool, len(operations))
for i, op := range operations {
var envelope Run
if err := json.Unmarshal(op.Envelope, &envelope); err != nil {
@@ -324,14 +324,18 @@ func (c *Collector) convertWith(request Waiting, open *pending, wait bool) (Land
case found && known.Session != "":
sessions[i] = known.Session
default:
- var missing bool
- sessions[i], missing = c.sessionOf(op, completed)
- if missing {
- unassigned++
- }
+ sessions[i], unsupplied[i] = c.sessionOf(op, completed)
}
if completed.Type != "" {
- open.started(completed, methodOf(op), sessions[i], c.Now())
+ // A run that supplied no conversation is not remembered in one
+ // yet. Which it takes is decided below, once its ancestry has
+ // arrived, and remembering the unassigned one here kept it when
+ // the request was tried again after its root had arrived.
+ remembered := sessions[i]
+ if unsupplied[i] {
+ remembered = ""
+ }
+ open.started(completed, methodOf(op), remembered, c.Now())
}
open.sawRun(completed, c.Now())
}
@@ -346,6 +350,24 @@ func (c *Collector) convertWith(request Waiting, open *pending, wait bool) (Land
}
}
+ // A run that supplied no conversation takes the one the nearest run it
+ // ran inside supplied. LangSmith JS passes a decorated function's
+ // metadata to none of the runs inside it, so a model call the function
+ // makes names no thread, although the function names one and the trace
+ // and the parent say the call ran inside it. Only when no run above it
+ // supplied one does it land unassigned, under its trace.
+ unassigned := 0
+ for i := range operations {
+ if !unsupplied[i] {
+ continue
+ }
+ if session := open.inherited(runs[i]); session != "" {
+ sessions[i] = session
+ continue
+ }
+ unassigned++
+ }
+
byRun := make(map[string]Run, len(operations))
for i, op := range operations {
if _, seen := byRun[op.RunID]; !seen || runs[i].Type != "" {
@@ -431,11 +453,12 @@ func (c *Collector) convertWith(request Waiting, open *pending, wait bool) (Land
return out, nil
}
-// sessionOf decides where a run's evidence lands.
+// sessionOf decides where a run's evidence lands, by what the run itself
+// supplied.
//
-// A run with no supplied key is not given one: it lands under its trace, in a
-// session whose name says the identity was not supplied, and nothing merges it
-// with anything else.
+// A run with no supplied key is not given one here: it is under its trace, in
+// a session whose name says the identity was not supplied, and the second
+// return says so. The caller then looks for a run above it that supplied one.
func (c *Collector) sessionOf(op Operation, envelope Run) (string, bool) {
// The client splits extra out of the envelope only when it is large
// enough to be worth a part of its own, so both places have to be read.
diff --git a/internal/adapters/langsmith/inherit_test.go b/internal/adapters/langsmith/inherit_test.go
new file mode 100644
index 0000000..1d1a1fb
--- /dev/null
+++ b/internal/adapters/langsmith/inherit_test.go
@@ -0,0 +1,210 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package langsmith
+
+import (
+ "net/http"
+ "strings"
+ "testing"
+)
+
+// A decorated function and the model call it makes, as LangSmith JS sends
+// them: the thread is on the function alone, since that client passes a
+// decorated function's metadata to none of the runs inside it.
+const (
+ inheritRoot = "14141414-1414-7414-8414-141414141410"
+ inheritChild = "14141414-1414-7414-8414-141414141411"
+ inheritOrder = "20260920T100000000000Z" + inheritRoot
+)
+
+func inheritFunction(thread string) string {
+ return `{"id":"` + inheritRoot + `","trace_id":"` + inheritRoot + `","dotted_order":"` + inheritOrder + `",` +
+ `"run_type":"chain","name":"review","start_time":"2026-09-20T10:00:00Z","end_time":"2026-09-20T10:00:03Z",` +
+ `"session_name":"asz","extra":{"metadata":{"thread_id":"` + thread + `"},"runtime":{"library":"langsmith"}},` +
+ `"inputs":{"question":"what changed in the docs"},"outputs":{"answer":"two pages"}}`
+}
+
+func inheritCall(metadata string) string {
+ return `{"id":"` + inheritChild + `","trace_id":"` + inheritRoot + `","parent_run_id":"` + inheritRoot + `",` +
+ `"dotted_order":"` + inheritOrder + `.20260920T100001000000Z` + inheritChild + `",` +
+ `"run_type":"llm","name":"FakeListChatModel","start_time":"2026-09-20T10:00:01Z",` +
+ `"end_time":"2026-09-20T10:00:02Z","session_name":"asz","extra":{"metadata":{` + metadata + `}},` +
+ `"outputs":{"generations":[[{"message":{"kwargs":{"content":"the call's own answer"}}}]]}}`
+}
+
+func postBatch(t *testing.T, base, body string) {
+ t.Helper()
+ resp := send(t, base, "/runs/batch", "application/json", []byte(body), "")
+ resp.Body.Close()
+ if resp.StatusCode != http.StatusAccepted {
+ t.Fatalf("status %d", resp.StatusCode)
+ }
+}
+
+// TestARunWithNoThreadTakesTheFunctionsItRanInside. The call names no thread,
+// but the function it ran inside does, and the trace and the parent on the
+// wire say the call ran there. Landed under its trace, the answer was a
+// conversation of its own, and the question's conversation had none. The
+// call is first in the batch, so the order inside a request decides nothing.
+func TestARunWithNoThreadTakesTheFunctionsItRanInside(t *testing.T) {
+ _, zone, base := started(t, "")
+ postBatch(t, base, `{"post":[`+inheritCall(``)+`,`+inheritFunction("t-inherit")+`]}`)
+ landed, err := collectorOver(zone).Collect()
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(landed.Sessions) != 1 || landed.Unassigned != 0 {
+ t.Fatalf("sessions %v, %d unassigned, want the function's thread alone", landed.Sessions, landed.Unassigned)
+ }
+ if !mentions(t, zone, landed.Sessions[0], "the call's own answer") {
+ t.Error("the call's answer is not in the function's conversation")
+ }
+}
+
+// TestARunThatArrivesBeforeItsFunctionStillTakesItsThread. The client sends
+// several batches at once and retries a failed one, so the call can arrive a
+// pass before the function. It waits for its ancestry, and its conversation
+// is decided when the function has arrived, not when it first came: the
+// unassigned name it would have had then was remembered, and kept when the
+// request was tried again.
+func TestARunThatArrivesBeforeItsFunctionStillTakesItsThread(t *testing.T) {
+ _, zone, base := started(t, "")
+ collector := collectorOver(zone)
+ postBatch(t, base, `{"post":[`+inheritCall(``)+`]}`)
+ landed, err := collector.Collect()
+ if err != nil {
+ t.Fatal(err)
+ }
+ if landed.Waiting != 1 || landed.Records != 0 {
+ t.Fatalf("%d waiting, %d records, want the call held for its function", landed.Waiting, landed.Records)
+ }
+ postBatch(t, base, `{"post":[`+inheritFunction("t-later")+`]}`)
+ landed, err = collector.Collect()
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(landed.Sessions) != 1 || landed.Unassigned != 0 {
+ t.Fatalf("sessions %v, %d unassigned, want the function's thread alone", landed.Sessions, landed.Unassigned)
+ }
+ if !mentions(t, zone, landed.Sessions[0], "the call's own answer") {
+ t.Error("the call's answer is not in the function's conversation")
+ }
+}
+
+// TestARunsOwnThreadIsKept. A run that supplies a thread keeps it, whatever
+// the run it ran inside supplied.
+func TestARunsOwnThreadIsKept(t *testing.T) {
+ _, zone, base := started(t, "")
+ postBatch(t, base, `{"post":[`+inheritFunction("t-outer")+`,`+inheritCall(`"thread_id":"t-own"`)+`]}`)
+ landed, err := collectorOver(zone).Collect()
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(landed.Sessions) != 2 {
+ t.Fatalf("sessions %v, want the function's and the call's own", landed.Sessions)
+ }
+ for _, session := range landed.Sessions {
+ answer := mentions(t, zone, session, "the call's own answer")
+ if strings.Contains(session, "t-own") != answer {
+ t.Errorf("%s: holds the call's answer %v", session, answer)
+ }
+ }
+}
+
+// TestATraceThatSuppliedNoThreadIsNotGivenOne. With no thread anywhere in the
+// trace, nothing is inherited: the function and the call land together under
+// the trace, in a session named as unassigned.
+func TestATraceThatSuppliedNoThreadIsNotGivenOne(t *testing.T) {
+ _, zone, base := started(t, "")
+ function := strings.Replace(inheritFunction("x"), `"thread_id":"x"`, ``, 1)
+ postBatch(t, base, `{"post":[`+function+`,`+inheritCall(``)+`]}`)
+ landed, err := collectorOver(zone).Collect()
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(landed.Sessions) != 1 || landed.Sessions[0] != UnassignedID(inheritRoot) || landed.Unassigned != 2 {
+ t.Fatalf("sessions %v, %d unassigned, want both runs under %s",
+ landed.Sessions, landed.Unassigned, UnassignedID(inheritRoot))
+ }
+}
+
+// TestARunLandedUnassignedStaysWhereItLanded. A call whose function never
+// arrived while it waited is landed under its trace. When the function and
+// an update to the call arrive later, the update stays with the call's first
+// arrival: one run is never split across two conversations.
+func TestARunLandedUnassignedStaysWhereItLanded(t *testing.T) {
+ _, zone, base := started(t, "")
+ collector := collectorOver(zone)
+ start := strings.Replace(inheritCall(``), `"end_time":"2026-09-20T10:00:02Z",`, ``, 1)
+ postBatch(t, base, `{"post":[`+start+`]}`)
+ for pass := 0; pass <= deferLimit; pass++ {
+ if _, err := collector.Collect(); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if !mentions(t, zone, UnassignedID(inheritRoot), inheritChild) {
+ t.Fatal("the call was not landed under its trace once it had waited as long as it may")
+ }
+ update := `{"id":"` + inheritChild + `","trace_id":"` + inheritRoot + `",` +
+ `"dotted_order":"` + inheritOrder + `.20260920T100001000000Z` + inheritChild + `",` +
+ `"end_time":"2026-09-20T10:00:02Z",` +
+ `"outputs":{"generations":[[{"message":{"kwargs":{"content":"the late answer"}}}]]}}`
+ postBatch(t, base, `{"post":[`+inheritFunction("t-late")+`],"patch":[`+update+`]}`)
+ if _, err := collector.Collect(); err != nil {
+ t.Fatal(err)
+ }
+ if !mentions(t, zone, UnassignedID(inheritRoot), "the late answer") {
+ t.Error("the call's update left the conversation its start landed in")
+ }
+}
+
+// TestAnUnassignedRunAboveIsPassedOver. The step a call ran inside supplied
+// no thread and landed unassigned, because the function above it did not
+// arrive while it waited. The function supplied a thread when it came, and the
+// call arriving with it takes that thread: the step's unassigned name was
+// never supplied by anyone, so it is not inherited.
+func TestAnUnassignedRunAboveIsPassedOver(t *testing.T) {
+ _, zone, base := started(t, "")
+ collector := collectorOver(zone)
+ const step = "14141414-1414-7414-8414-141414141412"
+ stepOrder := inheritOrder + ".20260920T100000500000Z" + step
+ postBatch(t, base, `{"post":[{"id":"`+step+`","trace_id":"`+inheritRoot+`","parent_run_id":"`+inheritRoot+`",`+
+ `"dotted_order":"`+stepOrder+`","run_type":"chain","name":"step","start_time":"2026-09-20T10:00:00.5Z",`+
+ `"end_time":"2026-09-20T10:00:02.5Z","session_name":"asz","extra":{"metadata":{}},"outputs":{"done":true}}]}`)
+ for pass := 0; pass <= deferLimit; pass++ {
+ if _, err := collector.Collect(); err != nil {
+ t.Fatal(err)
+ }
+ }
+ if !mentions(t, zone, UnassignedID(inheritRoot), step) {
+ t.Fatal("the step was not landed under its trace once it had waited as long as it may")
+ }
+ call := strings.Replace(inheritCall(``), `"parent_run_id":"`+inheritRoot+`","dotted_order":"`+inheritOrder,
+ `"parent_run_id":"`+step+`","dotted_order":"`+stepOrder, 1)
+ postBatch(t, base, `{"post":[`+inheritFunction("t-deep")+`,`+call+`]}`)
+ landed, err := collector.Collect()
+ if err != nil {
+ t.Fatal(err)
+ }
+ if len(landed.Sessions) != 1 || landed.Unassigned != 0 {
+ t.Fatalf("sessions %v, %d unassigned, want the function's thread alone", landed.Sessions, landed.Unassigned)
+ }
+ if !mentions(t, zone, landed.Sessions[0], "the call's own answer") {
+ t.Error("the call's answer is not in the function's conversation")
+ }
+}
diff --git a/internal/adapters/langsmith/pending.go b/internal/adapters/langsmith/pending.go
index 04b1ba2..663095c 100644
--- a/internal/adapters/langsmith/pending.go
+++ b/internal/adapters/langsmith/pending.go
@@ -244,6 +244,24 @@ func (p *pending) complete(run Run) (Run, pendingRun, bool) {
return run, known, true
}
+// inherited is the conversation of the nearest run this one ran inside whose
+// conversation was supplied, by that run or by one above it. A run whose own
+// is unassigned is passed over, since it supplied nothing to inherit. Nothing
+// when no run above supplied one, or none of them has been seen.
+func (p *pending) inherited(run Run) string {
+ for _, id := range ancestors(run.Dotted, run.ID) {
+ known := p.Runs[id]
+ trace := known.Trace
+ if trace == "" {
+ trace = id
+ }
+ if known.Session != "" && known.Session != UnassignedID(trace) {
+ return known.Session
+ }
+ }
+ return ""
+}
+
// sawRun remembers one run of a trace: that it was seen at all, and whether
// it was a tool.
func (p *pending) sawRun(run Run, now time.Time) {
diff --git a/tests/langchain_test.go b/tests/langchain_test.go
index 8a36f40..794e602 100644
--- a/tests/langchain_test.go
+++ b/tests/langchain_test.go
@@ -83,6 +83,11 @@ func TestALangChainConversationAssembles(t *testing.T) {
// and end sent as one post, its end time in milliseconds, each message
// serialized with its kind named only by the class that ends its id.
{"js-graph", 1, 1, 1},
+ // LangSmith JS's traceable around a chat model. The thread is on the
+ // function alone, and the model call takes it from the function it
+ // ran inside, so the question, the call and its answer are one
+ // conversation.
+ {"js-traceable", 1, 0, 1},
} {
t.Run(tc.kase, func(t *testing.T) {
zone, sessions := land(t, tc.kase)
diff --git a/tests/langchainjs_test.go b/tests/langchainjs_test.go
index 5bbed30..2f6fa62 100644
--- a/tests/langchainjs_test.go
+++ b/tests/langchainjs_test.go
@@ -140,40 +140,30 @@ func TestALangGraphJSConversationLandsWhatWasSaid(t *testing.T) {
// ls_method alone, the function was taken for a graph's own run and its
// arguments and result were dropped as a repeat; its system message is then
// named where it lands. LangSmith JS also passes the function's metadata to
-// none of the runs inside it, so the model call it made supplies no thread,
-// and lands under its trace, as any run that supplied none does.
+// none of the runs inside it, so the model call it made supplies no thread.
+// It takes the function's, the run it ran inside: landed under its trace, the
+// answer was in a conversation of its own and the question's had none.
func TestALangChainJSTracedFunctionKeepsItsArguments(t *testing.T) {
zone, sessions := land(t, "js-traceable")
- var thread, unassigned string
- for _, s := range sessions {
- if strings.HasPrefix(s, "ls-unassigned-") {
- unassigned = s
- } else {
- thread = s
- }
- }
- if thread == "" || unassigned == "" || len(sessions) != 2 {
- t.Fatalf("sessions %v, want the thread's and the model call's own", sessions)
+ if len(sessions) != 1 || strings.HasPrefix(sessions[0], "ls-unassigned-") {
+ t.Fatalf("sessions %v, want the thread's alone", sessions)
}
- kept := false
- for _, l := range transcriptLines(t, zone, thread) {
+ kept, answered := false, false
+ for _, l := range transcriptLines(t, zone, sessions[0]) {
if strings.Contains(l.raw, "You are a careful reviewer.") {
kept = strings.Contains(l.raw, `"limit"`)
if !slices.Contains(l.Flags, sessiondata.FlagSystemPrompt) {
t.Errorf("the function's arguments hold the system prompt unnamed: %.200s", l.raw)
}
}
- }
- if !kept {
- t.Error("the function's arguments were not kept")
- }
- answered := false
- for _, l := range transcriptLines(t, zone, unassigned) {
for _, p := range l.Parts {
answered = answered || p.Text == "Two pages changed: the setup page and the formats page."
}
}
+ if !kept {
+ t.Error("the function's arguments were not kept")
+ }
if !answered {
- t.Error("the model call's reply did not land")
+ t.Error("the model call's reply is not in the thread's conversation")
}
}