You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2026/09/20 14:21:32 UTC
(skywalking-ai-sessionizer) 02/23: Read what the LangSmith client sends, and decide what owns it
This is an automated email from the ASF dual-hosted git repository.
wu-sheng pushed a commit to branch langchain-harness
in repository https://gitbox.apache.org/repos/asf/skywalking-ai-sessionizer.git
commit 61469c503d2a534d8c18d2315c2891b000674652
Author: Wu Sheng <wu...@foxmail.com>
AuthorDate: Sun Sep 20 18:55:51 2026 +0800
Read what the LangSmith client sends, and decide what owns it
Two contracts, each with the corpus behind it.
The wire: a multipart ingest body is parsed back into one operation per
arrival, gathering an envelope and the fields sent beside it while keeping
every field's bytes as they came, because a landed record proves its source by
the digest of those bytes and not of something re-encoded. A part name this
cannot read is refused rather than guessed at: that is a client that changed,
and it has to be loud. The parser reads all sixteen captured requests and their
312 operations, and the tests hold two facts the design rests on — that runs
arrive unfinished, and that a process which dies mid-turn completes none of
them.
Ownership: a bare thread key is not an identity, because two applications can
both use the project "production" and the thread "123", and merging them would
put two histories in one conversation while every individual join stayed valid.
So ownership is a namespace, defaulting to project and thread, and an operator
who wants them merged says so rather than discovering it.
A supplied key is never a path. The corpus carries "../outside",
"team/customer", "_hidden" and "会话-1" as real thread keys, so the session
directory is derived: a readable slug, and a digest of the whole namespace so
two owners cannot collide when their slugs do. The "ls-" prefix keeps the name
off the leading underscore that enumeration skips and away from every Windows
device name.
A trace that supplies no key is not given one. Absence is a state to report,
and inventing an identity to fill it would be the one thing the model forbids.
---
internal/adapters/langsmith/identity.go | 170 ++++++++++++++++++++++
internal/adapters/langsmith/identity_test.go | 118 +++++++++++++++
internal/adapters/langsmith/langsmith.go | 82 +++++++++++
internal/adapters/langsmith/wire.go | 192 +++++++++++++++++++++++++
internal/adapters/langsmith/wire_test.go | 206 +++++++++++++++++++++++++++
5 files changed, 768 insertions(+)
diff --git a/internal/adapters/langsmith/identity.go b/internal/adapters/langsmith/identity.go
new file mode 100644
index 0000000..7a89064
--- /dev/null
+++ b/internal/adapters/langsmith/identity.go
@@ -0,0 +1,170 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package langsmith
+
+import (
+ "crypto/sha256"
+ "encoding/hex"
+ "strings"
+ "unicode"
+)
+
+// Ownership is how the evidence in a run says which conversation it belongs to.
+//
+// A bare thread key is not an identity. Two applications can both use the
+// project "production" and the thread "123", and merging them would put two
+// people's histories, tools and costs in one conversation while every
+// individual join stayed valid. So ownership is a namespace the operator
+// names, and the default names the two dimensions the client always carries.
+//
+// Nothing here invents an identity. A trace that supplies no key is not given
+// one: see Owner's second return value, and what the collector does with it.
+type Ownership struct {
+ // Keys are the metadata keys that may carry the thread, in the order they
+ // are tried. The client's own documentation names these three, and an
+ // application that uses another says so here.
+ Keys []string
+ // Scope names the dimensions that together own a conversation, in order.
+ // "project" is the client's session_name; anything else is a metadata
+ // key. Dropping "project" merges every project that shares a thread key,
+ // which is a choice an operator can make and this must not make for them.
+ Scope []string
+}
+
+// DefaultOwnership is project and thread, with the three keys the client
+// documents.
+func DefaultOwnership() Ownership {
+ return Ownership{
+ Keys: []string{"thread_id", "session_id", "conversation_id"},
+ Scope: []string{"project", "thread"},
+ }
+}
+
+// Owner is which conversation a run belongs to, as the run says it.
+type Owner struct {
+ // Values are the dimensions in Scope order, as they were supplied.
+ Values []string
+ // Thread is the thread key itself, kept apart because it is the one a
+ // reader recognises.
+ Thread string
+}
+
+// Owner reads the ownership of one run.
+//
+// The second return is false when no key was supplied. That is a state, not a
+// failure: such evidence is real and is held unassigned rather than landed
+// under something made up.
+func (o Ownership) Owner(project string, metadata map[string]any) (Owner, bool) {
+ thread := ""
+ for _, key := range o.Keys {
+ if v, ok := metadata[key]; ok {
+ if s, ok := v.(string); ok && s != "" {
+ thread = s
+ break
+ }
+ }
+ }
+ if thread == "" {
+ return Owner{}, false
+ }
+ out := Owner{Thread: thread}
+ for _, dimension := range o.Scope {
+ switch dimension {
+ case "project":
+ out.Values = append(out.Values, project)
+ case "thread":
+ out.Values = append(out.Values, thread)
+ default:
+ s, _ := metadata[dimension].(string)
+ out.Values = append(out.Values, s)
+ }
+ }
+ return out, true
+}
+
+// StorageID is the session directory an owner's evidence lands under.
+//
+// A supplied key is never a path. The storage root joins a session id straight
+// into a filesystem path, and the corpus shows what applications really send:
+// "../outside", "team/customer", "_hidden" and "会话-1" all arrived as thread
+// keys. So the id is derived rather than used: a readable slug for whoever
+// reads a directory listing, and a digest of the whole namespace so two owners
+// can never collide even when their slugs do.
+//
+// The "ls-" prefix does three jobs at once. It says which adapter landed the
+// session, it keeps the name off the leading underscore that session
+// enumeration skips, and it keeps every Windows device name (con, nul, com1)
+// from ever being a directory.
+func StorageID(owner Owner) string {
+ joined := strings.Join(owner.Values, "\x1f")
+ sum := sha256.Sum256([]byte(joined))
+ slug := slugOf(owner.Values)
+ if slug == "" {
+ return "ls-" + hex.EncodeToString(sum[:6])
+ }
+ return "ls-" + slug + "-" + hex.EncodeToString(sum[:6])
+}
+
+// UnassignedID is where a trace with no supplied identity lands.
+//
+// It is named for the trace because there is nothing else to name it for, and
+// the name says plainly that ownership was not supplied rather than implying
+// the trace is one.
+func UnassignedID(traceID string) string {
+ sum := sha256.Sum256([]byte(traceID))
+ return "ls-unassigned-" + hex.EncodeToString(sum[:6])
+}
+
+// maxSlug keeps a session directory readable and well inside the shortest
+// path limit worth worrying about. The digest carries the identity, so
+// truncation here costs nothing but legibility.
+const maxSlug = 48
+
+// slugOf renders the namespace for a human reading a directory listing.
+//
+// Anything that is not a letter, a digit or a dash becomes a dash, letters
+// fold to lower case, and runs of dashes collapse. A value in a script with no
+// ASCII at all slugs to nothing, which is why the digest and not this is what
+// makes the name unique.
+func slugOf(values []string) string {
+ var b strings.Builder
+ for i, v := range values {
+ if i > 0 {
+ b.WriteByte('-')
+ }
+ for _, r := range v {
+ switch {
+ case r >= 'a' && r <= 'z', r >= '0' && r <= '9':
+ b.WriteRune(r)
+ case unicode.IsUpper(r) && r < unicode.MaxASCII:
+ b.WriteRune(unicode.ToLower(r))
+ default:
+ b.WriteByte('-')
+ }
+ }
+ }
+ out := b.String()
+ for strings.Contains(out, "--") {
+ out = strings.ReplaceAll(out, "--", "-")
+ }
+ out = strings.Trim(out, "-")
+ if len(out) > maxSlug {
+ out = strings.Trim(out[:maxSlug], "-")
+ }
+ return out
+}
diff --git a/internal/adapters/langsmith/identity_test.go b/internal/adapters/langsmith/identity_test.go
new file mode 100644
index 0000000..d7e5892
--- /dev/null
+++ b/internal/adapters/langsmith/identity_test.go
@@ -0,0 +1,118 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package langsmith
+
+import (
+ "path/filepath"
+ "strings"
+ "testing"
+)
+
+// TestAThreadKeyIsNeverAPath runs the keys the corpus actually carries through
+// the derivation. The storage root joins a session id into a path, so any one
+// of these reaching it unchanged would escape the root, hide the session from
+// enumeration, or fail to open on Windows.
+func TestAThreadKeyIsNeverAPath(t *testing.T) {
+ own := DefaultOwnership()
+ for _, key := range []string{
+ "../outside", "team/customer", "_hidden", "会话-1",
+ "..", ".", "con", "nul", "com1", "a\\b", "with space", strings.Repeat("x", 500),
+ } {
+ owner, ok := own.Owner("asz-harness", map[string]any{"thread_id": key})
+ if !ok {
+ t.Fatalf("%q: no owner", key)
+ }
+ id := StorageID(owner)
+ switch {
+ case id != filepath.Base(id):
+ t.Errorf("%q became a path: %q", key, id)
+ case strings.HasPrefix(id, "_"):
+ t.Errorf("%q would be skipped by enumeration: %q", key, id)
+ case strings.ContainsAny(id, `/\:*?"<>| `):
+ t.Errorf("%q kept a character a path cannot: %q", key, id)
+ case id == "." || id == "..":
+ t.Errorf("%q became %q", key, id)
+ case len(id) > 64:
+ t.Errorf("%q became %d characters", key, len(id))
+ }
+ }
+}
+
+// TestOwnersDoNotCollide is the reason a digest is carried at all. Two keys
+// that slug to the same readable text are different conversations and must
+// land apart.
+func TestOwnersDoNotCollide(t *testing.T) {
+ own := DefaultOwnership()
+ seen := map[string]string{}
+ for _, key := range []string{
+ "team/customer", "team-customer", "team_customer", "TEAM/CUSTOMER",
+ "会话-1", "会话-2", "../outside", "..-outside",
+ } {
+ owner, _ := own.Owner("p", map[string]any{"thread_id": key})
+ id := StorageID(owner)
+ if was, ok := seen[id]; ok {
+ t.Errorf("%q and %q both land in %q", was, key, id)
+ }
+ seen[id] = key
+ }
+}
+
+// TestProjectIsPartOfOwnership is the two-applications case: one thread key,
+// two projects, two conversations.
+func TestProjectIsPartOfOwnership(t *testing.T) {
+ own := DefaultOwnership()
+ one, _ := own.Owner("application-one", map[string]any{"thread_id": "123"})
+ two, _ := own.Owner("application-two", map[string]any{"thread_id": "123"})
+ if StorageID(one) == StorageID(two) {
+ t.Fatal("two applications sharing a thread key landed in one conversation")
+ }
+ // And an operator who says project does not own a conversation gets the
+ // other answer, deliberately.
+ merged := Ownership{Keys: own.Keys, Scope: []string{"thread"}}
+ a, _ := merged.Owner("application-one", map[string]any{"thread_id": "123"})
+ b, _ := merged.Owner("application-two", map[string]any{"thread_id": "123"})
+ if StorageID(a) != StorageID(b) {
+ t.Fatal("with project out of scope the two should merge")
+ }
+}
+
+// TestNoKeyIsNotAnIdentity: absence has to be reported, not filled in.
+func TestNoKeyIsNotAnIdentity(t *testing.T) {
+ own := DefaultOwnership()
+ for _, metadata := range []map[string]any{
+ nil, {}, {"thread_id": ""}, {"thread_id": 7}, {"other": "x"},
+ } {
+ if _, ok := own.Owner("p", metadata); ok {
+ t.Errorf("%v was treated as an identity", metadata)
+ }
+ }
+ if got := UnassignedID("trace-1"); got == UnassignedID("trace-2") {
+ t.Error("two unassigned traces landed together")
+ }
+}
+
+// TestKeysAreTriedInOrder: an application that sets two keys gets the first
+// the ownership names, not whichever the map iterates to.
+func TestKeysAreTriedInOrder(t *testing.T) {
+ own := DefaultOwnership()
+ owner, ok := own.Owner("p", map[string]any{
+ "session_id": "second", "thread_id": "first", "conversation_id": "third"})
+ if !ok || owner.Thread != "first" {
+ t.Fatalf("got %q, want first", owner.Thread)
+ }
+}
diff --git a/internal/adapters/langsmith/langsmith.go b/internal/adapters/langsmith/langsmith.go
new file mode 100644
index 0000000..ebe7007
--- /dev/null
+++ b/internal/adapters/langsmith/langsmith.go
@@ -0,0 +1,82 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+// Package langsmith is the langsmith-ingest adapter: a receiver the LangSmith
+// tracing client is pointed at, and the dialect that reads what it sends.
+//
+// An application built on LangChain or LangGraph already carries this client,
+// because langsmith is a dependency of langchain-core. So four environment
+// variables are the whole integration, and nothing in the application changes:
+//
+// LANGSMITH_TRACING=true
+// LANGSMITH_ENDPOINT=http://<asz>:1985
+// LANGSMITH_API_KEY=<anything this receiver accepts>
+// LANGSMITH_PROJECT=<a name>
+//
+// This is interoperation, not impersonation: the receiver accepts what that
+// client sends, carries none of its code, and claims no affiliation.
+//
+// # What the client does, measured
+//
+// The fixtures under tests/apps/langchain hold what one client actually sent.
+// Four of its behaviours shape everything here.
+//
+// It asks GET /info before every send, and takes the batch configuration from
+// the answer. Compression is part of that answer: the client compresses only
+// when instance_flags.zstd_compression_enabled is set, so a receiver that does
+// not advertise it never has to read zstd.
+//
+// It posts runs that have not finished. Six of eighteen runs in one capture
+// arrived with no end time and were completed twelve seconds later by a patch.
+// So a run arrives one or more times, and a receiver that expects a finished
+// run will miss the work in progress that only this transport carries.
+//
+// It retries a rejected batch with the same bytes. Delivery is at-least-once
+// and deduplication is the receiver's job.
+//
+// It sends inputs and outputs as their own parts rather than as attributes, so
+// a 20,908-byte command and a 192,000-byte result arrive whole.
+package langsmith
+
+const (
+ // Name is the adapter's name in the configuration.
+ Name = "langsmith-ingest"
+ // Dialect is whose vocabulary the landed records were read in.
+ Dialect = "langsmith/1"
+ // Version is the adapter's contract version.
+ Version = "0.1.0"
+ // RuntimeName is what the landed data is attributed to by default. The
+ // wire is LangSmith's; what produced it is LangChain, and an application
+ // may say otherwise in the configuration.
+ RuntimeName = "LangChain"
+
+ // SourceRuns names the spool files this adapter writes: one per accepted
+ // request, holding the body as it arrived.
+ SourceRuns = "runs"
+
+ // MaxRequest bounds one request. The client is told 20 MiB in the batch
+ // configuration it reads from GET /info; this is the hard stop above it,
+ // since nothing makes a client obey what it was told.
+ MaxRequest = 64 << 20
+
+ // BatchSizeLimit and BatchSizeLimitBytes are what GET /info tells the
+ // client to send in one request. They are its numbers, not ours: the
+ // client's own defaults, repeated so a receiver that answers is not
+ // silently changing how it batches.
+ BatchSizeLimit = 100
+ BatchSizeLimitBytes = 20 << 20
+)
diff --git a/internal/adapters/langsmith/wire.go b/internal/adapters/langsmith/wire.go
new file mode 100644
index 0000000..3f1102b
--- /dev/null
+++ b/internal/adapters/langsmith/wire.go
@@ -0,0 +1,192 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package langsmith
+
+import (
+ "encoding/json"
+ "fmt"
+ "io"
+ "mime"
+ "mime/multipart"
+ "sort"
+ "strings"
+)
+
+// Operation is one arrival of evidence about one run.
+//
+// The client names each part <op>.<run id>[.<field>], sending the run's
+// envelope under the bare name and anything large beside it under its own.
+// An Operation gathers those back into one thing while keeping each field's
+// bytes as they arrived, because a landed record proves its source by the
+// digest of those bytes and not of something re-encoded.
+type Operation struct {
+ // Op is "post" for a run's first arrival and "patch" for a later one.
+ // The client sends both, and a run that finished before the batch went
+ // out arrives as a post that is already complete.
+ Op string
+ // RunID is the run this arrival concerns. It is shared by every arrival
+ // about that run, which is what makes it the call rather than the
+ // receipt.
+ RunID string
+ // Envelope is the run object: id, trace_id, parent_run_id, dotted_order,
+ // run_type, name, start_time, end_time, tags, session_name.
+ Envelope json.RawMessage
+ // Fields are the parts sent out of band, by name: inputs, outputs,
+ // events, extra, serialized, error. Bytes as received.
+ Fields map[string]json.RawMessage
+ // Order is the part's position in the request, counting from 1. It is
+ // what a landed record reports as its position in its source, since a
+ // request has no lines to count.
+ Order int
+ // Bytes is what this arrival took on the wire, envelope and fields
+ // together.
+ Bytes int
+}
+
+// Feedback and Attachment parts are accepted and counted, never landed.
+// Feedback is a score on a run rather than evidence of the conversation, and
+// an attachment needs a measurement of how large and how often before it has a
+// place. Dropping them silently would be the wrong kind of quiet, so the
+// receiver counts them and the status says so.
+const (
+ partFeedback = "feedback"
+ partAttachment = "attachment"
+)
+
+// Request is everything one accepted request carried.
+type Request struct {
+ Operations []Operation
+ // Feedback and Attachments count what was accepted and not landed.
+ Feedback int
+ Attachments int
+}
+
+// runFields are the field names the client sends out of band. A name outside
+// this set is kept anyway, under its own key: a client that grows a field must
+// not lose it here, and the dialect can say it did not understand rather than
+// the receiver pretending it never came.
+var runFields = map[string]bool{
+ "inputs": true, "outputs": true, "events": true,
+ "extra": true, "serialized": true, "error": true,
+}
+
+// ParseMultipart reads a multipart ingest body into the operations it carries.
+//
+// The part name is the whole protocol here: "post.<uuid>" is a run's envelope,
+// "post.<uuid>.inputs" is that run's inputs, and the two must end up in one
+// Operation however they were ordered in the body.
+func ParseMultipart(body io.Reader, contentType string) (*Request, error) {
+ _, params, err := mime.ParseMediaType(contentType)
+ if err != nil {
+ return nil, fmt.Errorf("langsmith: content type: %w", err)
+ }
+ boundary := params["boundary"]
+ if boundary == "" {
+ return nil, fmt.Errorf("langsmith: multipart body with no boundary")
+ }
+ reader := multipart.NewReader(body, boundary)
+ byRun := map[string]*Operation{}
+ var order []string
+ out := &Request{}
+ position := 0
+ for {
+ part, err := reader.NextPart()
+ if err == io.EOF {
+ break
+ }
+ if err != nil {
+ return nil, fmt.Errorf("langsmith: read part: %w", err)
+ }
+ position++
+ name := part.FormName()
+ data, err := io.ReadAll(part)
+ part.Close()
+ if err != nil {
+ return nil, fmt.Errorf("langsmith: read part %q: %w", name, err)
+ }
+ kind, runID, field, ok := splitPartName(name)
+ if !ok {
+ return nil, fmt.Errorf("langsmith: part %q is not named <op>.<run id>[.<field>]", name)
+ }
+ switch kind {
+ case partFeedback:
+ out.Feedback++
+ continue
+ case partAttachment:
+ out.Attachments++
+ continue
+ case "post", "patch":
+ default:
+ return nil, fmt.Errorf("langsmith: part %q names an unknown operation %q", name, kind)
+ }
+ key := kind + "." + runID
+ op := byRun[key]
+ if op == nil {
+ op = &Operation{Op: kind, RunID: runID, Fields: map[string]json.RawMessage{},
+ Order: position}
+ byRun[key] = op
+ order = append(order, key)
+ }
+ if field == "" {
+ op.Envelope = json.RawMessage(data)
+ } else {
+ op.Fields[field] = json.RawMessage(data)
+ }
+ op.Bytes += len(data)
+ }
+ for _, key := range order {
+ op := byRun[key]
+ if len(op.Envelope) == 0 {
+ // A field with no envelope is a part naming a run this request
+ // never described. Keeping it would mean landing a record with no
+ // run to attach it to.
+ return nil, fmt.Errorf("langsmith: %s carries fields but no run", key)
+ }
+ out.Operations = append(out.Operations, *op)
+ }
+ sort.SliceStable(out.Operations, func(i, j int) bool {
+ return out.Operations[i].Order < out.Operations[j].Order
+ })
+ return out, nil
+}
+
+// splitPartName reads "<op>.<run id>[.<field>]".
+//
+// A run id holds no dot and a field name holds none either, so the first two
+// dots are the whole grammar. Anything else is refused rather than guessed at:
+// a name this does not understand is a client that changed, which has to be
+// loud.
+func splitPartName(name string) (kind, runID, field string, ok bool) {
+ first := strings.IndexByte(name, '.')
+ if first <= 0 {
+ return "", "", "", false
+ }
+ kind, rest := name[:first], name[first+1:]
+ second := strings.IndexByte(rest, '.')
+ if second < 0 {
+ if rest == "" {
+ return "", "", "", false
+ }
+ return kind, rest, "", true
+ }
+ runID, field = rest[:second], rest[second+1:]
+ if runID == "" || field == "" {
+ return "", "", "", false
+ }
+ return kind, runID, field, true
+}
diff --git a/internal/adapters/langsmith/wire_test.go b/internal/adapters/langsmith/wire_test.go
new file mode 100644
index 0000000..dd54a19
--- /dev/null
+++ b/internal/adapters/langsmith/wire_test.go
@@ -0,0 +1,206 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package langsmith
+
+import (
+ "bytes"
+ "encoding/json"
+ "os"
+ "path/filepath"
+ "sort"
+ "testing"
+)
+
+// corpus is the demo application's captures: what one real client sent.
+const corpus = "../../../tests/apps/langchain/testdata"
+
+type capturedMeta struct {
+ Headers map[string]string `json:"headers"`
+ Parts []struct {
+ Name string `json:"name"`
+ Bytes int `json:"bytes"`
+ } `json:"parts"`
+}
+
+// captured walks the corpus, yielding each request's meta and body.
+func captured(t *testing.T, fn func(kase, request string, meta capturedMeta, body []byte)) {
+ t.Helper()
+ cases, err := os.ReadDir(corpus)
+ if err != nil {
+ t.Skipf("no corpus at %s: %v", corpus, err)
+ }
+ seen := 0
+ for _, c := range cases {
+ if !c.IsDir() {
+ continue
+ }
+ requests, err := os.ReadDir(filepath.Join(corpus, c.Name()))
+ if err != nil {
+ t.Fatalf("%s: %v", c.Name(), err)
+ }
+ for _, r := range requests {
+ dir := filepath.Join(corpus, c.Name(), r.Name())
+ raw, err := os.ReadFile(filepath.Join(dir, "meta.json"))
+ if err != nil {
+ continue
+ }
+ var meta capturedMeta
+ if err := json.Unmarshal(raw, &meta); err != nil {
+ t.Fatalf("%s: meta: %v", dir, err)
+ }
+ body, err := os.ReadFile(filepath.Join(dir, "body.bin"))
+ if err != nil {
+ t.Fatalf("%s: body: %v", dir, err)
+ }
+ seen++
+ fn(c.Name(), r.Name(), meta, body)
+ }
+ }
+ if seen == 0 {
+ t.Fatal("the corpus holds no requests")
+ }
+}
+
+// TestParsesWhatTheClientSent reads every captured request with the parser the
+// receiver uses. A part this cannot name is a client that changed, and the
+// corpus is the only place that shows up before a deployment does.
+func TestParsesWhatTheClientSent(t *testing.T) {
+ operations, requests := 0, 0
+ captured(t, func(kase, request string, meta capturedMeta, body []byte) {
+ got, err := ParseMultipart(bytes.NewReader(body), meta.Headers["Content-Type"])
+ if err != nil {
+ t.Fatalf("%s/%s: %v", kase, request, err)
+ }
+ // Every part of the request is accounted for: one envelope or one
+ // field each, plus whatever was accepted and not landed.
+ parts := 0
+ for _, op := range got.Operations {
+ parts += 1 + len(op.Fields)
+ if len(op.Envelope) == 0 {
+ t.Fatalf("%s/%s: %s has no envelope", kase, request, op.RunID)
+ }
+ }
+ parts += got.Feedback + got.Attachments
+ if parts != len(meta.Parts) {
+ t.Fatalf("%s/%s: read %d parts, the request carried %d",
+ kase, request, parts, len(meta.Parts))
+ }
+ operations += len(got.Operations)
+ requests++
+ })
+ t.Logf("%d requests, %d operations", requests, operations)
+}
+
+// run is what a test needs from a run's envelope.
+type run struct {
+ ID string `json:"id"`
+ Trace string `json:"trace_id"`
+ Parent string `json:"parent_run_id"`
+ Dotted string `json:"dotted_order"`
+ Type string `json:"run_type"`
+ Name string `json:"name"`
+ Start string `json:"start_time"`
+ End string `json:"end_time"`
+ Session string `json:"session_name"`
+}
+
+func operationsOf(t *testing.T, kase string) []Operation {
+ t.Helper()
+ var all []Operation
+ captured(t, func(c, request string, meta capturedMeta, body []byte) {
+ if c != kase {
+ return
+ }
+ got, err := ParseMultipart(bytes.NewReader(body), meta.Headers["Content-Type"])
+ if err != nil {
+ t.Fatalf("%s/%s: %v", c, request, err)
+ }
+ all = append(all, got.Operations...)
+ })
+ return all
+}
+
+func decode(t *testing.T, op Operation) run {
+ t.Helper()
+ var r run
+ if err := json.Unmarshal(op.Envelope, &r); err != nil {
+ t.Fatalf("%s: envelope: %v", op.RunID, err)
+ }
+ return r
+}
+
+// TestRunsArriveUnfinished holds the fact the whole update contract rests on.
+// If a client ever stopped posting work in progress, the contract would be
+// carrying weight nothing needs, and this is where that shows.
+func TestRunsArriveUnfinished(t *testing.T) {
+ for _, kase := range []string{"slow-tool", "abandoned-run"} {
+ open, patches := 0, 0
+ for _, op := range operationsOf(t, kase) {
+ if op.Op == "patch" {
+ patches++
+ continue
+ }
+ if decode(t, op).End == "" {
+ open++
+ }
+ }
+ if open == 0 {
+ t.Errorf("%s: no run arrived unfinished", kase)
+ }
+ t.Logf("%s: %d unfinished on arrival, %d completed later", kase, open, patches)
+ }
+ // A process that died mid-turn completes nothing: the open runs are all
+ // the evidence there will ever be, and the receiver must not wait for
+ // more.
+ for _, op := range operationsOf(t, "abandoned-run") {
+ if op.Op == "patch" {
+ t.Fatal("abandoned-run: a run was completed; the case no longer shows a crash")
+ }
+ }
+}
+
+// TestThreadKeysAreNotPaths keeps the identity contract honest. These are not
+// hypothetical: they are what the corpus carries, because an application
+// chooses its own thread key and nothing stops it choosing these.
+func TestThreadKeysAreNotPaths(t *testing.T) {
+ var keys []string
+ for _, op := range operationsOf(t, "unsafe-thread-key") {
+ var extra struct {
+ Metadata map[string]any `json:"metadata"`
+ }
+ if raw, ok := op.Fields["extra"]; ok {
+ _ = json.Unmarshal(raw, &extra)
+ }
+ if v, ok := extra.Metadata["thread_id"].(string); ok {
+ keys = append(keys, v)
+ }
+ }
+ sort.Strings(keys)
+ want := map[string]bool{"../outside": false, "team/customer": false,
+ "_hidden": false, "会话-1": false}
+ for _, k := range keys {
+ if _, ok := want[k]; ok {
+ want[k] = true
+ }
+ }
+ for k, found := range want {
+ if !found {
+ t.Errorf("the corpus no longer carries the thread key %q", k)
+ }
+ }
+}