You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "ArnauSB (via GitHub)" <gi...@apache.org> on 2026/10/06 15:44:39 UTC
[PR] Name what the runtime sent the model, and let a view withhold it [skywalking-ai-sessionizer]
ArnauSB opened a new pull request, #51:
URL: https://github.com/apache/skywalking-ai-sessionizer/pull/51
## What happens
Claude Code writes the system prompt and the tool schemas it sent the model into the transcript. A
`prompt_snapshot` attachment holds the system prompt, and often the tool schemas. A
`deferred_tools_record` attachment lists the tools offered on demand, each with its input schema. A
runtime built on the Claude Agent SDK writes the snapshot too. asz landed each as a
`context.injection` step, and nothing on the landed record said what it carried.
On one machine's Claude Code history, 44 sessions from Claude Code 2.1.259 to 2.1.286, all 736
snapshots held the prompt and 369 held the tools, and 244 deferred tools records held tool schemas.
On 47 conversations of an Agent SDK runtime, 100 snapshots held the prompt and 50 of them the tools
too.
asz view served all of it to every reader, from three places: the step's text in the asz.view
document, the record endpoint, which returns a record whole, and the files endpoint, which returned
any landed file whole by its sequence, transcripts included.
## Why it is a problem
Someone who shows conversations to the people an agent served may need to keep the system prompt
and the tool schemas from them, while still seeing them. asz has no notion of who is asking, and
should not have one. The identity lives in whatever puts the page in front of a person. But a
reader had nothing to withhold by except the text or the size, which is a guess, and asz view had no
way to withhold anything.
## What this does
### The adapters name what the runtime sent the model
- Two flags beside `injected`, `system_prompt` and `tool_schemas`, go on the landed record. They take
no bit of the index, and the assembler and the round chain are unchanged. The view already copies
a record's flags onto its step.
- **Claude Code.** A snapshot is named `system_prompt` by its type, since every one measured held
the prompt. It is named `tool_schemas` when a key whose name says tool holds something. A deferred
tools record is named `tool_schemas` when it holds tools. A line of either type that does not
decode lands whole and is named for everything its type could carry. The type and the keys
decide, never the text or the size.
- **LangChain.** The system message lands in the provider bodies, and the tools land nowhere. Two
records can hold a model call's request: the envelope of a call that arrived before it finished,
and the first input of a trace whose root is a model call. Each is named by whether it holds
inputs. A decorated function's values and a graph's first input are the application's own, so
they are named only by the shapes the LangChain page lists.
- The names apply to newly landed files. A root landed before keeps its records unnamed, and
collecting it again into a new root names them.
- An injection that carries either name never names a talk or a stream.
### asz view withholds by name
- `view.hide` in the configuration lists the names withheld from every reader of the instance. A
`hide` parameter on the document, record and files endpoints adds names for one request and never
removes one. A host that serves the API through its own route adds it per reader.
- A withheld step keeps its node, its flags and its size, loses its text, and says `omitted`. A
withheld record keeps its envelope and flags, and every part loses its content. `summary.withheld`
counts the records withheld under each name, zeros included, so a filter over a root landed before
the names shows that it found nothing.
- The document is built from records withheld as they are read. A step's text, a tool's result, and
a talk's label and reply all follow the rule.
- A reader that withholds anything is served no provider body, by file or by record, since a request
carries both again.
- The files endpoint serves provider body files only. It served any landed file whole.
- A name that cannot be withheld, the parameter spelled another way, such as `Hide` or `hide[]`, and
a query that does not parse are refused with 400. Each would otherwise read as no `hide` at all.
asz's own page passes a `hide` in its address on to every call it makes.
- The asz.view format page states the rules, so a server that mirrors the format withholds the same
way.
### Configuration
- The `view` section is read strictly, because a misspelled key there would show everything to
everyone. A refusal gives the key and its line.
- A `hide` written outside `view` that names a flag `view` does not withhold is refused, such as a
top-level `view.hide` or a `hide` under a misspelled `view`.
- A file with a later YAML document that holds anything is refused, since only the first is read.
- When `hide` is set, `asz view` and `asz server` say so as they start.
### Other changes
- The LangChain demo application and its captures use a neutral example. The captures were recorded
again with `make langchain-capture`, with the same requests, runs and threads, and the sizes on
the LangChain page were measured again.
- The contributing guide and the docs index point to the SkyWalking issue tracker, since this
repository's issues are turned off.
## Not done here
- **A notion of a user in asz.** A trusted header on one instance would be a claim asz cannot verify.
One instance per audience, each behind the deployment's own authentication, and a host that passes
`hide` per reader, cover it.
- **Leaving the material out at landing.** Landed files are write-once, and whoever runs asz would
lose it too.
- **Hiding in the renderer alone.** The record and files endpoints would still return the bytes.
- **Other context the runtime injects,** such as instruction files, skill and agent listings, MCP
server instructions and reminders. These are not named. A later flag could name them by the same
rule.
- **A tool's result that repeats part of the system prompt or a schema,** such as a tool that returns
a file the prompt also quotes, or a validation error. It is the tool's own output, and naming it
would mean reading its text.
- **A Horizon string for a withheld step.** The renderer draws it as "Content omitted", with a note
that says the source did not supply the content, which is not the reason here. That is an upstream
change.
## Checks
- `make check` and `make scenarios` pass.
- The scenarios `prompt-snapshot` and `prompt-snapshot-withheld` run in both formats. Every view
expectation now checks that no talk is named by an injection carrying either name. The `hide`
expectation checks that no withheld text comes back as a talk's label or reply.
- View tests cover the HTTP layer: the parameter's forms and refusals, the record endpoint, and the
files endpoint's 400 and 403. Configuration tests cover files that are refused and files that
load, with the values they set.
- 47 conversations of an Agent SDK runtime, collected fresh with this branch:
- 100 records are named `system_prompt` and 50 `tool_schemas`, all snapshots.
- Every document is identical to what `main` builds over the same root, apart from the new
`withheld` key.
- With both names withheld, all 4,833 records were read back. No named record shows its content,
and all 57 file requests are refused. Phrases of a named record appear only in 35 tool results,
from a tool that returns a file the system prompt also quotes.
- A root landed before this branch is served the same, apart from the `withheld` key, and withholding
over it counts zeros.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Name what the runtime sent the model, and let a view withhold it [skywalking-ai-sessionizer]
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng merged PR #51:
URL: https://github.com/apache/skywalking-ai-sessionizer/pull/51
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Name what the runtime sent the model, and let a view withhold it [skywalking-ai-sessionizer]
Posted by "ArnauSB (via GitHub)" <gi...@apache.org>.
ArnauSB commented on PR #51:
URL: https://github.com/apache/skywalking-ai-sessionizer/pull/51#issuecomment-6035474962
Thanks for measuring it. Both points are addressed in 2ef45ce and 62b5ea6.
**`summary.withheld`.** I reproduced the gap with a prompt run inside `@traceable` and no model call: two named records, no step, and a count of 0. The count is now taken when the view loads a conversation, in the pass that already reads every landed record for its time. A record counts whether or not a step stands on it, which is the same set the record endpoint withholds. Each record counts once by its id. Claude Code replays its last records before a reset under the same ids, and counting by position would have added 181 such copies on 45 local sessions. Counted by id, the Claude Code numbers stay as they were. A new LangChain test covers your case, and `prompt-snapshot-withheld` now writes two named records twice and checks that each counts once.
**Configuration.** `checkHides`, `readsAsHide` and `checkHide` are removed, about 100 lines plus their tests. The strict read of `view` stays, and so does the refusal of a second YAML document, since only the first one is read. A `hide` anywhere else is now ignored like any other unknown key, and the configuration page says so.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Name what the runtime sent the model, and let a view withhold it [skywalking-ai-sessionizer]
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on PR #51:
URL: https://github.com/apache/skywalking-ai-sessionizer/pull/51#issuecomment-6030637784
Thanks for this. I measured it on my local Claude Code history (65 sessions) and on LangChain
captures, including the JSON upload, the completion model at a trace root and the `@traceable`
paths. Withholding works as described, and a reader who hides nothing sees no change. One gap to fix:
**`summary.withheld` undercounts on LangChain.** `withheldCounts` counts only the records the
document reads for its steps. A named LangChain framework record has no step, so the record endpoint
withholds it but the count misses it. For a Prompt step inside `@traceable`, 2 records are withheld
and the count shows 0. Nothing leaks, but the docs say a zero means the filter found nothing.
Claude Code is not affected, because every named snapshot is drawn as a step.
Not blocking: the configuration checks, `checkKeys`, `checkHides` and `readsAsHide`, take about 190
lines. A strict read of the `view` section alone covers a misspelled key there.
The OAP side, mirroring these rules, I will deal with later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org