[PR] Keep only the metadata of any idle conversation, sent or not [skywalking-ai-sessionizer]

["wu-sheng (via GitHub)" <gi...@apache.org>]

wu-sheng opened a new pull request, #71:
URL: https://github.com/apache/skywalking-ai-sessionizer/pull/71

   Retention removed the landed files only of a conversation that was sent. One collected with no endpoint, or not sent yet, kept every byte, so a root that only collects grew for ever. Whether to remove now follows the conversation's idle time and the setting alone, never an endpoint or what was sent.
   
   ### What changes
   - **`storage.retention` becomes a section.** Its first setting is `metadata_only_after`: how long a conversation may be idle before only its metadata is kept, to save disk. What was said and run is removed from every landed file. Who did what, when, in what order and at what cost stays, so new activity continues the same conversation. `-1` is off, the default, and only off keeps everything. The level that removes the session itself, `retire_after`, comes in its own change together with what it does, so no setting exists before its behavior.
   - **The old single value is refused.** `retention: 30d` is a load error that names `storage.retention` as a section, not a Go type.
   - **A conversation never sent keeps only its metadata too.** Its content is then gone everywhere. Its skeletons say that no receiver confirmed them, and name an owner only where a send was tried. A later send carries its rounds and none of its removed files.
   - **Late metrics say what they cannot derive.** Metrics are derived only while an endpoint is set. A conversation that kept only its metadata with none set reaches a deriver enabled later with none of its executions counted. A skeleton keeps no execution record's protocol or server, so no MCP call is derived from it. The first pass of a process that saves its state now says, for each session, how many distinct execution observations its skeletons keep that no pass counted:
     ```
       metrics: <session>: 4 execution observations lack retained details; their MCP metrics cannot be derived
     ```
     Both the pipeline and `asz push` print it. The count reads only execution files. A file it cannot read keeps its session for another pass. A pass whose state was not saved says the count again on the next pass.
   - **Docs.** `asz.yaml`, the configuration page, the storage-root Retention section, the command-line page, the skeleton format (an absent `owner`), the scenario guide, the glossaries and the changelog. Each statement that a session is never removed now says "whole", since retention removes content in any root without `_scenario/`.
   
   ### Tests
   - `tests/chain`, `TestASessionKeepsOnlyItsMetadataSentOrNot`, five cases:
     - no endpoint;
     - an endpoint and nothing sent;
     - a send no answer confirmed;
     - the same with the endpoint then gone;
     - part of it sent with the endpoint then gone.
   
     Each reads back the skeletons' receiver states and owner, `asz verify`, the document route and the files route. The cases with an endpoint push afterwards.
   - `tests/chain`, `TestLateMetricsSayWhatTheyCannotDerive`: over `mcp-calls`, the tokens match an intact root and no MCP point comes from skeletons.
   - `internal/metrics`, the count:
     - a non-MCP observation, two observations of one call, and one landed twice;
     - one counted before the removal, also with the state lost and its receipt read again;
     - a text part, an empty id, and a file named for executions that says it is a transcript;
     - an unsaved pass, an unreadable skeleton tried again, and a transcript skeleton never opened.
   - `cmd/asz`:
     - a pipeline with no endpoint, read back with `asz show` and `asz verify`;
     - the report from the pipeline and from `asz push`, and the quiet next pass.
   - `internal/config`: the section through `Load`, including the old single value.
   
   Each new guard was proven by reverting it. `make test`, the JSON v1 step (`GOEXPERIMENT=nojsonv2`), `make scenarios`, lint, and the Windows build and vet pass.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Keep only the metadata of any idle conversation, sent or not [skywalking-ai-sessionizer]

["wu-sheng (via GitHub)" <gi...@apache.org>]

wu-sheng merged PR #71:
URL: https://github.com/apache/skywalking-ai-sessionizer/pull/71


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org