You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@maka.apache.org by "me2seeks (via GitHub)" <gi...@apache.org> on 2026/09/03 07:53:51 UTC
[I] History compaction always fails open on kimi-coding-plan/k3-256k: summarizer instruction as system prompt is ignored and breaks prefix cache [maka]
me2seeks opened a new issue, #4634:
URL: https://github.com/apache/maka/issues/4634
## What happened
Mid-turn history compaction never succeeds on the `kimi-coding-plan` / `k3-256k` connection, so long sessions grow past the context window and every agent-loop step slows down (10–30+ s per model call, request payload growing ~5 KB per step).
The capacity trigger itself works: the estimate anchors on real provider usage (~308K input tokens against a 262,144 window, high-water ≈ 245,760) and fires with reason `context_limit`. But every compaction attempt fails open with `failOpenReason: malformed_summary_missing_section` — in one send I counted **51 consecutive failed-open decisions, all for this reason**. Across the workspace there are **180 `history_compact` model calls on `k3-256k` and zero checkpoints** since 2026-08-20; the only historical checkpoints predate the sectioned-summary contract (#3029) and are legacy free-form text.
Consequences observed in production data:
- Sessions run at 250K–315K input tokens per step (~900 KB request payloads); TTFT 6–12 s, per-step latency 10–30+ s even with ~99% prefix-cache hits.
- Every turn wastes 1–2 summarizer calls that are guaranteed to be discarded (see root cause), and those calls are themselves slow: 8–43 s typical, up to **88 s** (see cache section below).
- Provider errors are not circuit-broken: one session logged **15 consecutive failed summarizer calls over ~47 minutes** before its main request.
### Root cause: `k3-256k` ignores the summarization instruction when it is a system prompt
`buildLlmHistorySummarizer` sends `SUMMARIZATION_SYSTEM_PROMPT` as the system prompt followed by the conversation. `k3-256k` is an agentic coding model (reasoning always on, default effort `max`); in this shape it does not switch into summarizer mode — it continues the conversation as the assistant, returning a stub (`content: null` / 7–72 output tokens, `end_turn`) that can never satisfy the required `## Goal / ## Progress / ## Next Steps / ## Critical Context` sections. The #3029 validator is working as designed; the request shape is incompatible with this model.
Reproduced against the production endpoint (`api.kimi.com/coding`, same key) with a **367-token toy conversation**, so input size is not the trigger:
| Request shape | k3-256k result |
|---|---|
| Anthropic wire, instruction in `system` field (maka's current shape) | `content: null`, 7 output tokens, `stop_reason: end_turn` — no summary at all |
| OpenAI-chat wire, instruction as system message | Ignores the format, answers the user's question as the assistant (1,567 tokens, no section headings) |
| Either wire, **identical instruction as a trailing user message** after the conversation | Fully compliant sectioned summary (all 4 required sections, in order, substantive content) |
This also explains why the same account/model compacts fine in other agents (e.g. OpenCode): they deliver the compaction instruction as a user message at the end of the conversation, not as a system prompt.
### Side effect: the system-prompt swap also destroys prefix caching for summarizer calls
Because the summarizer request starts with a *different* system prompt, it shares no prefix with the main-loop requests and pays a full uncached pass over the entire conversation. Observed:
- Main calls: cacheRead ≈ 316K/322K input (99.8% hit), 11–33 s.
- `history_compact` first attempts: `cacheReadInputTokens: 0` on 155K–253K inputs → 39 s, 65 s, **88 s** per wasted call. (The repair retry immediately after hits the first attempt's fresh cache: 180,736/181,894 cache-read, 17 s.)
Delivering the instruction as a trailing user message would make the summarizer request share the main loop's whole cached prefix (~99% hits on the first attempt too), so the fix for instruction-following also fixes this.
### Contributing defects found while tracing
1. **The summarizer never sets `maxOutputTokens`** — `generateSummary` calls `generateText` without it, while main-loop requests resolve 131,072 via `selectedModelMaxOutputTokens`. On reasoning models an unset output budget is at best unpredictable.
2. **The failure circuit only covers malformed-summary reasons.** `summarizeWithFailureCircuit` records only `isMalformedHistoryCompactSummaryReason`, and per-send `state.malformedSummaryFailure` latches only those too; `provider_error` / `output_length` therefore retry on *every* step — this produced the 15-call retry storm above.
3. **The summarizer input budget is enormous** (`capacity − reserve` ≈ 245K tokens), so a fold at high context feeds ~180–235K tokens into the summarizer — expensive, slow, and further degrades compliance, feeding the vicious cycle.
## How to reproduce
1. Run a session on `kimi-coding-plan` / `k3-256k` until it approaches the high-water mark.
2. Inspect the `token_usage` message diagnostics: `contextBudget.compactionDecisions` fills with `{stage: activeStep, decision: failedOpen, phase: mid_turn, reason: context_limit, failOpenReason: malformed_summary_missing_section}`.
3. `model_call_attempt_recorded` events with `callKind: history_compact` show `outputTokens` of 40–72 with `finishReason: end_turn` — far too short to contain the four required sections.
4. Minimal API-level repro: POST to `https://api.kimi.com/coding/v1/messages` with model `k3-256k`, `system` = `SUMMARIZATION_SYSTEM_PROMPT`, and any short user/assistant exchange → the model returns `content: null` / a few tokens instead of the sectioned summary. Moving the same instruction into a trailing user message returns a compliant summary.
## Environment
- Maka commit: `898b86d6b7` (source build)
- OS: Linux 7.0.0-30-generic x86_64
- Surface: Runtime Host / TUI
- Node.js: v26.3.0
- Connection: `kimi-coding-plan`, model `k3-256k` (contextWindow 262144, reasoning always on, anthropic-messages wire)
## Logs, screenshots, or additional context
Directions I'd like maintainer input on (happy to split into separate issues/PRs):
1. **Deliver the summarization instruction as a trailing user message** after the conversation (verified working on both wires for k3-256k, and restores prefix-cache reuse) — smallest change, in `buildLlmHistorySummarizer`. Open question: is the system-prompt placement load-bearing for other providers, or can this be unconditional?
2. Pass an explicit, bounded `maxOutputTokens` for summarizer calls.
3. Extend the failure circuit to `provider_error` / `output_length` so provider-side failures don't retry every step.
4. Bound the summarizer input budget well below `capacity − reserve` (e.g. fold in increments via the existing rolling `previousCheckpoint` mechanism).
Evidence is from the local workspace store: `runtime.sqlite` (`core_agent_run_events`, `usage_model_call_attempts`, `token_usage` diagnostics) and per-request capture artifacts. I can share redacted excerpts if useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org
Re: [I] History compaction always fails open on kimi-coding-plan/k3-256k: summarizer instruction as system prompt is ignored and breaks prefix cache [maka]
Posted by "Astro-Han (via GitHub)" <gi...@apache.org>.
Astro-Han closed issue #4634: History compaction always fails open on kimi-coding-plan/k3-256k: summarizer instruction as system prompt is ignored and breaks prefix cache
URL: https://github.com/apache/maka/issues/4634
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org
Re: [I] History compaction always fails open on kimi-coding-plan/k3-256k: summarizer instruction as system prompt is ignored and breaks prefix cache [maka]
Posted by "Astro-Han (via GitHub)" <gi...@apache.org>.
Astro-Han commented on issue #4634:
URL: https://github.com/apache/maka/issues/4634#issuecomment-5717640900
Fixed in #4653 (`6ab725176`). `packages/runtime/src/history-compact-summarizer.ts:159-178` no longer passes the sectioned summarizer prompt as provider `instructions`; it rides as the trailing user message together with the summary request, so the request still ends with an imperative and no system-role block precedes the conversation. That removes both symptoms reported here: the permanent `malformed_summary_missing_section` fail-open on `k3-256k`, and the uncached pass every discarded attempt paid. Closing as done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org
Re: [I] History compaction always fails open on kimi-coding-plan/k3-256k: summarizer instruction as system prompt is ignored and breaks prefix cache [maka]
Posted by "testikun (via GitHub)" <gi...@apache.org>.
testikun commented on issue #4634:
URL: https://github.com/apache/maka/issues/4634#issuecomment-5537230076
Stepping aside after reassessing the verification requirements. A responsible fix needs provider-specific Kimi production validation in addition to mocked request-shape coverage, which I cannot complete in the current environment. I have not created a branch, changed code, or opened a PR, so this issue is available for another contributor.
中文：该问题需要 Kimi 生产环境验证，目前无法完整完成，因此取消认领；尚未开始代码或 PR。
_Update posted with OpenAI Codex assistance on behalf of @testikun._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org
Re: [I] History compaction always fails open on kimi-coding-plan/k3-256k: summarizer instruction as system prompt is ignored and breaks prefix cache [maka]
Posted by "testikun (via GitHub)" <gi...@apache.org>.
testikun commented on issue #4634:
URL: https://github.com/apache/maka/issues/4634#issuecomment-5537201268
I would like to take a bounded first slice of this issue.
My initial scope will be the provider-compatible summarizer request shape and retry guard:
- keep the existing sectioned-summary validation fail-closed;
- preserve the main request prefix and deliver the compaction instruction as a trailing user message for the verified `kimi-coding-plan / k3-256k` path, without changing unrelated providers blindly;
- pass an explicit bounded `maxOutputTokens` value for the summarizer call;
- extend focused request-shape and compaction tests to cover the Kimi path, prefix preservation, valid section output, and bounded failed-open behavior.
I will leave the larger input-budget redesign and general provider-error circuit policy as separate follow-ups unless the first PR establishes a provider-neutral invariant.
中文摘要：我认领首个有界切片，先修复 Kimi k3-256k 的摘要指令位置、前缀缓存复用和输出上限，并用确定性测试守住现有 fail-closed 契约。
_Claimed with OpenAI Codex assistance on behalf of @testikun._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org
Re: [I] History compaction always fails open on kimi-coding-plan/k3-256k: summarizer instruction as system prompt is ignored and breaks prefix cache [maka]
Posted by "ggbdpq (via GitHub)" <gi...@apache.org>.
ggbdpq commented on issue #4634:
URL: https://github.com/apache/maka/issues/4634#issuecomment-5543153105
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org