You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@maka.apache.org by "Astro-Han (via GitHub)" <gi...@apache.org> on 2026/09/16 15:38:27 UTC
[I] Recall silently drops passages for unpaired UTF-16 surrogates (post-merge P2 from #5368) [maka]
Astro-Han opened a new issue, #5411:
URL: https://github.com/apache/maka/issues/5411
Follow-up to #5368 (merged as c9b0a2d9). Post-merge re-verification confirms the risk persists on current main.
**Finding (P2):** unpaired UTF-16 surrogates (e.g. U+D800) are not treated as JSON-unstable:
- `packages/core/src/recall.ts:127-144` does not classify unpaired surrogates as JSON-unstable;
- `packages/storage/src/recall-fold.ts:37-46` classifies them as stable, so the candidate SQL (`:64-75`, used via `sqlite-runtime-store.ts:1532-1553` and `sqlite-session-metadata-store.ts:2699-2787`) only searches the serialized payload with `instr(lower(...), term)`.
- `JSON.stringify(U+D800)` stores ASCII `\\ud800` with no raw surrogate, so the candidate query misses; but the JS verifier still matches the raw transcript (`recall.ts:843-896`), while `collectHits:828-840` accepts empty candidates without fallback — a valid passage is silently dropped. Confirmed with a direct Node 22 probe (serialized value has no raw surrogate, `isRecallFoldUnstable` is false).
**Suggested fix:** force full-scan fallback for unpaired surrogates, or make storage over-select JSON surrogate escapes, plus core/storage regression tests.
**Notes:** detected during post-merge re-verification by @Luna-Deep-Qronos; full evidence in `/home/ubuntu/.slock/agents/6cd31702-ca53-487e-8cf7-abc2b2f6728a/reports/pr5368-c9b0a2d-followup.md` (agent workspace).
---
> **Automated review notice:** This issue was filed by an AI review orchestrator operated by Astro-Han. It does not replace independent human review; merge/close decisions remain with humans.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org
Re: [I] Recall silently drops passages for unpaired UTF-16 surrogates (post-merge P2 from #5368) [maka]
Posted by "Totoro-qaq (via GitHub)" <gi...@apache.org>.
Totoro-qaq commented on issue #5411:
URL: https://github.com/apache/maka/issues/5411#issuecomment-5707483243
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org
Re: [I] Recall silently drops passages for unpaired UTF-16 surrogates (post-merge P2 from #5368) [maka]
Posted by "Astro-Han (via GitHub)" <gi...@apache.org>.
Astro-Han closed issue #5411: Recall silently drops passages for unpaired UTF-16 surrogates (post-merge P2 from #5368)
URL: https://github.com/apache/maka/issues/5411
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org