You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by "ChuanFF (via GitHub)" <gi...@apache.org> on 2026/03/14 17:17:07 UTC
[PR] fix(ai-drivers): handle SSE events split across multiple chunks [apisix]
ChuanFF opened a new pull request, #13090:
URL: https://github.com/apache/apisix/pull/13090
## Problem
The original SSE parser used `ngx.re.split("\n\n")`, which failed when events were fragmented across HTTP chunks. This caused data loss and truncated AI responses, especially in high-latency networks.
fix: https://github.com/apache/apisix/issues/12733
## Solution
- **State Machine Parser:** Replaced split logic with a character-by-character state machine.
- event_start_index: Start of current SSE event
- line_start_index: Start of current line within event
- value_start_index: Start of value after : separator
- **Buffer Management:** Preserves incomplete events between chunks to ensure correct reassembly.
- **Spec Compliant:** Properly handles multi-line `data:` fields and `[DONE]` markers.
## Testing
- Added `t/plugin/ai-proxy-sse-split.t` with 3 scenarios:
- Single event split across chunks
- Multi-line `data` fields
- JSON fragmentation within events
## Changes
- `apisix/plugins/ai-drivers/sse.lua`: New parser logic
- `apisix/plugins/ai-drivers/openai-base.lua`: Buffer state propagation
- `t/plugin/ai-proxy-sse-split.t`: New test suite
## Compatibility
✅ Fully backward compatible. Buffer parameter is optional.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(ai-drivers): handle SSE events split across multiple chunks [apisix]
Posted by "Baoyuantop (via GitHub)" <gi...@apache.org>.
Baoyuantop commented on PR #13090:
URL: https://github.com/apache/apisix/pull/13090#issuecomment-4212433730
The issue mentioned in this PR has been resolved in #13170. If you still have problems, please reopen this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(ai-drivers): handle SSE events split across multiple chunks [apisix]
Posted by "Baoyuantop (via GitHub)" <gi...@apache.org>.
Baoyuantop closed pull request #13090: fix(ai-drivers): handle SSE events split across multiple chunks
URL: https://github.com/apache/apisix/pull/13090
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(ai-drivers): handle SSE events split across multiple chunks [apisix]
Posted by "ChuanFF (via GitHub)" <gi...@apache.org>.
ChuanFF commented on PR #13090:
URL: https://github.com/apache/apisix/pull/13090#issuecomment-4064917395
@moonming Thanks for the suggestion. I will try various solutions and compare their performance, but this may take one or two weeks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org