You are viewing a plain text version of this content. The canonical link for it is here.

Posted to notifications@shenyu.apache.org by "Aias00 (via GitHub)" <gi...@apache.org> on 2026/09/01 13:06:46 UTC

[I] [BUG] AiProxy forwards internal fallbackConfig and credentials to primary upstream [shenyu]

Aias00 opened a new issue, #7020:
URL: https://github.com/apache/shenyu/issues/7020

## Description

PR #6341 changes AiProxy to build Spring AI `ChatCompletionRequest` directly from the raw client JSON. ShenYu's request-only `fallbackConfig` field is not removed before conversion.

Spring AI 1.1.2 stores unknown request fields in `ChatCompletionRequest.extraBody` through `@JsonAnySetter` and serializes them through `@JsonAnyGetter`. As a result, the gateway forwards the internal fallback configuration to the primary model provider before fallback is needed.

## Affected code

- `shenyu-plugin-ai-common/.../OpenAiProtocolAdapter.java`: raw request tree is converted without removing `fallbackConfig`.
- `shenyu-plugin-ai-proxy/.../AiProxyConfigService.java`: the same field is intentionally parsed as ShenYu dynamic fallback metadata.
- `shenyu-plugin-ai-proxy/.../AiProxyPlugin.java`: the resulting request is sent through `OpenAiApi` for both streaming and non-streaming calls.

Related PR: #6341, reviewed at head `8fa93d7446e052f149d3740bbe7ff2052c375e23`.

## Reproduction

Given:

```json
{
"model": "main",
"messages": [{"role": "user", "content": "hi"}],
"fallbackConfig": {
"baseUrl": "https://fallback.example",
"apiKey": "fallback-secret",
"model": "fallback-model"
}
}
```

A local capture of the actual `OpenAiApi.chatCompletionEntity` HTTP request body contained both:

```json
"fallbackConfig": {
"baseUrl": "https://fallback.example",
"apiKey": "fallback-secret",
"model": "fallback-model"
}
```

and the same object under `extra_body`.

## Impact

- Fallback provider credentials and routing metadata are disclosed to the primary provider.
- Strict OpenAI-compatible providers may reject requests containing ShenYu-only fields.
- Requests using dynamic fallback can fail before the fallback path is triggered.

## Expected behavior

Gateway-only metadata must never be serialized into upstream model requests. Dynamic fallback resolution should continue to use the field internally.

## Acceptance criteria

- [ ] Strip `fallbackConfig` and any other ShenYu-only request fields before creating `ChatCompletionRequest`.
- [ ] Apply the sanitization to both primary and fallback request construction.
- [ ] Add a regression test that captures or serializes the outbound request and asserts `fallbackConfig`, `baseUrl`, and the fallback API key are absent.
- [ ] Verify dynamic fallback still resolves and executes with its configured provider/model/key.
- [ ] Cover streaming and non-streaming request paths.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shenyu.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [BUG] AiProxy forwards internal fallbackConfig and credentials to primary upstream [shenyu]

Posted by "Aias00 (via GitHub)" <gi...@apache.org>.

Aias00 closed issue #7020: [BUG] AiProxy forwards internal fallbackConfig and credentials to primary upstream
URL: https://github.com/apache/shenyu/issues/7020


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shenyu.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [BUG] AiProxy forwards internal fallbackConfig and credentials to primary upstream [shenyu]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] commented on issue #7020:
URL: https://github.com/apache/shenyu/issues/7020#issuecomment-5494897520

   👋 Hello @yuluo-yx
   
   A new issue has been opened by @Aias00:
   - **Title**: [BUG] AiProxy forwards internal fallbackConfig and credentials to primary upstream
   - **Link**: https://github.com/apache/shenyu/issues/7020
   
   Please review when you have time. Thank you! 🙏


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shenyu.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [BUG] AiProxy forwards internal fallbackConfig and credentials to primary upstream [shenyu]

Posted by "zhang-arvin (via GitHub)" <gi...@apache.org>.

zhang-arvin commented on issue #7020:
URL: https://github.com/apache/shenyu/issues/7020#issuecomment-5495887213

   I would like to work on this issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shenyu.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org