You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by "sihyeonn (via GitHub)" <gi...@apache.org> on 2026/04/02 23:49:47 UTC
[I] feat(ai-proxy): add defaults field for fallback model parameters [apisix]
sihyeonn opened a new issue, #13149:
URL: https://github.com/apache/apisix/issues/13149
## Summary
Add a `defaults` field to the `ai-proxy` and `ai-proxy-multi` plugins that provides fallback model parameters applied only when the client request does not specify them.
## Motivation
Currently, the `options` field always overrides request values regardless of what the client sends. This is useful for enforcing parameters, but there is no way to provide sensible defaults that the client can still override.
A common use case is an LLM gateway where the operator wants to set default values (e.g., `max_tokens`, `temperature`) without restricting clients from customizing these values in their requests.
## Proposed Design
Add a `defaults` field alongside the existing `options` field:
```yaml
plugins:
ai-proxy:
model:
provider: openai
name: gpt-4o
defaults:
max_tokens: 512
temperature: 0.7
options:
model: gpt-4o # always enforced
```
**Priority order** (highest to lowest):
1. `options` — always overrides, even if the client sends a value
2. client request body — client-provided values
3. `defaults` — fallback, applied only when the client omits the field
## Difference from `options`
| Field | Behavior |
|-------|----------|
| `options` | Always overrides request values (enforcement) |
| `defaults` | Only applied when the client does not set the field (fallback) |
## Examples
Given `defaults: { max_tokens: 512, temperature: 0.7 }` and `options: { model: gpt-4o }`:
| Client sends | Effective value |
|-------------|-----------------|
| `max_tokens: 1024` | `1024` (client wins over defaults) |
| _(no max_tokens)_ | `512` (defaults applied) |
| `model: gpt-3.5-turbo` | `gpt-4o` (options always wins) |
## Implementation
A working implementation is available in PR #12895. Happy to revise based on community feedback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org