You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by "RGZingYang (via GitHub)" <gi...@apache.org> on 2026/07/31 08:02:40 UTC

[I] bug: ai-proxy-multi: Variable $request_llm_model reset in fallback_strategy [apisix]

RGZingYang opened a new issue, #13769:
URL: https://github.com/apache/apisix/issues/13769

   ### Current Behavior
   
   开启日志记录:
   nginx_config:
     http:
       access_log_format: "$remote_addr - $remote_user [$time_local] $http_host \"$request_line\" $status $body_bytes_sent $request_time \"$http_referer\" \"$http_user_agent\" $upstream_addr $upstream_status $apisix_upstream_response_time \"$upstream_scheme://$upstream_host$upstream_uri\" \"$apisix_request_id\" \"$request_type\" \"$llm_time_to_first_token\" \"$llm_model\" \"$request_llm_model\"  \"$llm_prompt_tokens\" \"$llm_completion_tokens\""
   使用ai-proxy-multi创建路由 开启fallback_strategy, 发生重试后$request_llm_model不是原始请求的model
   创建路由的curl:
   ```
   curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
     -H "X-API-KEY: ${admin_key}" \
     -d '{
       "id": "ai-proxy-multi-route",
       "uri": "/v1/chat/completions",
       "methods": ["POST"],
               "vars": [
               [
                   "post_arg.model",
                   "==",
                   "model_test"
               ]
           ],
       "plugins": {
    "ai-proxy-multi": {
           "instances": [
             {
               "provider": "openai-compatible",
               "options": {
                 "model": "upstream-model-A"
               },
               "override": {
                 "endpoint": "http://172.0.0.1:8081/test-upstream/v1/chat/completions"
               },
               "priority": 1,
               "name": "test1",
               "auth": {},
               "weight": 1
             },
             {
               "provider": "openai-compatible",
               "options": {
                 "model": "upstream-model-B"
               },
               "override": {
                 "endpoint": "http://172.0.0.1:8081/test-upstream/v1/chat/completions"
               },
               "priority": 0,
               "name": "test2",
                "auth": {},
               "weight": 1
             }
           ],
   
           "balancer": {
             "hash_on": "consumer",
             "algorithm": "chash"
           },
           "timeout": 500,
           "fallback_strategy": [
             "http_429",
             "http_5xx"
           ],
           "logging": {
             "summaries": true,
             "payloads": true
           },
           "keepalive": false
         },
       }
     }'
   ```
   步骤:
   发送请求:{
     "model": "model_test",
     "messages": [
       {
         "role": "user",
         "content": "返回1"
       }
     ]
   }
   
   上游正常返回 $request_llm_model 输出的是请求中的model参数 model_test  $llm_model输出 upstream-model-A 符合预期
     <img width="1630" height="26" alt="Image" src="https://github.com/user-attachments/assets/112db910-e300-438e-b40f-fbd759723b59" />
   发生重试后 $request_llm_model 输出为  upstream-model-A  $llm_model输出 upstream-model-B  不符合预期  $request_llm_model应还是输出 model_test :<img width="1960" height="32" alt="Image" src="https://github.com/user-attachments/assets/2372602b-8478-41bc-b701-3d5c4d6d8bab" />
   
   ### Expected Behavior
   
   $request_llm_model 应是保持原始的模型名称  重试不应该改变, 即使为了兼容不使用 "vars": [["post_arg.model","==","model_test"],请求参数不带model  也应该在赋值的时候判断下 是否为空  如果不为空的情况下不重新赋值
   
   ### Error Logs
   
   
   重试会调用base.before_proxy的方法重试  
   if request_model then
               ctx.var.request_llm_model = request_model
           end
   赋值的时候需要添加判断是否已经赋过值了<img width="1044" height="900" alt="Image" src="https://github.com/user-attachments/assets/0997ab09-fe30-4350-b21f-0d2924b32011" />
   
   ### Steps to Reproduce
   
   步骤如描述中
   
   ### Environment
   
   - APISIX version (run `apisix version`): 3.17.0
   - Operating system (run `uname -a`):
   - OpenResty / Nginx version (run `openresty -V` or `nginx -V`):
   - etcd version, if relevant (run `curl http://127.0.0.1:9090/v1/server_info`):
   - APISIX Dashboard version, if relevant:
   - Plugin runner version, for issues related to plugin runners:
   - LuaRocks version, for installation issues (run `luarocks --version`):
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] bug: ai-proxy-multi: Variable $request_llm_model reset in fallback_strategy [apisix]

Posted by "nic-6443 (via GitHub)" <gi...@apache.org>.
nic-6443 closed issue #13769: bug: ai-proxy-multi: Variable $request_llm_model reset in fallback_strategy
URL: https://github.com/apache/apisix/issues/13769


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org