You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tooling.apache.org by "andrewmusselman (via GitHub)" <gi...@apache.org> on 2026/09/03 01:27:45 UTC
[I] LiteLLM routes have no api_base, so every call goes to OpenAI (tooling-llmao)
andrewmusselman opened a new issue, #23:
URL: https://github.com/apache/tooling-llmao/issues/23
*Context for the stopgap and the phase 2 change, rather than separate work.
File it if a tracking issue for the outage is wanted; otherwise the stopgap and
"Move fleet state into LiteLLM" cover it.*
**Labels:** blocker, fleet
**Fixed properly by:** "Move fleet state into LiteLLM" + "Health-gated route registration". Stopgap: "add api_base to model_list.yaml".
Every call through the proxy fails with an OpenAI authentication error. The
routes have no `api_base`, so LiteLLM sends requests to `api.openai.com` with
the configured key and OpenAI rejects it.
```json
{"error": {"message": "litellm.AuthenticationError: OpenAIException -
Incorrect API key provided: sk-573e6... You can find your API key at
https://platform.openai.com/account/api-keys. Received Model
Group=qwen3-8b", "code": "401"}}
```
The error names `platform.openai.com`, which sends the reader after their own
key โ the one thing that is fine. Worth knowing when triaging.
Observed on `tooling-llmao-ec2-or`:
```yaml
- model_name: qwen3-8b
litellm_params:
model: openai/qwen3-8b
api_key: sk-573e6 # no api_base
```
An earlier `model_list.yaml` had the opposite failure โ
`api_base: https://llm.tooling.apache.org/v1`, which resolves to `/v1/v1/...`,
404s at the CDN, and surfaces as the same misleading OpenAI error. LiteLLM
appends `/v1`; the base must not include it.
`Server.api_base` in `fleet.py:218` already derives the right value from
placement. Nothing writes it into LiteLLM's routes. This is the
already-tracked "wire LiteLLM api_base to healthy fleet ports" item in
`vllm-fleet-design.md` ยง11 โ filed because it is currently a total outage
rather than a gap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For additional commands, e-mail: commits-help@tooling.apache.org
Re: [I] LiteLLM routes have no api_base, so every call goes to OpenAI (tooling-llmao)
Posted by "andrewmusselman (via GitHub)" <gi...@apache.org>.
andrewmusselman closed issue #23: LiteLLM routes have no api_base, so every call goes to OpenAI
URL: https://github.com/apache/tooling-llmao/issues/23
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For additional commands, e-mail: commits-help@tooling.apache.org
Re: [I] LiteLLM routes have no api_base, so every call goes to OpenAI (tooling-llmao)
Posted by "andrewmusselman (via GitHub)" <gi...@apache.org>.
andrewmusselman commented on issue #23:
URL: https://github.com/apache/tooling-llmao/issues/23#issuecomment-5630037756
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For additional commands, e-mail: commits-help@tooling.apache.org