You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tooling.apache.org by "andrewmusselman (via GitHub)" <gi...@apache.org> on 2026/09/03 01:19:38 UTC
[I] Remove the model_list include from litellm.yaml (tooling-llmao)
andrewmusselman opened a new issue, #13:
URL: https://github.com/apache/tooling-llmao/issues/13
`litellm.yaml` has:
```yaml
include:
- model_list.yaml
```
Once routes come from the database, that include is actively harmful.
`model_list.yaml` becomes the **catalog** — what each model is, its licence and
provenance, and the vLLM recipe — and its entries carry no `api_base`, because
that is per-instance and only known when a host is added.
LiteLLM reading catalog entries as routes therefore creates routes with no
`api_base`, and sends every call to `api.openai.com`. That is issue "LiteLLM routes have no api_base",
reproduced on every restart.
**So this is not optional cleanup.** Removing the include has to land with "Move fleet state into LiteLLM"
and "Health-gated route registration", not after.
After: LiteLLM starts with zero routes; llmao populates them on the healthy
transition. `model_list.yaml` is read by `llmao/models.py` only —
`public_models()` already strips `litellm_params` and surfaces `model_info` as
the catalog UX, so the file is already shaped for this.
Related header change in `model_list.yaml.example`, which currently reads:
```
# Production: do not use STORE_MODEL_IN_DB for inventory — this YAML is SoT.
# litellm.yaml includes this file.
```
Both statements are reversed by "Move fleet state into LiteLLM".
Also drop `api_base` and `api_key` from the example's `litellm_params`. Nothing
secret remains in the file afterwards, so the eyaml substitution note goes too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For additional commands, e-mail: commits-help@tooling.apache.org
Re: [I] Remove the model_list include from litellm.yaml (tooling-llmao)
Posted by "andrewmusselman (via GitHub)" <gi...@apache.org>.
andrewmusselman closed issue #13: Remove the model_list include from litellm.yaml
URL: https://github.com/apache/tooling-llmao/issues/13
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For additional commands, e-mail: commits-help@tooling.apache.org
Re: [I] Remove the model_list include from litellm.yaml (tooling-llmao)
Posted by "andrewmusselman (via GitHub)" <gi...@apache.org>.
andrewmusselman commented on issue #13:
URL: https://github.com/apache/tooling-llmao/issues/13#issuecomment-5613728417
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For additional commands, e-mail: commits-help@tooling.apache.org
Re: [I] Remove the model_list include from litellm.yaml (tooling-llmao)
Posted by "gstein (via GitHub)" <gi...@apache.org>.
gstein commented on issue #13:
URL: https://github.com/apache/tooling-llmao/issues/13#issuecomment-5613800835
FTR, `api_base` is (now) required for vendor/commercial models (`self_hosted:false`) rather than self-hosted models. This requirement is validated at startup with a fast-fail.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For additional commands, e-mail: commits-help@tooling.apache.org
Re: [I] Remove the model_list include from litellm.yaml (tooling-llmao)
Posted by "gstein (via GitHub)" <gi...@apache.org>.
gstein commented on issue #13:
URL: https://github.com/apache/tooling-llmao/issues/13#issuecomment-5518863436
This is quite likely true. Will review within totality, shortly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For additional commands, e-mail: commits-help@tooling.apache.org