Re: [PR] chore: refresh the models.dev catalog snapshot [maka]

["Astro-Han (via GitHub)" <gi...@apache.org>]

Astro-Han commented on PR #3324:
URL: https://github.com/apache/maka/pull/3324#issuecomment-5359636197

   @hqhq1025 Thanks — both findings reproduce, and the ZenMux one is fixed in 8dca808.
   
   **[P2] Fixed, narrowly.** Confirmed: 15 ZenMux models flip active→deprecated in this refresh; 2 carry a model-level protocol override and are excluded from the fallback source, so 13 reach it. `toolCallingModelIds` filters on tool-calling capability only, and the providers that exclude deprecated ids do it at the call site. ZenMux now does the same.
   
   One correction to the scope: this is not a ZenMux-specific gap, it is the default. Six providers still ship deprecated ids in their fallback lists — 28 models: `togetherai` (11), `mistral` (8), `xiaomi` (3), `deepinfra` (3), `nvidia` (2), `openai` (1, `gpt-4-turbo`). `openai` writes its list by hand, so converging every list onto the helper would not cover it either. Reverting this refresh changes none of them, so I kept this PR to what the refresh caused — ZenMux going 0 → 13 — and filed the rest as #3355. The new contract test records those six as an explicit boundary and fails both when a provider regresses into it and when a listed provider is cleaned up without being removed from the list, so it shrinks as #3355 lands instead of going stale.
   
   **[P1] Real, but it predates this branch — filed as #3354 rather than fixed here.** The chain reproduces as you describe, and `opencode-free` is the worst case: `modelDiscovery: { kind: 'fallback' }`, whose own reason string states the anonymous endpoint has no model-list contract, so there is no later discovery pass to repair the state at all — unlike Google, where the window is only a failed discovery.
   
   What I could not find is a precedent for a refresh fixing it. #2050 removed `hy3-free` from the same OpenCode Free list for the same reason — an upstream deprecation — and touched no persistence: `git show 41a983804 --name-only` matches zero files under `packages/storage/` or `packages/runtime-host/`. There is no model-catalog migration or tombstone mechanism; the tombstones in `sqlite-session-metadata-store.ts` serve session metadata only.
   
   The test I applied is whether reverting this PR removes the problem. For ZenMux it does — the 13 ids go back to active. For this one it does not: the three ids return to the fallback lists but remain retired upstream, so a persisted default still fails at call time. That makes it a standing gap this refresh surfaces rather than one it introduces, and closing it means building a model-catalog migration with upgrade-baseline tests — an intent that should not share a revert with a data refresh.
   
   Happy to take it as a follow-up if you would rather it block here.
   
   Verification on this head: 560 core tests, format, lint, core/runtime typechecks. Both directions of the new test were falsified before commit — dropping the ZenMux filter fails two tests naming the 13 ids, and adding an already-clean provider to the boundary list fails the second.
   
   AI-assisted review response: Claude Code reproduced both findings, measured the six-provider scope, and drafted this reply; the scoping decision and the #2050 precedent check were verified by me.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org