You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@maka.apache.org by "sunrioa (via GitHub)" <gi...@apache.org> on 2026/09/18 15:26:04 UTC
[I] Connection test probes a hard-coded premium model when the connection has no enabled model (OpenRouter → anthropic/claude-sonnet-5) [maka]
sunrioa opened a new issue, #5493:
URL: https://github.com/apache/maka/issues/5493
### What happened
I created an OpenRouter connection, had not enabled any model yet, and clicked "Test connection".
The OpenRouter activity log then showed a request to **Claude Sonnet 5** that I never selected
anywhere in Maka: 9 prompt tokens, 13 completion tokens, $0.000148, app "Unknown".
Maka's own usage statistics have no record of that call (connection tests are not written to the
usage tables), so it took a while to trace it back to the connection probe.
Root cause, verified against `main` @ 5b9db1ce8:
- `resolveConnectionTestModel` in `packages/runtime/src/test-connection.ts` picks the probe model in
this order (line 94): explicit model → the connection's enabled models → the provider's
hard-coded `fallbackModels` → the connection's discovered inventory.
- For OpenRouter, `fallbackModels` is `openrouterModelIds`
(`packages/core/src/provider-registry.ts` lines 516–519, wired at line 1441), whose first entry
is `anthropic/claude-sonnet-5`.
- Because the hard-coded list is consulted *before* the discovered inventory, a fresh connection
with no enabled model is probed with Sonnet 5 even when Maka has already fetched the account's
inventory (444 models in my case, including several `:free` ones).
- The probe body is `Hi` with `max_tokens: 16` (lines 420–421 / 461–462), which matches the
9 / 13 token counts exactly. At OpenRouter's $2/M input and $10/M output for Sonnet 5 the cost is
9 × 0.000002 + 13 × 0.00001 = $0.000148, the amount in the log.
This contradicts the principle documented right above that function (line 75, referring to #1584):
the probe must not be redirected onto a model the user never chose. The hard-coded fallback does
exactly that for every new connection.
Expected instead:
1. Probe an enabled model when one exists (already the case).
2. If none is enabled but an inventory was fetched, pick from that inventory, preferring a `:free`
model (or, once pricing is stored, the cheapest one) over the hard-coded list.
3. Only fall back to the hard-coded list when there is no inventory at all (legacy connections).
4. Optionally tell the user which model the test is about to use, or ask them to pick one first.
Impact beyond the (small) unexpected charge: OpenRouter keys can carry a model allow-list, and some
accounts don't have every provider enabled. When the hard-coded model is rejected, Maka records the
result as a failed connection although the key is fine.
### How to reproduce
1. Settings → Connections → add an OpenRouter connection with a valid API key.
2. Fetch the model list, but do not enable any model.
3. Click "Test connection" (the command-palette action "Test connection · OpenRouter" behaves the same;
neither passes a model).
4. Open https://openrouter.ai/activity: a request to `anthropic/claude-sonnet-5` with 9 prompt /
13 completion tokens appears, sent by Maka's probe.
5. Settings → Usage in Maka shows no corresponding call.
### Environment
- Maka Desktop 0.2.0, local packaged build from a branch based on main (behaviour verified unchanged on main @ 5b9db1ce8)
- Runtime Host: local
- macOS 26.5 (Darwin 25.5.0), Apple Silicon
- Provider: OpenRouter, connection created with the model inventory fetched and no model enabled
### Logs, screenshots, or additional context
OpenRouter activity row: Sep 17, 10:01 local · Claude Sonnet 5 · "Claude Platform on AWS" ·
9 tok / 13 tok · $0.000148.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org
Re: [I] Connection test probes a hard-coded premium model when the connection has no enabled model (OpenRouter → anthropic/claude-sonnet-5) [maka]
Posted by "Aaron-Ben (via GitHub)" <gi...@apache.org>.
Aaron-Ben commented on issue #5493:
URL: https://github.com/apache/maka/issues/5493#issuecomment-5872705917
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org
Re: [I] Connection test probes a hard-coded premium model when the connection has no enabled model (OpenRouter → anthropic/claude-sonnet-5) [maka]
Posted by "Lxr-max (via GitHub)" <gi...@apache.org>.
Lxr-max commented on issue #5493:
URL: https://github.com/apache/maka/issues/5493#issuecomment-5733775049
I'll take this — fixing `resolveConnectionTestModel` so connection test prefers an enabled/fallback model instead of hard-coding Sonnet when nothing is enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@maka.apache.org
For additional commands, e-mail: commits-help@maka.apache.org