You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "ai-yang (via GitHub)" <gi...@apache.org> on 2026/08/01 10:30:48 UTC
[I] [Studio][Bug] Stale LLM model responses can overwrite the active provider [rocketmq-dashboard]
ai-yang opened a new issue, #733:
URL: https://github.com/apache/rocketmq-dashboard/issues/733
## Affected branch
`rocketmq-studio` at `bbf1b7e0cf25a5065ba049b5450cc8155569f710`.
## Problem
The LLM settings page does not track ownership of asynchronous model-list requests. A response started for the previously selected provider can overwrite the fallback models for a provider that the user selected later. The model loading indicator also stops before the model request finishes.
## Deterministic reproduction
1. Configure the initial LLM provider as OpenAI and keep the `/llm/models` response pending.
2. Open the LLM settings page and wait until that model request has started.
3. Select DeepSeek. The page correctly displays its built-in `deepseek-chat` / `deepseek-reasoner` options.
4. Complete the older OpenAI model request with a model named `openai-only-late-model`.
5. Open the model selector.
The late OpenAI response replaces the DeepSeek options even though DeepSeek remains selected.
A second controlled-Promise regression keeps `/llm/models` pending and observes that the model selector's loading indicator has already disappeared.
The regression tests are:
```text
web/src/pages/studio/__tests__/LlmSettingsAsyncState.test.tsx
LlmSettingsPage async request ownership > does not replace a newly selected provider model list with an older response
LlmSettingsPage async request ownership > keeps the model selector loading while the provider model request is pending
```
On the unmodified branch, both business assertions failed in each of 5/5 isolated Vitest processes under Node 20.19.5. The tests use controlled Promises only: no timers, network access, fixed sleeps, or random scheduling.
## Root cause
`fetchModels` starts `getLlmModels()` inside the `getLlmConfig().then(...)` callback but does not return or await that inner Promise. Therefore, the outer `finally` executes as soon as the configuration request finishes and sets `modelsLoading` to `false` while the model request is still pending.
The eventual model response also updates `modelOptions` unconditionally. There is no request generation or selected-provider check, and changing the provider does not invalidate an older request.
## Impact
- The model dropdown can show models belonging to a different provider than the selected provider.
- A user can save or test a mismatched provider/model combination.
- The loading indicator reports completion before the provider model request has completed.
- A response arriving after the page is unmounted can still attempt to update component state.
## Expected behavior
- Only the latest model request that still belongs to the active provider may update model options or loading state.
- Changing providers must invalidate any older model request before applying the new provider's fallback options.
- The loading indicator must remain active until the owned model request resolves or rejects.
- Unmounting the page must invalidate outstanding work.
## Proposed atomic scope
- Move initial loading out of the render path into an effect with cleanup.
- Await/return the complete configuration-plus-model Promise chain.
- Use a request generation (or equivalent ownership token) so stale responses and stale `finally` callbacks cannot update state.
- Add controlled-Promise regression coverage for provider switching and loading lifetime.
This does not change the LLM API contract or provider configuration format.
## Related work
- The expected production change is limited to `web/src/pages/studio/LlmSettings.tsx`, with regression coverage in its page tests.
- Merged PR #483 introduced the settings page, and merged PR #651 introduced provider model loading and fallback options. PR #651 does not invalidate stale requests or await the inner model request.
- Merged PR #536 concerns API-key masking across provider changes, not asynchronous model-request ownership.
- Closed issue #545 tracks the backend LLM gateway and is not a frontend request-ordering report.
- Searches across open and closed issues, plus open/closed/merged pull requests, covered `LlmSettings`, `LlmSettings.tsx`, `getLlmModels`, provider switching, model-list loading, fallback models, out-of-order responses, and stale model responses. No equivalent report, fix, or claimed work was found.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Studio][Bug] Stale LLM model responses can overwrite the active provider [rocketmq-dashboard]
Posted by "ai-yang (via GitHub)" <gi...@apache.org>.
ai-yang commented on issue #733:
URL: https://github.com/apache/rocketmq-dashboard/issues/733#issuecomment-5159074519
Additional verification evidence is available in PR #737:
- On `rocketmq-studio` at `bbf1b7e0cf25a5065ba049b5450cc8155569f710`, both deterministic stale-response/loading assertions failed in 5/5 isolated Node 20.19.5 processes.
- With the fix, 20 isolated processes passed 5/5 tests each (100/100); related LLM/API/client tests passed 29/29 and the full frontend suite passed 270/270.
- ESLint reported 0 errors, the TypeScript/Vite build passed, and all three PR CI jobs succeeded.
The change does not alter the LLM API or provider configuration format.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Studio][Bug] Stale LLM model responses can overwrite the active provider [rocketmq-dashboard]
Posted by "lizhimins (via GitHub)" <gi...@apache.org>.
lizhimins commented on issue #733:
URL: https://github.com/apache/rocketmq-dashboard/issues/733#issuecomment-5161946062
Resolved by #737 (merged into rocketmq-studio as 46b03bb).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Studio][Bug] Stale LLM model responses can overwrite the active provider [rocketmq-dashboard]
Posted by "RockteMQ-AI (via GitHub)" <gi...@apache.org>.
RockteMQ-AI commented on issue #733:
URL: https://github.com/apache/rocketmq-dashboard/issues/733#issuecomment-5151550351
**Issue Evaluation**
Category: `bug` | Status: **Needs Verification**
Thank you for reporting this issue. The reported behavior will be verified against the current codebase by our automated analysis pipeline.
A detailed assessment will follow once verification is complete. If you have additional context (version, reproduction steps, logs), please share it to help speed up the process.
---
*Automated evaluation by github-manager*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Studio][Bug] Stale LLM model responses can overwrite the active provider [rocketmq-dashboard]
Posted by "lizhimins (via GitHub)" <gi...@apache.org>.
lizhimins closed issue #733: [Studio][Bug] Stale LLM model responses can overwrite the active provider
URL: https://github.com/apache/rocketmq-dashboard/issues/733
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org