You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "hf-13 (via GitHub)" <gi...@apache.org> on 2026/06/19 12:10:15 UTC
[I] SQL Lab shows "The database that was used to generate this query could not be found" for most users when there are > 20 databases [superset]
hf-13 opened a new issue, #41245:
URL: https://github.com/apache/superset/issues/41245
### Bug description
### Superset version
6.1.0
### What's the issue?
When SQL Lab initializes, it fetches the database list via:
GET /api/v1/database/?q=(filters:!((col:database_name,opr:neq,value:examples)))
No `page_size` is specified, so FAB defaults to **20 results**. The response is dispatched as `SET_DATABASES`, populating the Redux `databases` map with only those 20 entries (keyed by integer `id`).
When a user's saved tab has a `database_id` that doesn't appear in the first 20 results, `databases[dbId]` is `undefined` → the frontend evaluates `F = isEmpty(databases[dbId])` as `true` → the yellow warning
banner appears:
> **The database that was used to generate this query could not be found**
> Choose one of the available databases on the left panel.
The database exists, permissions are correct, and the tab state in the backend is valid. This is a pure frontend state population problem. Reloading the page sometimes resolves it (timing-dependent). Clicking
the database in the left panel always resolves it (that search uses `page_size=1000` and correctly finds the database).
### Steps to reproduce
1. Have a Superset instance with **more than 20 databases**
2. Ensure a user has a saved SQL Lab tab whose `database_id` is not among the first 20 results of `GET /api/v1/database/` (default sort)
3. Open SQL Lab
4. Observe the yellow "database not found" banner on that tab despite the database being visible and selectable in the left panel
### Expected behavior
SQL Lab loads without the "database not found" error. All saved tabs display correctly regardless of how many databases exist.
### Actual behavior
Any user whose saved tab references a database outside the first 20 API results sees the error on every fresh SQL Lab load.
### Root cause
`GET /api/v1/database/` without an explicit `page_size` returns FAB's default of 20 rows. The SQL Lab init code does not request a larger page size or paginate through results. The Redux `databases` map is
therefore incomplete.
Relevant frontend code (SQL Lab entry bundle, `SET_DATABASES` reducer):
```js
r.databases.forEach(t => { e[t.id] = { ...t, extra_json: JSON.parse(t.extra || "") } })
Relevant error condition:
const F = useMemo(() => !c || isEmpty(c), [c]) // c = databases[dbId || ""]
// F === true when dbId not in the 20-entry Redux map → shows error banner
Workaround
Patch DatabaseRestApi.page_size at app startup via FLASK_APP_MUTATOR:
def FLASK_APP_MUTATOR(app):
from superset.databases.api import DatabaseRestApi
DatabaseRestApi.page_size = 100000
This causes the unpaginated init call to return all databases. DatabaseRestApi.max_page_size is already -1 (unlimited), so no server-side cap blocks this.
Proposed fix
The correct fix is in the frontend. The SQL Lab init fetch should either:
Option A — explicitly request all databases:
// in the SQL Lab init fetch
GET /api/v1/database/?q=(page_size:100000,filters:!(...))
Option B — defer tab rendering until SET_DATABASES is fully populated, then fall back to a targeted fetch for any dbId still missing from the map.
Option B is more robust for very large installations but requires more frontend work. Option A is a one-line fix.
Environment
- Superset 6.1.0
- ~885 databases in the instance
- FAB default page_size: 20
- DatabaseRestApi.max_page_size: -1 (unlimited)
### Screenshots/recordings
_No response_
### Superset version
master / latest-dev
### Python version
3.9
### Node version
16
### Browser
Chrome
### Additional context
### Superset version
6.1.0
### What's the issue?
When SQL Lab initializes, it fetches the database list via:
GET /api/v1/database/?q=(filters:!((col:database_name,opr:neq,value:examples)))
No `page_size` is specified, so FAB defaults to **20 results**. The response is dispatched as `SET_DATABASES`, populating the Redux `databases` map with only those 20 entries (keyed by integer `id`).
When a user's saved tab has a `database_id` that doesn't appear in the first 20 results, `databases[dbId]` is `undefined` -> the frontend evaluates `F = isEmpty(databases[dbId])` as `true` -> the yellow warning banner appears:
> **The database that was used to generate this query could not be found**
> Choose one of the available databases on the left panel.
The database exists, permissions are correct, and the tab state in the backend is valid. This is a pure frontend state population problem. Reloading the page sometimes resolves it (timing-dependent). Clicking the database in the left panel always resolves it (that search uses `page_size=1000` and correctly finds the database).
## Steps to reproduce
1. Have a Superset instance with **more than 20 databases**
2. Ensure a user has a saved SQL Lab tab whose `database_id` is not among the first 20 results of `GET /api/v1/database/` (default sort)
3. Open SQL Lab
4. Observe the yellow "database not found" banner on that tab despite the database being visible and selectable in the left panel
### Expected behavior
SQL Lab loads without the "database not found" error. All saved tabs display correctly regardless of how many databases exist.
### Actual behavior
Any user whose saved tab references a database outside the first 20 API results sees the error on every fresh SQL Lab load.
### Root cause
`GET /api/v1/database/` without an explicit `page_size` returns FAB's default of 20 rows. The SQL Lab init code does not request a larger page size or paginate through results. The Redux `databases` map is
therefore incomplete.
Mainly - SQL Lab entry bundle, `SET_DATABASES` reducer.
### Workaround
Patch DatabaseRestApi.page_size at app startup via FLASK_APP_MUTATOR:
```python
def FLASK_APP_MUTATOR(app):
from superset.databases.api import DatabaseRestApi
DatabaseRestApi.page_size = 100000
```
This causes the unpaginated init call to return all databases. DatabaseRestApi.max_page_size is already -1 (unlimited), so no server-side cap blocks this.
### Proposed fix
Option A — explicitly request all databases
Option B — defer tab rendering until SET_DATABASES is fully populated, then fall back to a targeted fetch for any dbId still missing from the map.
### Environment
- Superset 6.1.0
- ~885 databases in the instance
- FAB default page_size: 20
- DatabaseRestApi.max_page_size: -1 (unlimited)
### Checklist
- [x] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [x] I have searched the GitHub issue tracker and didn't find a similar bug report.
- [x] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] SQL Lab shows "The database that was used to generate this query could not be found" for most users when there are > 20 databases [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed issue #41245: SQL Lab shows "The database that was used to generate this query could not be found" for most users when there are > 20 databases
URL: https://github.com/apache/superset/issues/41245
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] SQL Lab shows "The database that was used to generate this query could not be found" for most users when there are > 20 databases [superset]
Posted by "GagandeepSingh20 (via GitHub)" <gi...@apache.org>.
GagandeepSingh20 commented on issue #41245:
URL: https://github.com/apache/superset/issues/41245#issuecomment-4761371329
Hi, I investigated the code path locally and traced the issue to the **SQL Lab** database state flow. `DatabaseSelector` fetches paginated database results and passes them to `setDatabases()`. The `SET_DATABASES` reducer currently rebuilds and replaces the entire `state.databases` map, which appears to remove previously loaded database IDs from Redux. This seems consistent with the reported behavior. I'll work on a fix and add a regression test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] SQL Lab shows "The database that was used to generate this query could not be found" for most users when there are > 20 databases [superset]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
dosubot[bot] commented on issue #41245:
URL: https://github.com/apache/superset/issues/41245#issuecomment-4751539980
<!-- Answer -->
I can confirm this bug and the root cause analysis is spot-on, though the exact mechanism is slightly different from what's described in the issue.
**The actual flow:**
1. SQL Lab bootstraps via `GET /api/v1/sqllab/` which calls `DatabaseDAO.find_all()` — this fetches **all** databases without pagination [[1]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/sqllab/utils.py) [[2]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/daos/base.py). So the initial Redux state is complete.
2. Then `SqlEditorLeftBar` mounts and renders the `DatabaseSelector` component, which calls `loadDatabases` **with pagination** (`page` and `page_size` parameters) [[3]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset-frontend/src/components/DatabaseSelector/index.tsx).
3. The paginated result (default ~20 databases) is passed to the `handleDbList` callback, which dispatches `SET_DATABASES` [[4]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset-frontend/src/SqlLab/components/SqlEditorLeftBar/index.tsx#L193-L198).
4. The `SET_DATABASES` reducer **completely replaces** the `databases` state rather than merging with the existing bootstrap data [[5]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset-frontend/src/SqlLab/reducers/sqlLab.js#L667-L676):
```javascript
[actions.SET_DATABASES]() {
const databases = {};
action.databases.forEach(db => {
databases[db.id] = { ...db, extra_json: JSON.parse(db.extra || '') };
});
return { ...state, databases }; // full replacement, not merge
},
```
5. All databases from the bootstrap that weren't in the paginated response are now gone from Redux → the "database not found" warning fires [[6]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset-frontend/src/SqlLab/components/SqlEditor/index.tsx#L321-L324).
**The fix should target the overwrite, not the fetch.** A few approaches:
- **Change `SET_DATABASES` to merge** instead of replace: `return { ...state, databases: { ...state.databases, ...databases } }`
- **Remove the `getDbList` dispatch** from `SqlEditorLeftBar` since the bootstrap already provides the complete list, and let `DatabaseSelector` manage its own local state for the dropdown
- **Pass a large `page_size`** in the `loadDatabases` call (your Option A), though this only masks the real issue
Your `FLASK_APP_MUTATOR` workaround is valid for now but won't prevent the overwrite — it works because it changes the default page size for all endpoints, which happens to affect the `DatabaseSelector` fetch too. The cleanest short-term fix would be changing the reducer to merge.
<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=reply-with-mention&utm_term=apache%2Fsuperset).*
---
Docs are dead. Just use [Dosu](https://dosu.dev?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=static-docs-tagline&utm_term=apache%2Fsuperset).
[](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=static-docs-feedback&utm_term=apache%2Fsuperset&message_id=31648160-3312-4aab-9b58-9447103066ef) [](https://github.dosu.com/apache/superset?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=static-docs-ask-repo&utm_term=apache%2Fsuperset) [](https://app.dosu.dev/signup?referrer=openSource&source=github-footer&utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=static-docs-share-team&utm_term=apache%2Fsuperset)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] SQL Lab shows "The database that was used to generate this query could not be found" for most users when there are > 20 databases [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #41245:
URL: https://github.com/apache/superset/issues/41245#issuecomment-4770886673
Thanks for opening a PR. Maybe @sadpandajoe will want to take a look at this...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] SQL Lab shows "The database that was used to generate this query could not be found" for most users when there are > 20 databases [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #41245:
URL: https://github.com/apache/superset/issues/41245#issuecomment-5012303739
Fixed by https://github.com/apache/superset/pull/41281
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] SQL Lab shows "The database that was used to generate this query could not be found" for most users when there are > 20 databases [superset]
Posted by "GagandeepSingh20 (via GitHub)" <gi...@apache.org>.
GagandeepSingh20 commented on issue #41245:
URL: https://github.com/apache/superset/issues/41245#issuecomment-4761082855
Hi, I'd like to take a look at this issue. I'll try to reproduce it locally and investigate the reducer/state update flow described above. If I can confirm the root cause, I'll work on a fix and open a PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org