You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "A-little-bit-of-data (via GitHub)" <gi...@apache.org> on 2026/06/24 06:09:09 UTC
[D] Why do the tables generated in the `superset` database differ depending on the `db_type` when using version 6.1.0? [superset]
GitHub user A-little-bit-of-data created a discussion: Why do the tables generated in the `superset` database differ depending on the `db_type` when using version 6.1.0?
Total number of data tables generated using external PostgreSQL data sources:
`ab_group
ab_group_role
ab_permission
ab_permission_view
ab_permission_view_role
ab_register_user
ab_role
ab_user
ab_user_group
ab_user_role
ab_view_menu
alembic_version
annotation
annotation_layer
cache_keys
css_templates
dashboard_roles
dashboard_slices
dashboard_user
dashboards
database_user_oauth2_tokens
dbs
dynamic_plugin
embedded_dashboards
favstar
key_value
keyvalue
logs
query
report_execution_log
report_recipient
report_schedule
report_schedule_user
rls_filter_roles
rls_filter_tables
row_level_security_filters
saved_query
slice_user
slices
sql_metrics
sqlatable_user
ssh_tunnels
tab_state
table_columns
table_schema
tables
tag
tagged_object
task_subscribers
tasks
themes
user_attribute
user_favorite_tag`
The total number of data tables generated using external MySQL data sources is:
`ab_group
ab_group_role
ab_permission
ab_permission_view
ab_permission_view_role
ab_register_user
ab_role
ab_user
ab_user_group
ab_user_role
ab_view_menu
access_request
alembic_version
alert_logs
alert_owner
alerts
annotation
annotation_layer
cache_keys
clusters
columns
css_templates
dashboard_email_schedules
dashboard_roles
dashboard_slices
dashboard_user
dashboards
datasources
dbs
druiddatasource_user
dynamic_plugin
favstar
keyvalue
logs
metrics
query
report_execution_log
report_recipient
report_schedule
report_schedule_user
rls_filter_roles
rls_filter_tables
row_level_security_filters
saved_query
slice_email_schedules
slice_user
slices
sql_metrics
sql_observations
sqlatable_user
tab_state
table_columns
table_schema
tables
tag
tagged_object
url
user_attribute`
Using two different databases with different numbers and names of tables, will this have any impact on daily use?
I checked using `superset db heads` and `superset db current`, and the results were consistent:
`heads: 4b2a8c9d3e1f (head)`
`current: 4b2a8c9d3e1f (head)`
However, there is one significant difference:
```
# PG
Context impl PostgresqlImpl.
Will assume transactional DDL.
# MySQL
Context impl MySQLImpl.
Will assume non-transactional DDL.
```
Another question is that I need to migrate existing historical data from PostgreSQL to MySQL. Is there a quick way to
GitHub link: https://github.com/apache/superset/discussions/41354
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Why do the tables generated in the `superset` database differ depending on the `db_type` when using version 6.1.0? [superset]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
GitHub user dosubot[bot] added a comment to the discussion: Why do the tables generated in the `superset` database differ depending on the `db_type` when using version 6.1.0?
<!-- Answer -->
The missing tabs in Settings are **not caused by using MySQL** — they're controlled by **feature flags** in your `superset_config.py`. Several Settings menu items are hidden by default unless you explicitly enable their corresponding feature flags.
Key feature flags that control Settings tabs visibility [[1]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/initialization/__init__.py):
| Settings Item | Feature Flag |
|---|---|
| Tags | `TAGGING_SYSTEM` |
| Alerts & Reports | `ALERT_REPORTS` |
| Plugins | `DYNAMIC_PLUGINS` |
Additionally, configuration flags control the Security section items [[2]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/initialization/__init__.py#L289-L300):
- `SUPERSET_SECURITY_VIEW_MENU` → List Users, List Roles, List Groups
- `FAB_ADD_SECURITY_VIEWS` + `SUPERSET_LOG_VIEW` → Action Log
If these tabs are present in your PostgreSQL installation but missing in MySQL, it's almost certainly because your PostgreSQL instance has a `superset_config.py` with these feature flags enabled, while your fresh MySQL installation is using defaults.
To fix this, add the relevant flags to your `superset_config.py`:
```python
FEATURE_FLAGS = {
"TAGGING_SYSTEM": True,
"ALERT_REPORTS": True,
# add others as needed
}
```
Then restart Superset. The tabs should appear regardless of database backend.
<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=reply-with-mention&utm_term=apache%2Fsuperset).*
---
Share context across your team and agents. Try [Dosu](https://dosu.dev?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-tagline&utm_term=apache%2Fsuperset).
[](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-feedback&utm_term=apache%2Fsuperset&message_id=93925263-e4d6-4fa7-b8de-35dd11359086) [](https://github.dosu.com/apache/superset?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-learn-repo&utm_term=apache%2Fsuperset) [](https://app.dosu.dev/signup?referrer=openSource&source=github-footer&utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-add-team&utm_term=apache%2Fsuperset)
GitHub link: https://github.com/apache/superset/discussions/41354#discussioncomment-17416967
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Why do the tables generated in the `superset` database differ depending on the `db_type` when using version 6.1.0? [superset]
Posted by "A-little-bit-of-data (via GitHub)" <gi...@apache.org>.
GitHub user A-little-bit-of-data added a comment to the discussion: Why do the tables generated in the `superset` database differ depending on the `db_type` when using version 6.1.0?
Apart from using an external MySQL database, I haven't made any other changes. Regarding your comment—that the issue is almost certainly due to the PostgreSQL instance using a `superset_config.py` file with specific feature flags enabled, whereas the new MySQL installation uses default settings—could you specify exactly which features these are? I’d like to compare the configurations between the two environments.
GitHub link: https://github.com/apache/superset/discussions/41354#discussioncomment-17417307
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Why do the tables generated in the `superset` database differ depending on the `db_type` when using version 6.1.0? [superset]
Posted by "A-little-bit-of-data (via GitHub)" <gi...@apache.org>.
GitHub user A-little-bit-of-data added a comment to the discussion: Why do the tables generated in the `superset` database differ depending on the `db_type` when using version 6.1.0?
I cleared the Superset database in MySQL and performed a completely fresh installation. The number of tables and the table names now match, but in the settings section—using MySQL 6.1.0—several tabs are missing (specifically the ones on the far right). What is causing this issue?
<img width="663" height="840" alt="preview" src="https://github.com/user-attachments/assets/0e89e6c7-eac7-4500-b9c2-1c9c819edca3" />
GitHub link: https://github.com/apache/superset/discussions/41354#discussioncomment-17416910
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Why do the tables generated in the `superset` database differ depending on the `db_type` when using version 6.1.0? [superset]
Posted by "FrancescoCastaldi (via GitHub)" <gi...@apache.org>.
GitHub user FrancescoCastaldi added a comment to the discussion: Why do the tables generated in the `superset` database differ depending on the `db_type` when using version 6.1.0?
The difference in table count (53 tables on PostgreSQL vs 60 tables on MySQL) is **not** caused by different database dialects. It is caused by **migration history differences** between a clean database initialization versus a database that ran through legacy migrations.
### Why MySQL has extra tables
The extra tables present in your MySQL instance (`clusters`, `datasources`, `columns`, `metrics`, `druiddatasource_user`, `dashboard_email_schedules`, `slice_email_schedules`, `alerts`, `alert_logs`) are legacy tables from Superset 1.x through 3.x:
- `clusters`, `datasources`, `druiddatasource_user`: belonged to the legacy Druid connector that was deprecated in SIP-38 and removed.
- `dashboard_email_schedules`, `alerts`: belonged to the old email reporting system that was superseded by `report_schedule` and `report_execution_log`.
When Superset deprecates features, Alembic migrations deliberately avoid destructive `DROP TABLE` statements on production databases to prevent accidental data loss. As a result, an older database upgraded over time retains these legacy tables. A freshly initialized PostgreSQL database created directly on modern Superset only creates currently active models.
### Does this impact daily use?
**No, there is zero impact on daily operations.** The active SQLAlchemy ORM models in Superset 6.1.0 strictly query the 53 core tables. The remaining legacy tables in MySQL are completely inert and ignored by the application.
### Recommended Method to Migrate Historical Data from PostgreSQL to MySQL
Do not attempt raw SQL table dumps between PostgreSQL and MySQL because column data types (such as `BOOLEAN` vs `TINYINT`, `JSONB` vs `JSON/LONGTEXT`, and datetime microseconds) differ between the two engines.
The cleanest and officially supported migration paths are:
1. **Metadata Export/Import (Zero Risk):**
Use the Superset CLI to export your assets to portable YAML bundles:
```bash
superset export-dashboards -f dashboards.zip
superset export-datasources -f datasources.zip
```
Import them into your target instance using `superset import-dashboards` and `superset import-datasources`. This re-hydrates all charts, dashboards, and datasets cleanly into the destination schema.
2. **Using pgloader (Direct Database Migration):**
If migrating the entire database directly, use `pgloader` with a command file configured to map types cleanly, excluding the legacy tables mentioned above.
GitHub link: https://github.com/apache/superset/discussions/41354#discussioncomment-19164793
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Why do the tables generated in the `superset` database differ depending on the `db_type` when using version 6.1.0? [superset]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
GitHub user dosubot[bot] added a comment to the discussion: Why do the tables generated in the `superset` database differ depending on the `db_type` when using version 6.1.0?
<!-- Answer -->
Here are the **default values** for all feature flags that affect Settings menu visibility — they are all **`False` (disabled) by default** in a vanilla Superset installation [[1]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/config.py#L466-L582):
| Feature Flag | Default | Controls |
|---|---|---|
| `TAGGING_SYSTEM` | `False` | "Tags" tab in Settings → Manage |
| `ALERT_REPORTS` | `False` | "Alerts & Reports" tab in Settings → Manage |
| `DYNAMIC_PLUGINS` | `False` | "Plugins" tab in Settings → Manage |
| `EMBEDDED_SUPERSET` | `False` | Embedded dashboard functionality |
| `SSH_TUNNELING` | `False` | SSH tunnel options in database connections |
If **both** your PostgreSQL and MySQL installations use default settings with no custom `superset_config.py`, they should show the same tabs. The fact that your PostgreSQL setup shows more tabs suggests something is enabling those flags — likely through:
1. **Your deployment method** — Docker images, Helm charts, or managed distributions often ship a pre-configured `superset_config.py` or set environment variables like `SUPERSET_FEATURE_TAGGING_SYSTEM=true` [[2]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/config.py#L607-L614)
2. **Environment variables** — Any env var prefixed with `SUPERSET_FEATURE_` will override the default flags at runtime
To compare, check:
- The `superset_config.py` file in your PostgreSQL deployment (look for a `FEATURE_FLAGS` dictionary)
- Environment variables in both environments: `env | grep SUPERSET_FEATURE_`
If your PostgreSQL deployment uses Docker, check the `docker-compose.yml` or Dockerfile for any config files being mounted or env vars being set — that's likely where the difference lives.
<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=reply-with-mention&utm_term=apache%2Fsuperset).*
---
Docs are dead. Just use [Dosu](https://dosu.dev?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=static-docs-tagline&utm_term=apache%2Fsuperset).
[](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=static-docs-feedback&utm_term=apache%2Fsuperset&message_id=78976895-edb9-4114-8955-d1dc5d7a3f63) [](https://github.dosu.com/apache/superset?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=static-docs-ask-repo&utm_term=apache%2Fsuperset) [](https://app.dosu.dev/signup?referrer=openSource&source=github-footer&utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=static-docs-share-team&utm_term=apache%2Fsuperset)
GitHub link: https://github.com/apache/superset/discussions/41354#discussioncomment-17417323
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Why do the tables generated in the `superset` database differ depending on the `db_type` when using version 6.1.0? [superset]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
GitHub user dosubot[bot] added a comment to the discussion: Why do the tables generated in the `superset` database differ depending on the `db_type` when using version 6.1.0?
<!-- Answer -->
The table differences you're seeing are **not caused by using different database types** (PostgreSQL vs MySQL). They're caused by **migration history differences** — specifically, how far along the Alembic migration chain each database has been migrated.
**Your MySQL instance appears to be an older installation** that still contains legacy tables that were dropped in 2023–2024 migrations:
- `access_request`, `alert_logs`, `alert_owner`, `alerts`, `sql_observations`, `columns`, `metrics`, `datasources`, `clusters`, `druiddatasource_user`, `dashboard_email_schedules`, `slice_email_schedules`, `url` — these were all removed in cleanup migrations [[1]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/migrations/versions/2023-07-07_20-06_f92a3124dd66_drop_rouge_constraints_and_tables.py) [[2]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/migrations/versions/2023-06-01_13-13_83e1abbe777f_drop_access_request.py) [[3]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/migrations/versions/2024-01-19_10-03_e863403c0c50_drop_url_table.py)
**Your PostgreSQL instance appears to be a fresh or fully-migrated installation** that has the newer tables (`ssh_tunnels`, `key_value`, `embedded_dashboards`, `database_user_oauth2_tokens`, `user_favorite_tag`, etc.) added in 2022–2024 migrations [[4]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/migrations/versions/2022-10-20_10-48_f3c2d8ec8595_create_ssh_tunnel_credentials_tbl.py) [[5]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/migrations/versions/2022-03-04_09-59_6766938c6065_add_key_value_store.py) [[6]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/migrations/versions/2024-03-20_16-02_678eefb4ab44_add_access_token_table.py).
Even though `superset db heads` and `superset db current` show the same revision hash, it's possible your MySQL database was brought to head from a much older starting point where some migrations didn't execute cleanly (MySQL uses **non-transactional DDL**, meaning failed migrations can leave partial state). Superset uses a single unified migration path — there are no database-type-specific branches [[7]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/migrations/README).
**Impact on daily use**: If both databases report the same head revision and Superset starts without errors, it should function normally. The leftover legacy tables in MySQL are just orphaned — they aren't used by current code. The missing newer tables in MySQL would be a problem if they're actually missing (features like SSH tunnels, embedded dashboards, etc. wouldn't work).
**Regarding migrating from PostgreSQL to MySQL**: There is no officially supported tool for this. The recommended approach would be:
1. Set up a fresh MySQL instance and run `superset db upgrade` from scratch
2. Use Superset's built-in **dashboard/chart JSON export/import** to transfer your dashboards and charts
3. Manually recreate database connections and user/role configurations
4. Do **not** attempt a raw database dump-and-restore across dialects — data types, JSON handling, and text column types differ between PostgreSQL and MySQL [[8]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/migrations/versions/2019-05-06_14-30_afc69274c25a_alter_sql_column_data_type_in_query_mysql_table.py) [[9]](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/migrations/versions/2024-05-09_19-19_f7b6750b67e8_change_mediumtext_to_longtext.py)
I'd recommend verifying your MySQL installation by checking whether the newer tables actually exist — if they don't, you may need to troubleshoot your migration history.
<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=reply-with-mention&utm_term=apache%2Fsuperset).*
---
Share context across your team and agents. Try [Dosu](https://dosu.dev?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-tagline&utm_term=apache%2Fsuperset).
[](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-feedback&utm_term=apache%2Fsuperset&message_id=3cf0e07a-ebb3-4072-b33b-e94df6275a0c) [](https://github.dosu.com/apache/superset?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-learn-repo&utm_term=apache%2Fsuperset) [](https://app.dosu.dev/signup?referrer=openSource&source=github-footer&utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-add-team&utm_term=apache%2Fsuperset)
GitHub link: https://github.com/apache/superset/discussions/41354#discussioncomment-17416603
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org