You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "sadpandajoe (via GitHub)" <gi...@apache.org> on 2026/08/18 23:17:59 UTC
[PR] fix(dataset): derive virtual dataset schema from SQL Lab query [superset]
sadpandajoe opened a new pull request, #43313:
URL: https://github.com/apache/superset/pull/43313
### SUMMARY
When a virtual dataset is created from SQL Lab's "Save & Explore" action, the saved
dataset's `schema` is always set to whatever schema is selected in the SQL Lab
dropdown — even when the query itself explicitly references a different schema. For
example, selecting `information_schema` in the dropdown and then running
`select * from public."Vehicle Sales"` creates a dataset with `schema=information_schema`,
not `schema=public`. This has been reported since 2021 (#16791) and affects dataset
metadata correctness, RLS resolution, and permission checks that read the dataset's
stored `schema`.
This PR derives the schema from the query itself when it is unambiguous, and falls
back to today's dropdown-based behavior in every other case:
- The query is parsed with the existing Jinja-aware SQL parser (`process_jinja_sql`,
the same one already used by the access-control check on this path) — no new SQL
parsing is introduced.
- The dropdown schema is replaced with the query's schema **only** when the query
references exactly one `(catalog, schema)` location, every referenced table is
explicitly schema-qualified (zero unqualified references), the statement doesn't
mutate data, doesn't change the default schema mid-script, doesn't contain a
metadata (`SHOW ...`) statement, and doesn't reference any quoted catalog/schema
identifier (conservative, since quoting can carry case-sensitive identity on some
engines).
- Any other case — ambiguous, multi-schema, or unparseable queries — keeps today's
behavior unchanged. This is an intentional, documented limitation, not an oversight.
- Any failure in the optional parsing/derivation step (including malformed
`template_params` or a Jinja rendering error) safely falls back to the submitted
schema rather than blocking dataset creation.
Out of scope for this PR (see `RCA.md` for details): the broader multi-schema dataset
model proposed in community discussion, and permission/RLS enforcement based on the
derived schema — both would be separate, larger changes.
`RCA.md` in this PR documents the full investigation: root cause, why it wasn't
caught, the exact safety conditions and why each is needed, a verified matrix of
parser behavior across ~20 query shapes, and known limitations of this fix.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Backend-only change (dataset creation logic); no UI change. Behavior before/after for
the reported repro:
- **Before:** dropdown schema `information_schema` + query `select * from public."Vehicle Sales"` → saved dataset `schema` = `information_schema` (wrong)
- **After:** same inputs → saved dataset `schema` = `public` (matches the query)
### TESTING INSTRUCTIONS
1. `pytest tests/unit_tests/commands/dataset/test_create.py` — covers the primary
repro case plus every fallback condition (zero-schema, multi-schema, unqualified
reference, mutating statement, metadata statement, quoted identifier, unparseable/
template-error SQL, Jinja partition-macro tables, template-parameter-driven schema
selection, and positive catalog+schema derivation).
2. Manual repro: in SQL Lab, select a database and a schema in the dropdowns, run a
query that explicitly references a *different* schema (e.g.
`select * from <other_schema>.<table>`), click Explore → name the dataset → Save &
Explore. Confirm the saved dataset's schema matches the query, not the dropdown.
### ADDITIONAL INFORMATION
- [x] Has associated issue: Fixes #16791
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(dataset): derive virtual dataset schema from SQL Lab query [superset]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #43313:
URL: https://github.com/apache/superset/pull/43313#issuecomment-5361118550
🎪 [Showtime](https://github.com/mistercrunch/superset-showtime) deployed environment on [GHA](https://github.com/apache/superset/actions/runs/32411221728) for [fe676c7](https://github.com/apache/superset/commit/fe676c7)
• **Environment:** http://35.87.109.234:8080 (admin/admin)
• **Lifetime:** 48h auto-cleanup
• **Updates:** New commits create fresh environments automatically
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(dataset): derive virtual dataset schema from SQL Lab query [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43313:
URL: https://github.com/apache/superset/pull/43313#issuecomment-5363049905
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(dataset): derive virtual dataset schema from SQL Lab query [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43313:
URL: https://github.com/apache/superset/pull/43313#issuecomment-5335402546
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(dataset): derive virtual dataset schema from SQL Lab query [superset]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #43313:
URL: https://github.com/apache/superset/pull/43313#issuecomment-5360966384
🎪 [Showtime](https://github.com/mistercrunch/superset-showtime) is building environment on [GHA](https://github.com/apache/superset/actions/runs/32411221728) for [fe676c7](https://github.com/apache/superset/commit/fe676c7)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(dataset): derive virtual dataset schema from SQL Lab query [superset]
Posted by "netlify[bot] (via GitHub)" <gi...@apache.org>.
netlify[bot] commented on PR #43313:
URL: https://github.com/apache/superset/pull/43313#issuecomment-5348202958
### <span aria-hidden="true">✅</span> Deploy Preview for *superset-docs-preview* ready!
| Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | d86607a875613f9e8eda16fdcf4a1cfa65e9fa51 |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/projects/superset-docs-preview/deploys/6a861f42c92e6a000895f74a |
|<span aria-hidden="true">😎</span> Deploy Preview | [https://deploy-preview-43313--superset-docs-preview.netlify.app](https://deploy-preview-43313--superset-docs-preview.netlify.app) |
|<span aria-hidden="true">📱</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br /><br /><br />_Use your smartphone camera to open QR code link._</details> |
|<span aria-hidden="true">🤖</span> Make changes | [Run an agent on this branch](https://app.netlify.com/projects/superset-docs-preview/agent-runs#DctbDoMgEEbhvcyzk1CjVLsDl_HLJZJARRhNG9O9l7fz8J2bctlTFnoRdbQWvM3W2ocP1yNGrHyFIiciWwiqE65mcwmcQk2QhjuyLsf9u9j2aUz64YfezL3TUEpN8-ifA-j3Bw) |
---
<!-- [superset-docs-preview Preview](https://deploy-preview-43313--superset-docs-preview.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify project configuration](https://app.netlify.com/projects/superset-docs-preview/configuration/notifications#deploy-notifications)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(dataset): derive virtual dataset schema from SQL Lab query [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on PR #43313:
URL: https://github.com/apache/superset/pull/43313#issuecomment-5546203358
This is a nice, careful fix, the fallback conditions and test matrix are thorough. I'd approve it, but wondering if we need `RCA.md` checked into the repo root though? Feels like investigation notes that belong in the PR description rather than a permanent file in the tree.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(dataset): derive virtual dataset schema from SQL Lab query [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas merged PR #43313:
URL: https://github.com/apache/superset/pull/43313
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(dataset): derive virtual dataset schema from SQL Lab query [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on PR #43313:
URL: https://github.com/apache/superset/pull/43313#issuecomment-5607502924
Good call — moved the RCA notes out of the repo and folded them into the PR description (collapsible section) instead of keeping `RCA.md` in the tree. Thanks for the review!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(dataset): derive virtual dataset schema from SQL Lab query [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #43313:
URL: https://github.com/apache/superset/pull/43313#issuecomment-5335433017
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/43313?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `16.00000%` with `42 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 57.25%. Comparing base ([`34cd50c`](https://app.codecov.io/gh/apache/superset/commit/34cd50cc4840d72636f5d33c196ba3e0fcb8a7db?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`a322256`](https://app.codecov.io/gh/apache/superset/commit/a32225646cccbc52a15b94526f10bc2eb427477b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 2 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/superset/pull/43313?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [superset/commands/dataset/create.py](https://app.codecov.io/gh/apache/superset/pull/43313?src=pr&el=tree&filepath=superset%2Fcommands%2Fdataset%2Fcreate.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvY29tbWFuZHMvZGF0YXNldC9jcmVhdGUucHk=) | 6.06% | [31 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/43313?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [superset/sql/parse.py](https://app.codecov.io/gh/apache/superset/pull/43313?src=pr&el=tree&filepath=superset%2Fsql%2Fparse.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvc3FsL3BhcnNlLnB5) | 35.29% | [11 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/43313?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (34cd50c) and HEAD (a322256). Click for more details.
>
> <details><summary>HEAD has 9 uploads less than BASE</summary>
>
>| Flag | BASE (34cd50c) | HEAD (a322256) |
>|------|------|------|
>|hive|2|1|
>|python|9|2|
>|presto|2|1|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #43313 +/- ##
==========================================
- Coverage 66.66% 57.25% -9.42%
==========================================
Files 2876 2876
Lines 164007 164056 +49
Branches 37834 37848 +14
==========================================
- Hits 109341 93929 -15412
- Misses 52520 69234 +16714
+ Partials 2146 893 -1253
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/43313/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/43313/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.09% <16.00%> (-0.02%)` | :arrow_down: |
| [mysql](https://app.codecov.io/gh/apache/superset/pull/43313/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [postgres](https://app.codecov.io/gh/apache/superset/pull/43313/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [presto](https://app.codecov.io/gh/apache/superset/pull/43313/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `40.02% <16.00%> (-0.02%)` | :arrow_down: |
| [python](https://app.codecov.io/gh/apache/superset/pull/43313/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `40.08% <16.00%> (-19.10%)` | :arrow_down: |
| [sqlite](https://app.codecov.io/gh/apache/superset/pull/43313/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unit](https://app.codecov.io/gh/apache/superset/pull/43313/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/43313?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org