You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "tien238lnd (via GitHub)" <gi...@apache.org> on 2026/10/10 18:26:29 UTC
[PR] fix(datasets): list datasets through the dataset API when semantic layers are off [superset]
tien238lnd opened a new pull request, #45176:
URL: https://github.com/apache/superset/pull/45176
### SUMMARY
Since #37815 the Datasets list always loads its rows from `GET /api/v1/datasource/`, the combined endpoint that lists semantic views next to datasets, even when `SEMANTIC_LAYERS` is off. When the flag is off every row is a dataset, so the combined endpoint adds nothing, and it has been the source of several regressions on this page (#45174): filters that are not applied, and a read gate that differed from `/api/v1/dataset/`.
This makes the page call `/api/v1/dataset/` when `SEMANTIC_LAYERS` is off, and the combined endpoint only when it is on. It is the same gate `DatabaseList` (`showSemanticLayers`) and `DatasetSelect` (`useSemanticLayers`) already use, and it is what 6.1.0 did for everyone.
Request and response shapes are compatible: the page reads `kind`, `sql`, `rls_filters`, `editors`, `changed_by*` and the optional `source_type`, all of which `/api/v1/dataset/` returns.
With the flag on nothing changes; the filters of that path are handled in #45175.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
No visual change. Network panel with `SEMANTIC_LAYERS` off: the list request goes to `/api/v1/dataset/?q=...` instead of `/api/v1/datasource/?q=...`, and the Editor, Modified by, Schema and Certified filters narrow the list.
### TESTING INSTRUCTIONS
```bash
cd superset-frontend
npm run test -- src/pages/DatasetList
```
Two new tests in `DatasetList.test.tsx` assert which endpoint the list uses with the flag off and on. The assertions in the other `DatasetList.*.test.tsx` files that counted calls to the combined endpoint now count calls to the dataset endpoint, since that is what the default (flag off) page calls.
Manual: with `SEMANTIC_LAYERS` off, open the Datasets list and apply the Editor, Modified by and Schema filters.
### ADDITIONAL INFORMATION
- [x] Has associated issue: part of #45174
- [x] Required feature flags: `SEMANTIC_LAYERS` (behavior with it on is unchanged)
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
### Question for reviewers
cc @betodealmeida (combined endpoint) and @mikebridge (reviewed #43887): is routing the page by the feature flag the intended direction, or is the combined endpoint meant to be the single list endpoint for this page even without semantic layers? If the latter, #45175 makes it filter correctly and this PR is not needed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(datasets): list datasets through the dataset API when semantic layers are off [superset]
Posted by "tien238lnd (via GitHub)" <gi...@apache.org>.
tien238lnd commented on PR #45176:
URL: https://github.com/apache/superset/pull/45176#issuecomment-6196674518
Addressed @mikebridge's review in the latest push: with `SEMANTIC_LAYERS` off, a saved `source_type` is no longer sent to `/api/v1/dataset/`, and a saved `sl:` connection is neither sent as `semantic_layer_uuid` nor kept in the filter state (it is cleared through the same `clearFilterById` the cascade uses), so a rolled-back deployment does not leave the list stuck on a rejected request. Two regression tests mount the page with those saved URLs (the render helper now accepts initial route entries); both failed before the change, all 7 DatasetList suites / 132 tests pass. The OpenAPI drift check was master being red on #45071 and is green again; the Jest shard 3 failure is `AddSemanticViewModal` (not touched here).
_🤖 Addressed by [Claude Code](https://claude.com/claude-code)_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(datasets): list datasets through the dataset API when semantic layers are off [superset]
Posted by "netlify[bot] (via GitHub) via notifications" <no...@superset.apache.org>.
netlify[bot] commented on PR #45176:
URL: https://github.com/apache/superset/pull/45176#issuecomment-6576243902
### <span aria-hidden="true">✅</span> Deploy Preview for *superset-docs-preview* ready!
| Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | b9c3b275c5b447745c64746d8ce155878376cb7e |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/projects/superset-docs-preview/deploys/6ae87640bea2d20118fb11fa |
|<span aria-hidden="true">😎</span> Deploy Preview | [https://deploy-preview-45176--superset-docs-preview.netlify.app](https://deploy-preview-45176--superset-docs-preview.netlify.app) |
|<span aria-hidden="true">📱</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br /><br /><br />_Use your smartphone camera to open QR code link._</details> |
|<span aria-hidden="true">🤖</span> Make changes | [Run an agent on this branch](https://app.netlify.com/projects/superset-docs-preview/agent-runs#q1YqKMrPLShRslJS0lFKKkrMS84AstMyK_RTEksSi1NLinVzMotLdKE83cSCTKDClNSCnPxKzxSgUrPEVAtzMxODpNREoxQjA0NDi7QkQ8O0RKVaAA) |
---
<!-- [superset-docs-preview Preview](https://deploy-preview-45176--superset-docs-preview.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify project configuration](https://app.netlify.com/projects/superset-docs-preview/configuration/notifications#deploy-notifications)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(datasets): list datasets through the dataset API when semantic layers are off [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on PR #45176:
URL: https://github.com/apache/superset/pull/45176#issuecomment-6100842877
## CodeAnt PR Risk: Low Risk
- The PR appears safe to merge; dataset listing uses the canonical dataset API when semantic layers are disabled and the combined datasource API when enabled.
- Tests cover endpoint selection for both feature-flag states, alongside updates to existing dataset-list request assertions.
**Assessed commit:** `02f40ab96841`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(datasets): list datasets through the dataset API when semantic layers are off [superset]
Posted by "tien238lnd (via GitHub) via notifications" <no...@superset.apache.org>.
tien238lnd commented on code in PR #45176:
URL: https://github.com/apache/superset/pull/45176#discussion_r4441910609
##########
superset-frontend/src/pages/DatasetList/index.tsx:
##########
@@ -517,6 +517,10 @@ const DatasetList: FunctionComponent<DatasetListProps> = ({
setLastFetchConfig(config);
setLoading(true);
const { pageIndex, pageSize, sortBy, filters: filterValues } = config;
+ // The Source control and the semantic-layer connections only exist with
+ // SEMANTIC_LAYERS on, but a saved URL can still carry them. The dataset API
+ // rejects both, and with the control hidden the user could not clear them.
+ const semanticLayersEnabled = isFeatureEnabled(SEMANTIC_LAYERS_FLAG);
Review Comment:
Done in b9c3b275c5: The saved `Type = Semantic View` is now dropped from the request when `SEMANTIC_LAYERS` is off and cleared from the filter state, like the `sl:` connection. A third saved-URL test mounts the page with `sql:semantic_view` and asserts nothing reaches `/api/v1/dataset/` (it failed before the change).
_🤖 Addressed by [Claude Code](https://claude.com/claude-code)_
##########
superset-frontend/src/pages/DatasetList/index.tsx:
##########
@@ -542,7 +546,7 @@ const DatasetList: FunctionComponent<DatasetListProps> = ({
sourceTypeFilter?.value && typeof sourceTypeFilter.value === 'object'
? (sourceTypeFilter.value as { value: string }).value
: (sourceTypeFilter?.value as string | undefined);
- if (sourceTypeValue) {
+ if (sourceTypeValue && semanticLayersEnabled) {
Review Comment:
Done in b9c3b275c5: `currentSourceFilter` now returns `''` when `SEMANTIC_LAYERS` is off, so a saved Source no longer empties the Database options. The saved-Source test now also opens the Database control and picks a database, and asserts the dataset request carries the `database` filter (it could not find any option before the change).
_🤖 Addressed by [Claude Code](https://claude.com/claude-code)_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(datasets): list datasets through the dataset API when semantic layers are off [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #45176:
URL: https://github.com/apache/superset/pull/45176#issuecomment-6100867839
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/45176?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 82.85%. Comparing base ([`b9e8188`](https://app.codecov.io/gh/apache/superset/commit/b9e8188ad606f61317403c3be5ef9450dbe0e381?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`02f40ab`](https://app.codecov.io/gh/apache/superset/commit/02f40ab968417a16350efb1c1aa866d0bff2474a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #45176 +/- ##
=======================================
Coverage 82.85% 82.85%
=======================================
Files 3017 3017
Lines 192923 192925 +2
Branches 44976 44978 +2
=======================================
+ Hits 159841 159843 +2
Misses 30042 30042
Partials 3040 3040
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/45176/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [javascript](https://app.codecov.io/gh/apache/superset/pull/45176/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `78.60% <100.00%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/45176?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(datasets): list datasets through the dataset API when semantic layers are off [superset]
Posted by "EnxDev (via GitHub)" <gi...@apache.org>.
EnxDev commented on code in PR #45176:
URL: https://github.com/apache/superset/pull/45176#discussion_r4256175798
##########
superset-frontend/src/pages/DatasetList/DatasetList.test.tsx:
##########
@@ -650,3 +636,36 @@ test.each([false, true])(
}
},
);
+
+const listRequestUrls = (pattern: RegExp) =>
+ fetchMock.callHistory
+ .calls()
+ .map(call => call.url)
+ .filter(url => pattern.test(url));
+
+test('lists datasets through the dataset API when semantic layers are off', async () => {
+ renderDatasetList(mockAdminUser);
+
+ await waitFor(() => {
+ expect(listRequestUrls(/\/api\/v1\/dataset\/\?q=/).length).toBeGreaterThan(
+ 0,
+ );
+ });
+ expect(listRequestUrls(/\/api\/v1\/datasource\/\?q=/)).toHaveLength(0);
+});
+
+test('lists datasets through the combined datasource API when semantic layers are on', async () => {
Review Comment:
After the rename, this is the only test that runs the page against the combined endpoint. Sorting, filtering and the refresh after delete or duplicate are now only asserted on the dataset endpoint.
Worth keeping one or two of those running with the flag on? That path is where the regressions in #45174 came from.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(datasets): list datasets through the dataset API when semantic layers are off [superset]
Posted by "tien238lnd (via GitHub)" <gi...@apache.org>.
tien238lnd commented on code in PR #45176:
URL: https://github.com/apache/superset/pull/45176#discussion_r4267292378
##########
superset-frontend/src/pages/DatasetList/DatasetList.test.tsx:
##########
@@ -650,3 +636,36 @@ test.each([false, true])(
}
},
);
+
+const listRequestUrls = (pattern: RegExp) =>
+ fetchMock.callHistory
+ .calls()
+ .map(call => call.url)
+ .filter(url => pattern.test(url));
+
+test('lists datasets through the dataset API when semantic layers are off', async () => {
+ renderDatasetList(mockAdminUser);
+
+ await waitFor(() => {
+ expect(listRequestUrls(/\/api\/v1\/dataset\/\?q=/).length).toBeGreaterThan(
+ 0,
+ );
+ });
+ expect(listRequestUrls(/\/api\/v1\/datasource\/\?q=/)).toHaveLength(0);
+});
+
+test('lists datasets through the combined datasource API when semantic layers are on', async () => {
Review Comment:
Added in the latest push: two tests that run with `SEMANTIC_LAYERS` on against `/api/v1/datasource/`, one for the name filter reaching the combined endpoint and one for sorting (`order_column=table_name`), both asserting nothing goes to `/api/v1/dataset/`. The refresh after delete or duplicate stays asserted on the dataset endpoint only, since the flag-on path of that refresh goes through the same `fetchData`.
_🤖 Addressed by [Claude Code](https://claude.com/claude-code)_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(datasets): list datasets through the dataset API when semantic layers are off [superset]
Posted by "EnxDev (via GitHub)" <gi...@apache.org>.
EnxDev commented on code in PR #45176:
URL: https://github.com/apache/superset/pull/45176#discussion_r4403227123
##########
superset-frontend/src/pages/DatasetList/index.tsx:
##########
@@ -542,7 +546,7 @@ const DatasetList: FunctionComponent<DatasetListProps> = ({
sourceTypeFilter?.value && typeof sourceTypeFilter.value === 'object'
? (sourceTypeFilter.value as { value: string }).value
: (sourceTypeFilter?.value as string | undefined);
- if (sourceTypeValue) {
+ if (sourceTypeValue && semanticLayersEnabled) {
Review Comment:
This keeps `source_type` out of the request, but it stays in `lastFetchConfig.filters`, so `currentSourceFilter` still reads `semantic_layer`. In `fetchConnectionOptions` that sets `showDatabases` to false, and with the flag off `showSemanticLayers` is false too. The Database dropdown opens empty until some other filter change drops the URL value.
`clearFilterById('source_type')` can't help here, since there's no configured filter with that id when the flag is off. Could `currentSourceFilter` return `''` when `SEMANTIC_LAYERS` is off? The existing saved-URL test could also assert that the Database select still lists databases.
##########
superset-frontend/src/pages/DatasetList/index.tsx:
##########
@@ -517,6 +517,10 @@ const DatasetList: FunctionComponent<DatasetListProps> = ({
setLastFetchConfig(config);
setLoading(true);
const { pageIndex, pageSize, sortBy, filters: filterValues } = config;
+ // The Source control and the semantic-layer connections only exist with
+ // SEMANTIC_LAYERS on, but a saved URL can still carry them. The dataset API
+ // rejects both, and with the control hidden the user could not clear them.
+ const semanticLayersEnabled = isFeatureEnabled(SEMANTIC_LAYERS_FLAG);
Review Comment:
A saved `Type = Semantic View` (`sql:semantic_view`) also reaches this point with the flag off, and it goes straight through `otherFilters` to `/api/v1/dataset/`. `DatasetIsNullOrEmptyFilter` reads the truthy string as `true`, so I'd expect the list to show physical datasets under a Type pill whose value isn't one of the two options.
Might be misreading how the select renders an unknown value, but it seems worth dropping `semantic_view` here like the `sl:` connection, and covering it with a third saved-URL test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org