You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "msyavuz (via GitHub)" <gi...@apache.org> on 2026/08/25 15:34:05 UTC
[PR] feat(filters): search filter values server-side in Explore [superset]
msyavuz opened a new pull request, #43518:
URL: https://github.com/apache/superset/pull/43518
### SUMMARY
Explore's filter value picker fetched an unfiltered `SELECT DISTINCT col FROM tbl LIMIT 10000` once and matched it in the browser, so a value past that limit on a high-cardinality column simply could not be found. Dashboard native filters already search server-side via `searchAllOptions`; Explore had no equivalent.
`values_for_column` now takes a `search` term and applies a case-insensitive containment predicate with LIKE wildcards escaped (a typed `%` matches a literal percent instead of every row). The endpoint accepts `?q=`, includes it in the cache key, caps the TTL for searched results, and returns the applied `limit` so the client can tell a short list from a truncated one. The comparator control moves to `AsyncSelect`, which debounces and queries the server.
Three fixes to dashboard native filters in the same area:
- `allowNewOptions` was switched off whenever `searchAllOptions` was on, leaving no way to apply a value the bounded search page did not return — enabling search made things strictly worse.
- Searching sorted by the searched column, so the engine sorted every match before applying the row limit. Dropped while a search is active; a sort metric still selects which rows come back, so it stays.
- The term sent to the server is normalised the same way as the creatable option shown in the dropdown.
Both dropdowns now say when a list is capped, rather than a partial list reading as the whole set.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Before: typing a value beyond the row limit yields no match and no explanation.
After: the query reaches the database, and a capped list says so — "Only the first 10000 values are listed. Type to search all of them, or enter a value that is not listed."
### TESTING INSTRUCTIONS
Set `FILTER_SELECT_ROW_LIMIT = 5` and `NATIVE_FILTER_DEFAULT_ROW_LIMIT = 5` so any column truncates against the examples DB.
1. Explore, chart on `birth_names`, filter on `name` with `IN`. Five values plus the notice. Type `Michael` — a `/values/?q=Michael` request fires and the value is selectable. Search `%`: matches nothing, not everything.
2. Pick a value, close and reopen the popover, remove it. Works.
3. Dashboard native select filter on `name`: notice appears. With *Dynamically search all filter values* on, an unlisted value can still be entered.
`pytest tests/unit_tests/models/helpers_test.py`, `npm run test -- AdhocFilterEditPopoverSimpleTabContent`, `npm run test -- src/filters/components/Select`.
Not verified: the performance gain from dropping `ORDER BY` needs a real high-cardinality warehouse column; SQLite shows the behaviour but not the win. Substring search remains unsargable by nature.
### ADDITIONAL INFORMATION
- [ ] Has associated issue: No
- [ ] Required feature flags: None
- [x] Changes UI — dropdown gains a header line when the list is capped; Explore's comparator becomes async
- [ ] Includes DB Migration: No — no schema change
- [ ] Migration is atomic, supports rollback & is backwards-compatible: N/A
- [ ] Confirm DB migration upgrade and downgrade tested: N/A
- [ ] Runtime estimates and downtime expectations provided: N/A
- [x] Introduces new feature or API — optional `?q=` on the existing column-values endpoint; omitting it preserves current behaviour
- [ ] Removes existing feature or API: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(filters): search filter values server-side in Explore [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43518:
URL: https://github.com/apache/superset/pull/43518#issuecomment-5452923355
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(filters): search filter values server-side in Explore [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #43518:
URL: https://github.com/apache/superset/pull/43518#issuecomment-5451456154
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/43518?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `85.05747%` with `13 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 74.85%. Comparing base ([`a30e4a4`](https://app.codecov.io/gh/apache/superset/commit/a30e4a4350d01925853a6f3ea3b40f42a3b1a109?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`99bd6ff`](https://app.codecov.io/gh/apache/superset/commit/99bd6fff7708c6bd43701cbbd2ca2dee9ffc5f8b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/superset/pull/43518?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...l/AdhocFilterEditPopoverSimpleTabContent/index.tsx](https://app.codecov.io/gh/apache/superset/pull/43518?src=pr&el=tree&filepath=superset-frontend%2Fsrc%2Fexplore%2Fcomponents%2Fcontrols%2FFilterControl%2FAdhocFilterEditPopoverSimpleTabContent%2Findex.tsx&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL2V4cGxvcmUvY29tcG9uZW50cy9jb250cm9scy9GaWx0ZXJDb250cm9sL0FkaG9jRmlsdGVyRWRpdFBvcG92ZXJTaW1wbGVUYWJDb250ZW50L2luZGV4LnRzeA==) | 88.52% | [7 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/43518?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [superset/datasource/api.py](https://app.codecov.io/gh/apache/superset/pull/43518?src=pr&el=tree&filepath=superset%2Fdatasource%2Fapi.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvZGF0YXNvdXJjZS9hcGkucHk=) | 16.66% | [5 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/43518?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...c/filters/components/Select/SelectFilterPlugin.tsx](https://app.codecov.io/gh/apache/superset/pull/43518?src=pr&el=tree&filepath=superset-frontend%2Fsrc%2Ffilters%2Fcomponents%2FSelect%2FSelectFilterPlugin.tsx&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL2ZpbHRlcnMvY29tcG9uZW50cy9TZWxlY3QvU2VsZWN0RmlsdGVyUGx1Z2luLnRzeA==) | 90.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/43518?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #43518 +/- ##
==========================================
- Coverage 79.19% 74.85% -4.34%
==========================================
Files 2879 2879
Lines 165876 165915 +39
Branches 38355 38371 +16
==========================================
- Hits 131361 124199 -7162
- Misses 32038 39200 +7162
- Partials 2477 2516 +39
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/43518/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/43518/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.93% <28.57%> (-0.01%)` | :arrow_down: |
| [javascript](https://app.codecov.io/gh/apache/superset/pull/43518/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `74.66% <89.04%> (+0.03%)` | :arrow_up: |
| [mysql](https://app.codecov.io/gh/apache/superset/pull/43518/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [postgres](https://app.codecov.io/gh/apache/superset/pull/43518/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [presto](https://app.codecov.io/gh/apache/superset/pull/43518/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `39.85% <28.57%> (-0.01%)` | :arrow_down: |
| [python](https://app.codecov.io/gh/apache/superset/pull/43518/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `75.00% <64.28%> (-8.75%)` | :arrow_down: |
| [sqlite](https://app.codecov.io/gh/apache/superset/pull/43518/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unit](https://app.codecov.io/gh/apache/superset/pull/43518/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `73.96% <64.28%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/43518?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(filters): search filter values server-side in Explore [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on code in PR #43518:
URL: https://github.com/apache/superset/pull/43518#discussion_r3854754184
##########
tests/integration_tests/datasource/api_tests.py:
##########
@@ -170,6 +170,79 @@ def test_get_column_values_array_elements_param(self, values_for_column_mock):
)
assert values_for_column_mock.call_args.kwargs["array_elements"] is True
+ @pytest.mark.usefixtures("app_context", "virtual_dataset")
Review Comment:
<div>
<div id="suggestion">
<div id="issue"><b>Missing search kwarg assertion</b></div>
<div id="fix">
The endpoint always passes `search=search` to `values_for_column`. This test only asserts on `array_elements`; the `search=None` kwarg is silently present and untested. Add `assert values_for_column_mock.call_args.kwargs["search"] is None` to prevent the `assert_called_with` failing if the call signature is later asserted explicitly.
</div>
</div>
<small><i>Code Review Run #3b684a</i></small>
</div>
---
Should Bito avoid suggestions like this for future reviews? (<a href=https://alpha.bito.ai/home/ai-agents/review-rules>Manage Rules</a>)
- [ ] Yes, avoid them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(filters): search filter values server-side in Explore [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #43518:
URL: https://github.com/apache/superset/pull/43518#discussion_r3854656853
##########
superset-frontend/src/explore/components/controls/FilterControl/AdhocFilterEditPopoverSimpleTabContent/index.tsx:
##########
@@ -495,76 +561,89 @@ const AdhocFilterEditPopoverSimpleTabContent: FC<Props> = props => {
onChange: onDatePickerChange,
});
- useEffect(() => {
- const refreshComparatorSuggestions = () => {
- const { datasource } = props;
- const col = props.adhocFilter.subject;
- const having = props.adhocFilter.clause === Clauses.Having;
-
- if (col && datasource && datasource.filter_select && !having) {
- const controller = new AbortController();
- const { signal } = controller;
- if (loadingComparatorSuggestions) {
- controller.abort();
- }
- // Element-level array operators (Contains any / Contains all) search
- // inside the array, so suggest individual elements; whole-array
- // operators (=, In, …) keep the default distinct-array suggestions.
- const { operatorId } = props.adhocFilter;
- const arrayElements =
- operatorId === Operators.ContainsAny ||
- operatorId === Operators.ContainsAll;
- setLoadingComparatorSuggestions(true);
- SupersetClient.get({
- signal,
- endpoint: `/api/v1/datasource/${datasource.type}/${datasource.id}/column/${col}/values/${
- arrayElements ? '?array_elements=true' : ''
- }`,
- })
- .then(({ json }) => {
- setSuggestions(
- json.result.map((suggestion: unknown) => {
- // Complex column values arrive as JS arrays or objects: whole
- // arrays for MULTI_VALUE columns (e.g. [5, 6, 7]) and Map/Tuple
- // objects for nested-container columns (e.g. {"a": ["x","y"]}).
- // A raw array/object is neither a valid single-select value
- // (antd collapses an array to its first element) nor renderable
- // as a React child (an object throws). Render it as its literal
- // string, which is also exactly what the backend's
- // parse_array_literal expects for the whole-array operators.
- if (suggestion !== null && typeof suggestion === 'object') {
- const literal = JSON.stringify(suggestion);
- return { value: literal, label: literal };
- }
- return {
- value: suggestion as null | number | boolean | string,
- label: optionLabel(
- suggestion as null | number | boolean | string,
- ),
- };
- }),
- );
- setLoadingComparatorSuggestions(false);
- })
- .catch(() => {
- setSuggestions([]);
- setLoadingComparatorSuggestions(false);
- });
+ // Element-level array operators (Contains any / Contains all) search inside
+ // the array, so suggest individual elements; whole-array operators (=, In, …)
+ // keep the default distinct-array suggestions.
+ const arrayElements =
+ props.adhocFilter.operatorId === Operators.ContainsAny ||
+ props.adhocFilter.operatorId === Operators.ContainsAll;
+
+ // AsyncSelect throws away every loaded option when the identity of its
+ // `options` callback changes, so this depends on plain values rather than on
+ // `props.datasource`, whose identity the parent does not guarantee.
+ const datasourceType = props.datasource?.type;
+ const datasourceId = props.datasource?.id;
+
+ const loadComparatorOptions = useCallback(
+ async (search: string): Promise<SelectOptionsTypePage> => {
+ const col = subjectString;
+ if (!col || !canSuggestComparatorValues) {
+ return { data: [], totalCount: 0 };
}
- };
- if (!datePicker) {
- refreshComparatorSuggestions();
- }
- // loadingComparatorSuggestions intentionally omitted - set inside effect, would cause infinite loop
- // eslint-disable-next-line react-hooks/exhaustive-deps
- }, [
- props.adhocFilter.subject,
- props.adhocFilter.clause,
- props.adhocFilter.operatorId,
- props.datasource,
- datePicker,
- ]);
+ const params = new URLSearchParams();
+ if (arrayElements) {
+ params.set('array_elements', 'true');
+ }
+ if (search) {
+ params.set('q', search);
+ }
+ const query = params.toString();
+
+ try {
+ const { json } = await SupersetClient.get({
+ endpoint:
+ `/api/v1/datasource/${datasourceType}/${datasourceId}` +
+ `/column/${encodeURIComponent(col)}/values/${query ? `?${query}` : ''}`,
+ });
+ const data = json.result.map((suggestion: unknown) => {
+ // Complex column values arrive as JS arrays or objects: whole arrays
+ // for MULTI_VALUE columns (e.g. [5, 6, 7]) and Map/Tuple objects for
+ // nested-container columns (e.g. {"a": ["x","y"]}). A raw
+ // array/object is neither a valid single-select value (antd collapses
+ // an array to its first element) nor renderable as a React child (an
+ // object throws). Render it as its literal string, which is also
+ // exactly what the backend's parse_array_literal expects for the
+ // whole-array operators.
+ if (suggestion !== null && typeof suggestion === 'object') {
+ const literal = JSON.stringify(suggestion);
+ return { value: literal, label: literal };
+ }
+ return {
+ value: suggestion as null | number | boolean | string,
+ label: optionLabel(suggestion as null | number | boolean | string),
+ };
+ });
+
+ setLoadedOptionCount(data.length);
+ setOptionsTruncated(isDefined(json.limit) && data.length >= json.limit);
Review Comment:
**Suggestion:** An older request can resolve after a newer search and overwrite `loadedOptionCount` and `optionsTruncated`, even though `AsyncSelect` discards the stale option data. This can display the wrong truncation message for the current search. Track the requested search term and only apply these state updates when the response still matches it. [race condition]
<details>
<summary><b>Severity Level:</b> Minor 🧹</summary>
```mdx
- ⚠️ Explore comparator helper text can show stale counts.
- ⚠️ Truncation messaging may not match displayed search results.
```
</details>
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=7697716c732642dd978934b1b50f8072&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=7697716c732642dd978934b1b50f8072&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset-frontend/src/explore/components/controls/FilterControl/AdhocFilterEditPopoverSimpleTabContent/index.tsx
**Line:** 618:619
**Comment:**
*Race Condition: An older request can resolve after a newer search and overwrite `loadedOptionCount` and `optionsTruncated`, even though `AsyncSelect` discards the stale option data. This can display the wrong truncation message for the current search. Track the requested search term and only apply these state updates when the response still matches it.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43518&comment_hash=5e1cb45faa0eebf8a79782f2732fc8408d16f627ae5df5edc6d4ae883712cdd8&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43518&comment_hash=5e1cb45faa0eebf8a79782f2732fc8408d16f627ae5df5edc6d4ae883712cdd8&reaction=dislike'>👎</a>
##########
superset-frontend/src/explore/components/controls/FilterControl/AdhocFilterEditPopoverSimpleTabContent/index.tsx:
##########
@@ -495,76 +561,89 @@ const AdhocFilterEditPopoverSimpleTabContent: FC<Props> = props => {
onChange: onDatePickerChange,
});
- useEffect(() => {
- const refreshComparatorSuggestions = () => {
- const { datasource } = props;
- const col = props.adhocFilter.subject;
- const having = props.adhocFilter.clause === Clauses.Having;
-
- if (col && datasource && datasource.filter_select && !having) {
- const controller = new AbortController();
- const { signal } = controller;
- if (loadingComparatorSuggestions) {
- controller.abort();
- }
- // Element-level array operators (Contains any / Contains all) search
- // inside the array, so suggest individual elements; whole-array
- // operators (=, In, …) keep the default distinct-array suggestions.
- const { operatorId } = props.adhocFilter;
- const arrayElements =
- operatorId === Operators.ContainsAny ||
- operatorId === Operators.ContainsAll;
- setLoadingComparatorSuggestions(true);
- SupersetClient.get({
- signal,
- endpoint: `/api/v1/datasource/${datasource.type}/${datasource.id}/column/${col}/values/${
- arrayElements ? '?array_elements=true' : ''
- }`,
- })
- .then(({ json }) => {
- setSuggestions(
- json.result.map((suggestion: unknown) => {
- // Complex column values arrive as JS arrays or objects: whole
- // arrays for MULTI_VALUE columns (e.g. [5, 6, 7]) and Map/Tuple
- // objects for nested-container columns (e.g. {"a": ["x","y"]}).
- // A raw array/object is neither a valid single-select value
- // (antd collapses an array to its first element) nor renderable
- // as a React child (an object throws). Render it as its literal
- // string, which is also exactly what the backend's
- // parse_array_literal expects for the whole-array operators.
- if (suggestion !== null && typeof suggestion === 'object') {
- const literal = JSON.stringify(suggestion);
- return { value: literal, label: literal };
- }
- return {
- value: suggestion as null | number | boolean | string,
- label: optionLabel(
- suggestion as null | number | boolean | string,
- ),
- };
- }),
- );
- setLoadingComparatorSuggestions(false);
- })
- .catch(() => {
- setSuggestions([]);
- setLoadingComparatorSuggestions(false);
- });
+ // Element-level array operators (Contains any / Contains all) search inside
+ // the array, so suggest individual elements; whole-array operators (=, In, …)
+ // keep the default distinct-array suggestions.
+ const arrayElements =
+ props.adhocFilter.operatorId === Operators.ContainsAny ||
+ props.adhocFilter.operatorId === Operators.ContainsAll;
+
+ // AsyncSelect throws away every loaded option when the identity of its
+ // `options` callback changes, so this depends on plain values rather than on
+ // `props.datasource`, whose identity the parent does not guarantee.
+ const datasourceType = props.datasource?.type;
+ const datasourceId = props.datasource?.id;
+
+ const loadComparatorOptions = useCallback(
+ async (search: string): Promise<SelectOptionsTypePage> => {
+ const col = subjectString;
+ if (!col || !canSuggestComparatorValues) {
+ return { data: [], totalCount: 0 };
}
- };
- if (!datePicker) {
- refreshComparatorSuggestions();
- }
- // loadingComparatorSuggestions intentionally omitted - set inside effect, would cause infinite loop
- // eslint-disable-next-line react-hooks/exhaustive-deps
- }, [
- props.adhocFilter.subject,
- props.adhocFilter.clause,
- props.adhocFilter.operatorId,
- props.datasource,
- datePicker,
- ]);
+ const params = new URLSearchParams();
+ if (arrayElements) {
+ params.set('array_elements', 'true');
+ }
+ if (search) {
+ params.set('q', search);
+ }
+ const query = params.toString();
+
+ try {
+ const { json } = await SupersetClient.get({
+ endpoint:
+ `/api/v1/datasource/${datasourceType}/${datasourceId}` +
+ `/column/${encodeURIComponent(col)}/values/${query ? `?${query}` : ''}`,
+ });
+ const data = json.result.map((suggestion: unknown) => {
+ // Complex column values arrive as JS arrays or objects: whole arrays
+ // for MULTI_VALUE columns (e.g. [5, 6, 7]) and Map/Tuple objects for
+ // nested-container columns (e.g. {"a": ["x","y"]}). A raw
+ // array/object is neither a valid single-select value (antd collapses
+ // an array to its first element) nor renderable as a React child (an
+ // object throws). Render it as its literal string, which is also
+ // exactly what the backend's parse_array_literal expects for the
+ // whole-array operators.
+ if (suggestion !== null && typeof suggestion === 'object') {
+ const literal = JSON.stringify(suggestion);
+ return { value: literal, label: literal };
+ }
+ return {
+ value: suggestion as null | number | boolean | string,
+ label: optionLabel(suggestion as null | number | boolean | string),
+ };
+ });
+
+ setLoadedOptionCount(data.length);
+ setOptionsTruncated(isDefined(json.limit) && data.length >= json.limit);
+
+ // The count has to exceed what was returned. AsyncSelect treats
+ // `loaded >= totalCount` as "that is every value", sets allValuesLoaded
+ // and from then on serves searches by filtering the loaded page
+ // client-side -- which is the behaviour this whole change exists to
+ // replace. Pagination is held off by COMPARATOR_PAGE_SIZE instead.
+ return { data, totalCount: data.length + 1 };
+ } catch {
+ setLoadedOptionCount(0);
+ setOptionsTruncated(false);
+ return { data: [], totalCount: 0 };
+ }
+ },
+ [
+ subjectString,
+ canSuggestComparatorValues,
+ datasourceType,
+ datasourceId,
+ arrayElements,
+ ],
+ );
+
+ // Options are cached per search term inside AsyncSelect; a different column
+ // or a switch to element-level suggestions invalidates all of them.
+ useEffect(() => {
+ comparatorSelectRef.current?.clearCache();
+ }, [subjectString, arrayElements]);
Review Comment:
**Suggestion:** Changing the subject or array-element mode clears AsyncSelect's option cache but leaves `loadedOptionCount` and `optionsTruncated` unchanged until the next request completes. During that interval the new column displays the previous column's placeholder count and capped-list helper text. Reset both pieces of state when invalidating the cache. [stale reference]
<details>
<summary><b>Severity Level:</b> Minor 🧹</summary>
```mdx
- ⚠️ Explore filter shows the previous column's option count.
- ⚠️ Capped-list guidance can describe an unrelated column.
```
</details>
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=f650717ab82740c799bbedb159ac1d97&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=f650717ab82740c799bbedb159ac1d97&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset-frontend/src/explore/components/controls/FilterControl/AdhocFilterEditPopoverSimpleTabContent/index.tsx
**Line:** 644:646
**Comment:**
*Stale Reference: Changing the subject or array-element mode clears AsyncSelect's option cache but leaves `loadedOptionCount` and `optionsTruncated` unchanged until the next request completes. During that interval the new column displays the previous column's placeholder count and capped-list helper text. Reset both pieces of state when invalidating the cache.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43518&comment_hash=8c093acfcb72f7e5590be7b0b039ec065ea4eac1038a18df104924b6f8bb4375&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43518&comment_hash=8c093acfcb72f7e5590be7b0b039ec065ea4eac1038a18df104924b6f8bb4375&reaction=dislike'>👎</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(filters): search filter values server-side in Explore [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43518:
URL: https://github.com/apache/superset/pull/43518#issuecomment-5412822491
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(filters): search filter values server-side in Explore [superset]
Posted by "msyavuz (via GitHub)" <gi...@apache.org>.
msyavuz merged PR #43518:
URL: https://github.com/apache/superset/pull/43518
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(filters): search filter values server-side in Explore [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #43518:
URL: https://github.com/apache/superset/pull/43518#discussion_r3854655163
##########
superset/datasource/api.py:
##########
@@ -225,11 +242,15 @@ def get_column_values(
timeout = datasource.cache_timeout or app.config.get(
"CACHE_DEFAULT_TIMEOUT", 300
)
+ if search:
+ # Every distinct search term is its own key, so a few users typing
+ # would otherwise pin one entry per keystroke for the full timeout.
+ timeout = min(timeout, SEARCH_CACHE_TIMEOUT)
cache_manager.data_cache.set(cache_key, payload, timeout=timeout)
logger.debug(
"column-values cache MISS: uid=%s col=%s", datasource.uid, column_name
)
- response = self.response(200, result=payload)
+ response = self.response(200, result=payload, limit=row_limit)
Review Comment:
**Suggestion:** The endpoint now returns a `limit` property, but the OpenAPI response schema still documents only `result`. API consumers and generated clients will therefore receive a response field that is absent from the declared contract, preventing them from reliably discovering or typing the truncation metadata. Add `limit` to the documented 200 response schema. [api mismatch]
<details>
<summary><b>Severity Level:</b> Minor 🧹</summary>
```mdx
- ⚠️ Generated clients omit the runtime `limit` field.
- ⚠️ API consumers cannot rely on truncation metadata.
- ⚠️ OpenAPI documentation disagrees with endpoint responses.
```
</details>
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=1eefcbd358ca43b09e8986c55e4fcb91&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=1eefcbd358ca43b09e8986c55e4fcb91&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/datasource/api.py
**Line:** 253:253
**Comment:**
*Api Mismatch: The endpoint now returns a `limit` property, but the OpenAPI response schema still documents only `result`. API consumers and generated clients will therefore receive a response field that is absent from the declared contract, preventing them from reliably discovering or typing the truncation metadata. Add `limit` to the documented 200 response schema.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43518&comment_hash=af7dd2476ecbc2e97305aa554ec206f52072cac8124b4272f27153be13d6aaa7&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43518&comment_hash=af7dd2476ecbc2e97305aa554ec206f52072cac8124b4272f27153be13d6aaa7&reaction=dislike'>👎</a>
##########
superset-frontend/src/filters/components/Select/buildQuery.ts:
##########
@@ -54,14 +54,21 @@ const buildQuery: BuildQuery<PluginFilterSelectQueryFormData> = (
}
const sortColumns = sortMetric ? [sortMetric] : columns;
+ // Sorting by the searched column makes the engine scan and sort every
+ // match before applying the row limit, which is the dominant cost of
+ // search-as-you-type on a high-cardinality column. The dropdown re-sorts
+ // the returned page client-side, so the server sort buys nothing here. A
+ // sort metric is different: it selects *which* rows come back, so it has
+ // to stay.
+ const skipOrderBy = !!search && !sortMetric;
const query: QueryObject[] = [
{
...baseQueryObject,
columns,
metrics: sortMetric ? [sortMetric] : [],
filters: filters.concat(extraFilters),
orderby:
- sortMetric || sortAscending !== undefined
+ !skipOrderBy && (sortMetric || sortAscending !== undefined)
? sortColumns.map(column => [column, !!sortAscending])
: [],
Review Comment:
**Suggestion:** When a search is active without `sortMetric`, removing `ORDER BY` makes the limited result page nondeterministic. The dropdown can reorder the rows it receives, but it cannot restore the previously deterministic selection of which rows enter the page, so repeated searches can omit different matching values and produce unstable results. Preserve a deterministic ordering or use a database-side strategy that avoids the full sort without changing page membership. [api mismatch]
<details>
<summary><b>Severity Level:</b> Major ⚠️</summary>
```mdx
- ⚠️ Search results can omit different matching values.
- ⚠️ Select filter suggestions become unstable across requests.
- ⚠️ Client sorting cannot restore excluded database rows.
```
</details>
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=321fbf2a937c446c987d8bb111aa08f3&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=321fbf2a937c446c987d8bb111aa08f3&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset-frontend/src/filters/components/Select/buildQuery.ts
**Line:** 63:73
**Comment:**
*Api Mismatch: When a search is active without `sortMetric`, removing `ORDER BY` makes the limited result page nondeterministic. The dropdown can reorder the rows it receives, but it cannot restore the previously deterministic selection of which rows enter the page, so repeated searches can omit different matching values and produce unstable results. Preserve a deterministic ordering or use a database-side strategy that avoids the full sort without changing page membership.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43518&comment_hash=a796131c278bf724ca83a4f97700f10267198c32b9d3fc31e850562e3483a6ff&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43518&comment_hash=a796131c278bf724ca83a4f97700f10267198c32b9d3fc31e850562e3483a6ff&reaction=dislike'>👎</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(filters): search filter values server-side in Explore [superset]
Posted by "netlify[bot] (via GitHub)" <gi...@apache.org>.
netlify[bot] commented on PR #43518:
URL: https://github.com/apache/superset/pull/43518#issuecomment-5412857183
### <span aria-hidden="true">✅</span> Deploy Preview for *superset-docs-preview* ready!
| Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | 972d9bc51b366f2b5030b6e32ab96ea6607a33f1 |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/projects/superset-docs-preview/deploys/6a8db5edeb43340007da1eb8 |
|<span aria-hidden="true">😎</span> Deploy Preview | [https://deploy-preview-43518--superset-docs-preview.netlify.app](https://deploy-preview-43518--superset-docs-preview.netlify.app) |
|<span aria-hidden="true">📱</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br /><br /><br />_Use your smartphone camera to open QR code link._</details> |
|<span aria-hidden="true">🤖</span> Make changes | [Run an agent on this branch](https://app.netlify.com/projects/superset-docs-preview/agent-runs#FYtJCoAwDAD_knOlFVf8gc9ITYoFlxJrQcW_W08zh5kHguxriDAAKLCC2zRnX48L03lrxxi180tkKQ6W9MMTZ0fJoQLisOzXSPlpsSfbMLGtq6o2xnSEJdse3g8) |
---
<!-- [superset-docs-preview Preview](https://deploy-preview-43518--superset-docs-preview.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify project configuration](https://app.netlify.com/projects/superset-docs-preview/configuration/notifications#deploy-notifications)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org