You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "b-yacoubi (via GitHub)" <gi...@apache.org> on 2026/02/24 14:15:06 UTC

[I] Dynamic query filters use data cache instead of filter state cache [superset]

b-yacoubi opened a new issue, #38219:
URL: https://github.com/apache/superset/issues/38219

   ### Bug description
   
   Consider a dashboard, with a dashboard filter that has the "Load filter values dynamically" option enabled.
   
   Our superset users have an RLS applied, which filters down on a dataset column, only returning rows featuring certain values in the result set. This is the same column whose values the dashboard filter is supposed to display.
   
   When this list of values is modified, the choices displayed in the filter dropdown still correspond to the old list until the timeout for DATA_CACHE_CONFIG has elapsed, instead of using the timeout for FILTER_STATE_CACHE_CONFIG. Indeed, the correct values display if one clicks on the dashboard dropdown menu and selects the "force refresh dashboard" option.
   
   It is then impossible to set a timeout for these query filters that differs from the timeout for all dashboard data.
   
   ### Screenshots/recordings
   
   _No response_
   
   ### Superset version
   
   5.0.0
   
   ### Python version
   
   3.9
   
   ### Node version
   
   Not applicable
   
   ### Browser
   
   Firefox
   
   ### Additional context
   
   _No response_
   
   ### Checklist
   
   - [x] I have searched Superset docs and Slack and didn't find a solution to my problem.
   - [x] I have searched the GitHub issue tracker and didn't find a similar bug report.
   - [x] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Dynamic query filters use data cache instead of filter state cache [superset]

Posted by "b-yacoubi (via GitHub)" <gi...@apache.org>.
b-yacoubi commented on issue #38219:
URL: https://github.com/apache/superset/issues/38219#issuecomment-3952692702

   The issue persists even as I make sure no browser- or database-level caching is going on.
   I'm pretty sure it's related to the `data` prop that gets passed into SelectFilterPlugin, which has some cache applied to it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Dynamic query filters use data cache instead of filter state cache [superset]

Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed issue #38219: Dynamic query filters use data cache instead of filter state cache
URL: https://github.com/apache/superset/issues/38219


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Dynamic query filters use data cache instead of filter state cache [superset]

Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
dosubot[bot] commented on issue #38219:
URL: https://github.com/apache/superset/issues/38219#issuecomment-3952798082

   <!-- Answer -->
   You're right! I stand corrected on my earlier response. Looking more closely at the code, the dynamic filter queries for SelectFilterPlugin go through a different path than I initially described.
   
   When "Load filter values dynamically" is enabled, the filter [builds its query via `buildQuery.ts`](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset-frontend/src/filters/components/Select/buildQuery.ts#L30-L74) and executes through the `/api/v1/chart/data` endpoint—the same endpoint used for regular chart queries. This means filter dropdown queries **do** go through [`QueryContextProcessor`](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/common/query_context_processor.py#L130-L307) and [`QueryCacheManager`](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/common/utils/query_cache_manager.py#L116-L124), which uses `DATA_CACHE_CONFIG`.
   
   So your observation is accurate: dynamic filter queries are being cached with `DATA_CACHE_CONFIG` timeout rather than `FILTER_STATE_CACHE_CONFIG` (which only stores user's filter *selections*, not the available options). This appears to be a legitimate bug/feature gap—there's no separate cache configuration for filter dropdown value queries.
   
   A workaround would be to lower your `DATA_CACHE_CONFIG` timeout, though that affects all chart queries. A proper fix would likely require either a separate cache config for filter queries or bypassing the cache entirely for filter value lookups.
   
   
   <!-- Dosu Comment Footer -->
   *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).*
   
   ---
   <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/2a0ad91a-d98b-4902-b6fd-c0e834c04ac9?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/2a0ad91a-d98b-4902-b6fd-c0e834c04ac9?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/2a0ad91a-d98b-4902-b6fd-c0e834c04ac9?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/2a0ad91a-d98b-4902-b6fd-c0e834c04ac9?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/2a0ad91a-d98b-4902-b6fd-c0e834c04ac9?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/2a0ad91a-d98b-4902-b6fd-c0e834c04ac9?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/2a0ad91a-d98b-4902-b6fd-c0e834c04ac9?feedback_type=other)</sup>
   
   [![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=github)&nbsp;[![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=You%27re%20right%21%20I%20stand%20corrected%20on%20my%20earlier%20response.%20Looking%20more%20closely%20at%20the%20code%2C%20the%20dynamic%20filter%20queries%20for%20SelectFilterPlugin%20go%20through%20a%20different%20path%20than%20I%20initially%20described.%0A%0AWhen%20%22Load%20filter%20values%20dynamically%22%20is%20enabled%2C%20the%20filter%20%5Bbuilds%20its%20query%20via%20%60buildQuery.ts%60%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset-frontend/src/filters/components/Select/buildQuery.ts%23L30-L74%29%20and%20executes%20through%20the%20%60/api/v1/chart/data%60%20endpoint%E2%80%94the%20same%20endpoint%20used%20for%20regular%20chart%20queries.%20This%20means%20filter%20dropdown%20queries%20%2A
 %2Ado%2A%2A%20go%20through%20%5B%60QueryContextProcessor%60%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/common/query_context_processor.py%23L130-L307%29%20and%20%5B%60QueryCacheManager%60%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/common/utils/query_cache_manager.py%23L116-L124%29%2C%20which%20uses%20%60DATA_CACHE_CONFIG%60.%0A%0ASo%20your%20observation%20is%20accurate%3A%20dynamic%20filter%20queries%20are%20being%20cached%20with%20%60DATA_CACHE_CONFIG%60%20timeout%20rather%20than%20%60FILTER_STATE_CACHE_CONFIG%60%20%28which%20only%20stores%20user%27s%20filter%20%2Aselections%2A%2C%20not%20the%20available%20options%29.%20This%20appears%20to%20be%20a%20legitimate%20bug/feature%20gap%E2%80%94there%27s%20no%20separate%20cache%20configuration%20for%20filter%20dropdown%20value%20queries.%0A%0AA%20workaround%20would%20be%20to%20lower%20your%20%60DATA_CACHE_CONFIG%60%20timeout%2C%20th
 ough%20that%20affects%20all%20chart%20queries.%20A%20proper%20fix%20would%20likely%20require%20either%20a%20separate%20cache%20config%20for%20filter%20queries%20or%20bypassing%20the%20cache%20entirely%20for%20filter%20value%20lookups.)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/issues/38219)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Dynamic query filters use data cache instead of filter state cache [superset]

Posted by "Ujjwaljain16 (via GitHub)" <gi...@apache.org>.
Ujjwaljain16 commented on issue #38219:
URL: https://github.com/apache/superset/issues/38219#issuecomment-4142300486

   Thanks for raising this 
   i was able to reproduce it and confirm the root cause
   
   when dynamic filter values are enabled, the request actually goes through `/api/v1/chart/data`, so it ends up using `DATA_CACHE_CONFIG`
   that’s why the dropdown values stay stale they’re being cached like chart queries, not like filter state
   
   i’ve put together a small fix that detects native filter option queries (`native_filter_id` + `filter_*` viz type) and uses `FILTER_STATE_CACHE_CONFIG` timeout instead
   no changes to cache keys or pipeline  just aligning TTL with expected filter behavior
   
   added tests to cover both filter and non-filter cases.
   
   happy to adjust if there’s a preferred way to detect these queries.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Dynamic query filters use data cache instead of filter state cache [superset]

Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
dosubot[bot] commented on issue #38219:
URL: https://github.com/apache/superset/issues/38219#issuecomment-3952272116

   <!-- Answer -->
   Interesting issue! Based on my research, the dynamic filter values endpoint (`/api/v1/datasource/.../column/.../values/`) [actually doesn't use any caching](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/datasource/api.py#L38-L125)—it queries the database directly on each request via the [`values_for_column()` method](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/models/helpers.py#L1326-L1378).
   
   Some clarification on the cache configs:
   - **FILTER_STATE_CACHE_CONFIG** stores the user's *selected filter values* (their choices), not the available dropdown options
   - **DATA_CACHE_CONFIG** caches *chart query results*, not filter value queries
   
   Since dynamic filter values aren't cached at the Superset level, the stale values you're seeing might be caused by:
   1. **Browser-level caching** of the API response
   2. **Database-level caching** (some databases cache query results)
   3. **A reverse proxy or CDN** caching the API calls
   
   Could you verify whether the issue persists if you open the browser dev tools, disable cache, and reload the filter dropdown? Also, checking your network tab to see if the `/values/` endpoint returns a 304 Not Modified vs a fresh 200 response could help narrow this down.
   
   
   <!-- Dosu Comment Footer -->
   *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).*
   
   ---
   <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/ede54643-a246-4839-956c-6b7ab6bfd681?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/ede54643-a246-4839-956c-6b7ab6bfd681?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/ede54643-a246-4839-956c-6b7ab6bfd681?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/ede54643-a246-4839-956c-6b7ab6bfd681?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/ede54643-a246-4839-956c-6b7ab6bfd681?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/ede54643-a246-4839-956c-6b7ab6bfd681?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/ede54643-a246-4839-956c-6b7ab6bfd681?feedback_type=other)</sup>
   
   [![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=github)&nbsp;[![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=Interesting%20issue%21%20Based%20on%20my%20research%2C%20the%20dynamic%20filter%20values%20endpoint%20%28%60/api/v1/datasource/.../column/.../values/%60%29%20%5Bactually%20doesn%27t%20use%20any%20caching%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/datasource/api.py%23L38-L125%29%E2%80%94it%20queries%20the%20database%20directly%20on%20each%20request%20via%20the%20%5B%60values_for_column%28%29%60%20method%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/models/helpers.py%23L1326-L1378%29.%0A%0ASome%20clarification%20on%20the%20cache%20configs%3A%0A-%20%2A%2AFILTER_STATE_CACHE_CONFIG%2A%2A%20stores%20the%20user%27s%20%2Aselected%20filter%20values%2A%20
 %28their%20choices%29%2C%20not%20the%20available%20dropdown%20options%0A-%20%2A%2ADATA_CACHE_CONFIG%2A%2A%20caches%20%2Achart%20query%20results%2A%2C%20not%20filter%20value%20queries%0A%0ASince%20dynamic%20filter%20values%20aren%27t%20cached%20at%20the%20Superset%20level%2C%20the%20stale%20values%20you%27re%20seeing%20might%20be%20caused%20by%3A%0A1.%20%2A%2ABrowser-level%20caching%2A%2A%20of%20the%20API%20response%0A2.%20%2A%2ADatabase-level%20caching%2A%2A%20%28some%20databases%20cache%20query%20results%29%0A3.%20%2A%2AA%20reverse%20proxy%20or%20CDN%2A%2A%20caching%20the%20API%20calls%0A%0ACould%20you%20verify%20whether%20the%20issue%20persists%20if%20you%20open%20the%20browser%20dev%20tools%2C%20disable%20cache%2C%20and%20reload%20the%20filter%20dropdown%3F%20Also%2C%20checking%20your%20network%20tab%20to%20see%20if%20the%20%60/values/%60%20endpoint%20returns%20a%20304%20Not%20Modified%20vs%20a%20fresh%20200%20response%20could%20help%20narrow%20this%20down.)&nbsp;[![Join Discord]
 (https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/issues/38219)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org