You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "neuromantik33 (via GitHub)" <gi...@apache.org> on 2025/06/15 10:24:47 UTC

[I] Superset no longer accepts custom jinja template functions for caching [superset]

neuromantik33 opened a new issue, #33775:
URL: https://github.com/apache/superset/issues/33775

   ### Bug description
   
   There is a regression in the PR fix(Jinja): Extra cache keys to consider vars with set #30549 . I mentioned it as a comment within the PR but I guess since it's merged I'm filing a proper bug report.
   
   This PR breaks are existing usage of RLS (thank heavens one for our campus users notified us 😅 ). This is our input string:
   ```sql
   campus_id IN UNNEST( {{ cache_key_wrapper(current_user_campus_ids()) }} )
   ```
   which doesn't match using the pattern : `cache_key_wrapper\([^()]*\)`,
   but does using the old one (v4.0.2): `cache_key_wrapper\(.*\)`.
   
   The function we use looks as follows:
   ```python
   def current_user_campus_ids() -> list[int]:
       """
       Get the list of campus ids associated with the current user
       """
       user: FortyTwoUser = g.user
       return user.campus_ids()
   ```
   and is populated using the oauth2 jwt from keycloak.
   Is the only workaround to monkey patch the jinja `ExtraCache` processor to add our macro within? How is the `cache_key_wrapper()` to be used if it can't invoke another function?
   
   ### Screenshots/recordings
   
   _No response_
   
   ### Superset version
   
   master / latest-dev
   
   ### Python version
   
   3.10
   
   ### Node version
   
   I don't know
   
   ### Browser
   
   Chrome
   
   ### Additional context
   
   _No response_
   
   ### Checklist
   
   - [x] I have searched Superset docs and Slack and didn't find a solution to my problem.
   - [x] I have searched the GitHub issue tracker and didn't find a similar bug report.
   - [x] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [I] Superset no longer accepts custom jinja template functions for caching [superset]

Posted by "neuromantik33 (via GitHub)" <gi...@apache.org>.
neuromantik33 commented on issue #33775:
URL: https://github.com/apache/superset/issues/33775#issuecomment-2979530137

   @dosu Ok that is great but according to the docs you mentioned : 
   
   > Explicitly Including Values in Cache Key
   > The {{ cache_key_wrapper() }} function explicitly instructs Superset to add a value to the accumulated list of values used in the calculation of the cache key.
   > This function is only needed when you want to wrap your own custom function return values in the cache key.
   
   If I can't include a function in the `cache_key_wrapper` what exactly is it wrapping? I don't know how to call a function without including parentheses unless it's a property accessor. Can you at least agree that the regexp for `cache_key_wrapper` should be a little more lenient?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [I] Superset no longer accepts custom jinja template functions for caching [superset]

Posted by "neuromantik33 (via GitHub)" <gi...@apache.org>.
neuromantik33 commented on issue #33775:
URL: https://github.com/apache/superset/issues/33775#issuecomment-3076552346

   Any news on this issue? This is a pretty big security regression IMHO, I guess I'll have to resort to monkeypatching the cache_key_wrapper.. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [I] Superset no longer accepts custom jinja template functions for caching [superset]

Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina closed issue #33775: Superset no longer accepts custom jinja template functions for caching
URL: https://github.com/apache/superset/issues/33775


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [I] Superset no longer accepts custom jinja template functions for caching [superset]

Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #33775:
URL: https://github.com/apache/superset/issues/33775#issuecomment-4035399823

   Can anyone from the core team or Preset provide an update on whether accepting custom Jinja template functions for caching is planned to be restored? The reporter noted this is a security-relevant regression. @rusackas / @betodealmeida — any guidance appreciated.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [I] Superset no longer accepts custom jinja template functions for caching [superset]

Posted by "villebro (via GitHub)" <gi...@apache.org>.
villebro commented on issue #33775:
URL: https://github.com/apache/superset/issues/33775#issuecomment-4035441556

   I agree agree this is a regression. A workaround would be to split this in two parts, which seems unnecessary:
   
   ```python
   {% abc = myfunc() %}
   {{ cache_key_wrapper(abc) }}
   ```
   
   is the same as
   
   ```python
   {{ cache_key_wrapper(myfunc()) }}
   ```
   
   I can open a PR to fix this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [I] Superset no longer accepts custom jinja template functions for caching [superset]

Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
dosubot[bot] commented on issue #33775:
URL: https://github.com/apache/superset/issues/33775#issuecomment-2979545649

   <!-- Answer -->
   You're right—there's a disconnect between the docs and the current implementation. The docs say `cache_key_wrapper()` is for wrapping custom function return values, but the regex in the code now only allows simple, non-nested expressions (like a single variable or property accessor) inside `cache_key_wrapper`, not function calls with parentheses or any nested expressions. So, in practice, you can't use it to wrap the result of a function call like `current_user_campus_ids()` anymore—only something like `cache_key_wrapper(variable)` will work, which is pretty limiting for dynamic use cases like yours.
   
   I agree that the regex should be more lenient to match the documented intent and real-world needs. Right now, the only way around this is to monkey patch the `ExtraCache` processor or revert the regex change, since there's no config or extension point for this part of the logic. No relevant discussions or plans to relax the regex have come up in the repo or docs so far.
   
   If you need this flexibility, advocating for a change to the regex or opening a discussion about restoring support for nested calls would be reasonable next steps. For now, the restriction is hardcoded and not aligned with the documentation or common Jinja usage patterns.  
   [Reference: implementation and regex restriction](https://github.com/apache/superset/blob/4d6cdf4fea44164e5953357ceceb44c93cdee8e3/superset/jinja_context.py)  
   [Reference: docs on intended usage](https://github.com/apache/superset/blob/4d6cdf4fea44164e5953357ceceb44c93cdee8e3/docs/docs/configuration/sql-templating.mdx)
   
   
   <!-- Dosu Comment Footer -->
   *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).*
   
   ---
   <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/17b0dead-3e18-4e03-94b8-308c044956de?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/17b0dead-3e18-4e03-94b8-308c044956de?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/17b0dead-3e18-4e03-94b8-308c044956de?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/17b0dead-3e18-4e03-94b8-308c044956de?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/17b0dead-3e18-4e03-94b8-308c044956de?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/17b0dead-3e18-4e03-94b8-308c044956de?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/17b0dead-3e18-4e03-94b8-308c044956de?feedback_type=other)</sup>&nbsp;&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[!
 [Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/issues/33775)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [I] Superset no longer accepts custom jinja template functions for caching [superset]

Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #33775:
URL: https://github.com/apache/superset/issues/33775#issuecomment-3001983590

   Pinging @geido (the author) and @villebro (the reviewer) of the PR in question, for good measure. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [I] Superset no longer accepts custom jinja template functions for caching [superset]

Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
dosubot[bot] commented on issue #33775:
URL: https://github.com/apache/superset/issues/33775#issuecomment-2973643081

   <!-- Answer -->
   This regression is due to a change in the regex used by Superset to detect cache key wrappers in Jinja templates—it now only allows simple, non-nested expressions like `cache_key_wrapper(foo)` and blocks nested calls like `cache_key_wrapper(current_user_campus_ids())` that worked in previous versions. This breaks use cases like yours for RLS and dynamic user-based cache keys. There’s currently no official workaround or extensibility for this in the code or docs, and no open issues or discussions about it in the repo.
   
   Monkey patching the Jinja `ExtraCache` processor is the only available workaround if you need to restore the previous behavior, since the regex restriction is hardcoded and there’s no config or plugin hook for extending cache key computation. If you go this route, be aware it’s an unsupported hack and could break with future updates. Otherwise, you’d need to refactor your templates to avoid nested calls, but that may not be possible for your use case.
   
   For reference, here’s the relevant implementation and restriction in the code:  
   [cache_key_wrapper and regex in ExtraCache](https://github.com/apache/superset/blob/8fd0fd673f3307ae2f7555272d68d8511651e685/superset/jinja_context.py)  
   And the intended usage in the docs:  
   [SQL templating and cache_key_wrapper](https://github.com/apache/superset/blob/8fd0fd673f3307ae2f7555272d68d8511651e685/docs/docs/configuration/sql-templating.mdx)
   
   If you need to keep your current pattern, monkey patching is your only option until the project provides a more flexible or documented solution.
   
   
   <!-- Dosu Comment Footer -->
   *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).*
   
   ---
   <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/88e1f76e-ceb5-4fa8-8deb-1e98dc04b987?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/88e1f76e-ceb5-4fa8-8deb-1e98dc04b987?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/88e1f76e-ceb5-4fa8-8deb-1e98dc04b987?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/88e1f76e-ceb5-4fa8-8deb-1e98dc04b987?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/88e1f76e-ceb5-4fa8-8deb-1e98dc04b987?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/88e1f76e-ceb5-4fa8-8deb-1e98dc04b987?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/88e1f76e-ceb5-4fa8-8deb-1e98dc04b987?feedback_type=other)</sup>&nbsp;&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[!
 [Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/issues/33775)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [I] Superset no longer accepts custom jinja template functions for caching [superset]

Posted by "villebro (via GitHub)" <gi...@apache.org>.
villebro commented on issue #33775:
URL: https://github.com/apache/superset/issues/33775#issuecomment-4035415775

   Let me take a look


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org