You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "eschutho (via GitHub)" <gi...@apache.org> on 2026/09/15 15:13:13 UTC
[PR] fix(sqla): wrap TypeError from virtual dataset Jinja rendering as a validation error [superset]
eschutho opened a new pull request, #44312:
URL: https://github.com/apache/superset/pull/44312
### SUMMARY
Fixes a production Sentry crash ([SUPERSET-PYTHON-16YX](https://preset-inc.sentry.io/issues/7733609759/)) — `TypeError: sequence item 0: expected str instance, int found`, surfacing as an unhandled **500** from `ChartDataRestApi.data`.
**Root cause**
`ExploreMixin.get_rendered_sql` (`superset/models/helpers.py`) renders a virtual dataset's custom SQL through the sandboxed Jinja engine and already catches Jinja-specific exceptions — `UndefinedError`, `TemplateError`, and `SupersetSyntaxErrorException` — re-raising them as a friendly `QueryObjectValidationError` (400). The gap: a `TypeError` raised by a Python builtin invoked from *within* the template is none of those exception types, so it escapes the guard as an unhandled 500.
The concrete trigger is the documented `filter_values()` usage pattern (from the `BaseTemplateProcessor.filter_values()` docstring in `superset/jinja_context.py`):
```
WHERE action in ({{ "'" + "','".join(filter_values('action_type')) + "'" }})
```
`filter_values()` returns `list[Any]` — filter values come from dashboard/native filters and are **not** coerced to `str`. When the filtered column holds numeric values (e.g. an int ID column), `filter_values()` returns `list[int]` and `str.join()` raises `TypeError: sequence item 0: expected str instance, int found` — exactly this issue's message.
**Evidence (Sentry traceback, top of stack first):**
```
superset/common/query_actions.py _get_full
superset/common/query_context.py get_df_payload
superset/common/query_context_processor.py get_df_payload / get_query_result
superset/connectors/sqla/models.py query
superset/models/helpers.py get_query_str_extended / get_sqla_query
superset/connectors/sqla/models.py get_from_clause
superset/models/helpers.py get_from_clause -> get_rendered_sql
superset/jinja_context.py process_template
jinja2/environment.py render
<template>:16 top-level template code
jinja2/sandbox.py call <-- str.join on list[int]
```
Event count / issue: https://preset-inc.sentry.io/issues/7733609759/ (Sentry issue SUPERSET-PYTHON-16YX).
**The fix**
Add a `TypeError` except clause in `get_rendered_sql` alongside the existing Jinja guards, wrapping it in `QueryObjectValidationError` using the existing message style (`"Error while rendering virtual dataset query: %(msg)s"`). This turns the crash into the same friendly 400 that Jinja-syntax errors already produce. This mirrors the guard pattern established by #44172 for `get_timestamp_expression`/`convert_tbl_column_to_sqla_col` in the same class (which did not touch `get_rendered_sql`).
### TESTING INSTRUCTIONS
Added a regression unit test `test_get_rendered_sql_wraps_type_error` in `tests/unit_tests/connectors/sqla/models_test.py`, matching the existing style there (`ExploreMixin.get_rendered_sql.__get__(datasource)`, `template_processor.process_template.side_effect = TypeError(...)`, assert `QueryObjectValidationError`).
Test run:
```
$ pytest tests/unit_tests/connectors/sqla/models_test.py -q
86 passed in 2.82s
```
Lint:
```
$ ruff check superset/models/helpers.py tests/unit_tests/connectors/sqla/models_test.py
All checks passed!
$ ruff format --check superset/models/helpers.py tests/unit_tests/connectors/sqla/models_test.py
2 files already formatted
```
To reproduce manually: create a virtual dataset whose SQL uses `{{ "','".join(filter_values('some_numeric_col')) }}`, apply a dashboard filter on a numeric column, and load the chart. Before this change the request returns a 500; after it returns a 400 `QueryObjectValidationError`.
### TRADEOFFS
- The guard is intentionally scoped to `TypeError` **only**. Other non-Jinja exception types (e.g. `ValueError`, `KeyError`) raised from Python builtins invoked inside user template macros will still surface as 500s. Broadening `get_rendered_sql` to a catch-all `Exception` would be a larger behavioral change (it could mask genuinely unexpected server errors as user-facing validation errors) and is out of scope here — flagged as a possible future follow-up, not fixed in this PR.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
---
Shortcut: https://app.shortcut.com/preset/story/121034
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(sqla): wrap TypeError from virtual dataset Jinja rendering as a validation error [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #44312:
URL: https://github.com/apache/superset/pull/44312#issuecomment-5769836182
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(sqla): wrap TypeError from virtual dataset Jinja rendering as a validation error [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #44312:
URL: https://github.com/apache/superset/pull/44312#issuecomment-5721917583
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/44312?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 73.71%. Comparing base ([`ae240a8`](https://app.codecov.io/gh/apache/superset/commit/ae240a8e6f0332acb709943d0884eae73927c9de?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`13b5b83`](https://app.codecov.io/gh/apache/superset/commit/13b5b83c93a1fe792a45e20ff4844e8b9bfaca46?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 86 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #44312 +/- ##
==========================================
- Coverage 75.91% 73.71% -2.21%
==========================================
Files 2926 2929 +3
Lines 173023 185182 +12159
Branches 40139 43170 +3031
==========================================
+ Hits 131349 136504 +5155
- Misses 38995 45740 +6745
- Partials 2679 2938 +259
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/44312/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/44312/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.30% <50.00%> (?)` | |
| [presto](https://app.codecov.io/gh/apache/superset/pull/44312/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `39.19% <50.00%> (?)` | |
| [python](https://app.codecov.io/gh/apache/superset/pull/44312/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `71.94% <100.00%> (-4.21%)` | :arrow_down: |
| [unit](https://app.codecov.io/gh/apache/superset/pull/44312/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `76.14% <100.00%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/44312?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(sqla): wrap TypeError from virtual dataset Jinja rendering as a validation error [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas merged PR #44312:
URL: https://github.com/apache/superset/pull/44312
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(sqla): wrap TypeError from virtual dataset Jinja rendering as a validation error [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #44312:
URL: https://github.com/apache/superset/pull/44312#issuecomment-5682767086
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(sqla): wrap TypeError from virtual dataset Jinja rendering as a validation error [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho closed pull request #44312: fix(sqla): wrap TypeError from virtual dataset Jinja rendering as a validation error
URL: https://github.com/apache/superset/pull/44312
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(sqla): wrap TypeError from virtual dataset Jinja rendering as a validation error [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho closed pull request #44312: fix(sqla): wrap TypeError from virtual dataset Jinja rendering as a validation error
URL: https://github.com/apache/superset/pull/44312
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(sqla): wrap TypeError from virtual dataset Jinja rendering as a validation error [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho closed pull request #44312: fix(sqla): wrap TypeError from virtual dataset Jinja rendering as a validation error
URL: https://github.com/apache/superset/pull/44312
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(sqla): wrap TypeError from virtual dataset Jinja rendering as a validation error [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on PR #44312:
URL: https://github.com/apache/superset/pull/44312#issuecomment-5682928049
@rebenitez1802 — assigning you as reviewer here for **continuity** on the Jinja template-error / raw-exception-leak bug-class you've carried end-to-end (#42366, #42401, and most recently #44172, also in `ExploreMixin`/`superset/models/helpers.py`).
**What it is:** `ExploreMixin.get_rendered_sql` only caught `UndefinedError`/`TemplateError`/`SupersetSyntaxErrorException`, so a `TypeError` from `str.join()` on the `list[int]` that `filter_values()` returns (virtual-dataset SQL template joining a numeric-column filter) escaped as an unhandled 500 (Sentry `SUPERSET-PYTHON-16YX`, culprit `ChartDataRestApi.data`). The fix adds a narrowly-scoped `except TypeError` around the `template_processor.process_template(sql)` call, mapping it to the `QueryObjectValidationError` (400) its sibling exception types already get. +32/-0 across `superset/models/helpers.py` and a regression test.
**Note given #44172:** the self-review empirically verified (live MRO/dispatch check under an initialized app) that the patched `ExploreMixin.get_rendered_sql` **is** the method actually dispatched on the real path (`SqlaTable` → `ExploreMixin`, no override) — i.e. this is not a repeat of the #44172 dead-code-guard trap where the guard sat on a class the runtime never hit. Worth confirming independently since you're the one who caught that last time.
**Load flag:** you're carrying the highest active-review count on the roster right now — happy to **re-route** this to another roster reviewer if you'd rather not take it; just say the word.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org