You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "eschutho (via GitHub)" <gi...@apache.org> on 2026/09/12 16:46:51 UTC
[PR] fix(reports): catch JSONDecodeError in report extra validation (dashboard position_json/json_metadata) [superset]
eschutho opened a new pull request, #44204:
URL: https://github.com/apache/superset/pull/44204
### SUMMARY
`BaseReportScheduleCommand._validate_report_extra` and `_validate_native_filters`
(in `superset/commands/report/base.py`) call `json.loads` on a dashboard's
`position_json` and `json_metadata` without guarding against malformed JSON.
Both methods are reached from `CreateReportScheduleCommand.validate()` and
`UpdateReportScheduleCommand.validate()`, invoked from `POST /api/v1/report/`
and `PUT /api/v1/report/{id}`. When the referenced dashboard (via
`extra.dashboard`) has a `position_json` or `json_metadata` that is not valid
JSON, `json.loads` raises `simplejson.JSONDecodeError`. The report API's
`except` clauses only catch `ReportScheduleNotFoundError`,
`ReportScheduleInvalidError`, `ReportScheduleCreateFailedError`,
`ReportScheduleForbiddenError`/`ReportScheduleUpdateFailedError` and
marshmallow's `ValidationError` โ `JSONDecodeError` is none of those, so it
escapes as a raw, opaque HTTP 500 instead of a clean validation error.
Dashboards should not normally reach this state given write-time validation,
but legacy/imported dashboards predating stricter validation can (see issue/PR
history around #43574 / #43575).
### PROBLEM
Two unguarded `json.loads` sites:
- `_validate_report_extra`: `position_data = json.loads(dashboard.position_json or "{}")`
- `_validate_native_filters`: `json_metadata = json.loads(dashboard.json_metadata or "{}")`
The same method already demonstrates the correct pattern for the `anchor`
field (`try/except json.JSONDecodeError`), so this fix simply mirrors it.
### FIX
Wrap both `json.loads` calls in `try/except json.JSONDecodeError` and append a
marshmallow `ValidationError` to the `exceptions` list (surfaced as a
`ReportScheduleInvalidError` โ 422) instead of letting the raw exception
propagate:
- `position_json`: append `ValidationError("extra.dashboard.position_json is
not valid JSON", "extra")` and `return` early (mirroring the existing
`not isinstance(dashboard_state, dict)` early-return branch, since
`position_data` is required by everything below).
- `json_metadata`: append `ValidationError("extra.nativeFilters could not be
validated: dashboard metadata is not valid JSON", "extra")` and `break` out
of the native-filter loop (since `valid_filter_ids` can no longer be
computed).
This touches two sites in the same file for the identical bug class, mirroring
how #42401 covered four sites in one file.
This is part of the same recurring raw-exception-leak cleanup pipeline as
#42366 and #42401 (raw system/library exception escaping instead of a proper
Superset/marshmallow validation error).
### TESTING INSTRUCTIONS
Added two regression tests in
`tests/unit_tests/commands/report/create_test.py`
(`test_validate_report_extra_malformed_position_json`,
`test_validate_report_extra_malformed_json_metadata`), following the existing
house style in that file. Each drives validation with a dashboard whose
`position_json` / `json_metadata` is non-JSON and asserts a `ValidationError`
(field `extra`) is collected rather than a raw `JSONDecodeError` escaping.
Verified fail-before / pass-after: on pre-fix code both tests fail with an
escaping `simplejson.errors.JSONDecodeError`; after the fix both pass. Full
`tests/unit_tests/commands/report/` suite (381 tests) passes. `ruff check`,
`ruff format --check`, and `mypy` are green on the changed files.
```
pytest tests/unit_tests/commands/report/create_test.py -k malformed
```
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(reports): catch JSONDecodeError in report extra validation (dashboard position_json/json_metadata) [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #44204:
URL: https://github.com/apache/superset/pull/44204#issuecomment-5647307106
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/44204?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `16.66667%` with `10 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 57.44%. Comparing base ([`16fc356`](https://app.codecov.io/gh/apache/superset/commit/16fc35667e466dd13f0a7d297bce9c41fca3b1f7?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`9968556`](https://app.codecov.io/gh/apache/superset/commit/996855622383d3a73c899044782a4051b155b6e1?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/superset/pull/44204?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [superset/commands/report/base.py](https://app.codecov.io/gh/apache/superset/pull/44204?src=pr&el=tree&filepath=superset%2Fcommands%2Freport%2Fbase.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvY29tbWFuZHMvcmVwb3J0L2Jhc2UucHk=) | 16.66% | [10 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/44204?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (16fc356) and HEAD (9968556). Click for more details.
>
> <details><summary>HEAD has 14 uploads less than BASE</summary>
>
>| Flag | BASE (16fc356) | HEAD (9968556) |
>|------|------|------|
>|python|14|2|
>|presto|2|1|
>|hive|2|1|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #44204 +/- ##
===========================================
- Coverage 80.16% 57.44% -22.72%
===========================================
Files 2925 2925
Lines 172563 172571 +8
Branches 40058 40058
===========================================
- Hits 138339 99142 -39197
- Misses 31627 72497 +40870
+ Partials 2597 932 -1665
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/44204/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage ฮ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/44204/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.33% <16.66%> (-0.01%)` | :arrow_down: |
| [mysql](https://app.codecov.io/gh/apache/superset/pull/44204/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [postgres](https://app.codecov.io/gh/apache/superset/pull/44204/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [presto](https://app.codecov.io/gh/apache/superset/pull/44204/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `39.22% <16.66%> (-0.01%)` | :arrow_down: |
| [python](https://app.codecov.io/gh/apache/superset/pull/44204/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `39.27% <16.66%> (-45.28%)` | :arrow_down: |
| [sqlite](https://app.codecov.io/gh/apache/superset/pull/44204/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unit](https://app.codecov.io/gh/apache/superset/pull/44204/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/44204?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(reports): catch JSONDecodeError in report extra validation (dashboard position_json/json_metadata) [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #44204:
URL: https://github.com/apache/superset/pull/44204#issuecomment-5647281126
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(reports): catch JSONDecodeError in report extra validation (dashboard position_json/json_metadata) [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #44204:
URL: https://github.com/apache/superset/pull/44204#discussion_r3996902585
##########
superset/commands/report/base.py:
##########
@@ -198,7 +200,16 @@ def _validate_report_extra(self, exceptions: list[ValidationError]) -> None:
)
return
- position_data = json.loads(dashboard.position_json or "{}")
+ try:
+ position_data = json.loads(dashboard.position_json or "{}")
Review Comment:
**Suggestion:** Valid JSON such as `[]` or `null` is not rejected here, so the later `position_data.keys()` call raises `AttributeError` and still produces an HTTP 500. [type error]
**Assessment:** ๐ `Major` ยท ๐ `Occurrence: Sometimes`
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=a15245963d1e457aa0f98703932563a9&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=a15245963d1e457aa0f98703932563a9&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/commands/report/base.py
**Line:** 204:204
**Comment:**
*Type Error: Valid JSON such as `[]` or `null` is not rejected here, so the later `position_data.keys()` call raises `AttributeError` and still produces an HTTP 500.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F44204&comment_hash=a7d0f001816144a6caa5892b7e444e60a63c01fcdf94a0fbba6b24bcd2633839&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F44204&comment_hash=a7d0f001816144a6caa5892b7e444e60a63c01fcdf94a0fbba6b24bcd2633839&reaction=dislike'>๐</a>
##########
superset/commands/report/base.py:
##########
@@ -293,7 +304,19 @@ def _validate_native_filters(
)
continue
if valid_filter_ids is None:
- json_metadata = json.loads(dashboard.json_metadata or "{}")
+ try:
+ json_metadata = json.loads(dashboard.json_metadata or "{}")
Review Comment:
**Suggestion:** Valid JSON such as `[]` or `null` is not rejected here, so `json_metadata.get(...)` raises `AttributeError` instead of returning a validation error. [type error]
**Assessment:** ๐ `Major` ยท ๐ `Occurrence: Sometimes`
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=84f7c096137242439c8d93c93c9e054c&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=84f7c096137242439c8d93c93c9e054c&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/commands/report/base.py
**Line:** 308:308
**Comment:**
*Type Error: Valid JSON such as `[]` or `null` is not rejected here, so `json_metadata.get(...)` raises `AttributeError` instead of returning a validation error.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F44204&comment_hash=12a957ddb09797775f6dcd7f63c7c4bf610316345cbc7630ed14fa4ae3d5d159&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F44204&comment_hash=12a957ddb09797775f6dcd7f63c7c4bf610316345cbc7630ed14fa4ae3d5d159&reaction=dislike'>๐</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(reports): catch JSONDecodeError in report extra validation (dashboard position_json/json_metadata) [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho merged PR #44204:
URL: https://github.com/apache/superset/pull/44204
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org