You are viewing a plain text version of this content. The canonical link for it is here.

Posted to notifications@superset.apache.org by "eschutho (via GitHub)" <gi...@apache.org> on 2026/09/24 16:48:21 UTC

[PR] fix(db_engine_specs): raise SupersetGenericDBErrorException instead of bare-reraising JSONDecodeError [superset]

eschutho opened a new pull request, #44616:
URL: https://github.com/apache/superset/pull/44616

   ### SUMMARY
   
   **Problem.** `BaseEngineSpec.get_extra_params` and `BaseEngineSpec.update_params_from_encrypted_extra` (`superset/db_engine_specs/base.py`) catch `json.JSONDecodeError` when `database.extra` / `database.encrypted_extra` holds malformed JSON. They log it and then bare-`raise`. `superset.utils.json` re-exports `simplejson.JSONDecodeError`, which is a plain `ValueError` and not a `SupersetException`, so the raw exception escapes untyped. Both methods run on every engine creation (`Database._get_sqla_engine` → `get_extra()` / `update_params_from_encrypted_extra()`). That covers every engine spec that doesn't override them, plus snowflake, databricks, duckdb and trino, which call `BaseEngineSpec.get_extra_params` directly.
   
   The same class already handles this input safely elsewhere. `mask_encrypted_extra` and `unmask_encrypted_extra` catch `(TypeError, json.JSONDecodeError)` and don't let the raw exception out. These two methods are the outliers.
   
   **Fix.** Keep the log and raise `SupersetGenericDBErrorException(message=str(ex)) from ex` (status 400, `GENERIC_DB_ENGINE_ERROR`) instead of re-raising the raw exception. Only these two methods change. A config that is actually malformed still fails, but with a typed Superset error.
   
   **Where the behavior changes.** Any caller that handles `SupersetException` / `SupersetErrorException` now returns a typed error, where it used to fall through to a generic 500. For example, `GET /api/v1/database/<pk>/schemas/` and `/catalogs/` have an `except SupersetException` branch, and `Database.get_all_schema_names` passes the exception through `get_dbapi_mapped_exception` unchanged. Those routes now return a 400 with the decode message instead of FAB's `@safe` generic 500. The same applies to routes behind `@handle_api_exception` and the SIP-40 app error handlers. Some routes, such as `GET /api/v1/datasource/<type>/<id>/column/<col>/values/`, have no `SupersetException` handler before `@safe`. They still return a generic 500, but the exception reaching them is typed now.
   
   Same effort and shape as #42401 (sibling-drift fix, several sites in one file) and #44463 / #44502 (wrap the raw exception at the existing method boundary).
   
   ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
   
   N/A (backend only)
   
   ### TESTING INSTRUCTIONS
   
   - `pytest tests/unit_tests/db_engine_specs/test_base.py` (100 passed)
   - New tests `test_get_extra_params_malformed_json` and `test_update_params_from_encrypted_extra_malformed_json` fail on pre-fix code with a raw `simplejson.errors.JSONDecodeError` and pass with the fix.
   - Manual: set a sqlite database's `extra` to `{not valid json` directly in the metadata DB, then call `GET /api/v1/database/<pk>/schemas/`. You get a 400 with the decode message instead of a generic 500.
   
   ### ADDITIONAL INFORMATION
   - [ ] Has associated issue:
   - [ ] Required feature flags:
   - [ ] Changes UI
   - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
     - [ ] Migration is atomic, supports rollback & is backwards-compatible
     - [ ] Confirm DB migration upgrade and downgrade tested
     - [ ] Runtime estimates and downtime expectations provided
   - [ ] Introduces new feature or API
   - [ ] Removes existing feature or API
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [PR] fix(db_engine_specs): raise SupersetGenericDBErrorException instead of bare-reraising JSONDecodeError [superset]

Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.

bito-code-review[bot] commented on PR #44616:
URL: https://github.com/apache/superset/pull/44616#issuecomment-5818356553

   AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [PR] fix(db_engine_specs): raise SupersetGenericDBErrorException instead of bare-reraising JSONDecodeError [superset]

Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.

codeant-ai-for-open-source[bot] commented on code in PR #44616:
URL: https://github.com/apache/superset/pull/44616#discussion_r4096197153


##########
superset/db_engine_specs/base.py:
##########
@@ -2727,7 +2728,7 @@ def get_extra_params(
                 extra = json.loads(database.extra)
             except json.JSONDecodeError as ex:
                 logger.error(ex, exc_info=True)
-                raise
+                raise SupersetGenericDBErrorException(message=str(ex)) from ex

Review Comment:
   **Suggestion:** The typed-error guarantee applies only through this base method. MySQL, Databricks, and Snowflake overrides still bare-reraise JSONDecodeError for malformed encrypted extras.
   
   **Assessment:** 🟠 `Major` · 🔁 `Occurrence: Rarely` · 🏷️ `Api mismatch`
   
   [![Use CodeAnt Skill](https://new-codeant-butcket.s3.us-west-1.amazonaws.com/badges/use-codeant-skill-flat-v2.svg)](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [![Fix in Cursor](https://new-codeant-butcket.s3.us-west-1.amazonaws.com/badges/fix-in-cursor-flat.svg)](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=4ad35cafe41c49a2a6d92774e4e0e2eb&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [![Fix in VSCode Claude](https://new-codeant-butcket.s3.us-west-1.amazonaws.com/badges/fix-in-vscode-claude-flat.svg)](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=4ad35cafe41c49a2a6d92774e4e0e2eb&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
   <details>
   <summary><b>Prompt for AI Agent 🤖 </b></summary>
   
   ```mdx
   This is a comment left during a code review.
   
   **Path:** superset/db_engine_specs/base.py
   **Line:** 2731:2731
   **Comment:**
   	*Api Mismatch: The typed-error guarantee applies only through this base method. MySQL, Databricks, and Snowflake overrides still bare-reraise JSONDecodeError for malformed encrypted extras.
   
   Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
   Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
   ```
   </details>
   <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F44616&comment_hash=6003b5ff7eb16c3ae5f7c5b9895c9dc49247e0743a3a8a755839e20b9e92d74d&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F44616&comment_hash=6003b5ff7eb16c3ae5f7c5b9895c9dc49247e0743a3a8a755839e20b9e92d74d&reaction=dislike'>👎</a>



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [PR] fix(db_engine_specs): raise SupersetGenericDBErrorException instead of bare-reraising JSONDecodeError [superset]

Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.

codecov[bot] commented on PR #44616:
URL: https://github.com/apache/superset/pull/44616#issuecomment-5818759409

   ## [Codecov](https://app.codecov.io/gh/apache/superset/pull/44616?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `0%` with `2 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 65.88%. Comparing base ([`613033b`](https://app.codecov.io/gh/apache/superset/commit/613033b294bb827ebbdfa662d99852c6c4459734?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`36f6208`](https://app.codecov.io/gh/apache/superset/commit/36f620867a426ce1908d7ad6e6e409c934173a3b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/superset/pull/44616?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [superset/db\_engine\_specs/base.py](https://app.codecov.io/gh/apache/superset/pull/44616?src=pr&el=tree&filepath=superset%2Fdb_engine_specs%2Fbase.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL2Jhc2UucHk=) | 0.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/44616?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (613033b) and HEAD (36f6208). Click for more details.
   > 
   > <details><summary>HEAD has 8 uploads less than BASE</summary>
   >
   >| Flag | BASE (613033b) | HEAD (36f6208) |
   >|------|------|------|
   >|python|9|3|
   >|presto|2|1|
   >|hive|2|1|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff             @@
   ##           master   #44616       +/-   ##
   ===========================================
   - Coverage   81.07%   65.88%   -15.19%     
   ===========================================
     Files        2955     2955               
     Lines      178324   178324               
     Branches    41313    41313               
   ===========================================
   - Hits       144573   117497    -27076     
   - Misses      31041    58309    +27268     
   + Partials     2710     2518      -192     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/superset/pull/44616/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hive](https://app.codecov.io/gh/apache/superset/pull/44616/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `36.80% <0.00%> (ø)` | |
   | [mysql](https://app.codecov.io/gh/apache/superset/pull/44616/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [postgres](https://app.codecov.io/gh/apache/superset/pull/44616/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [presto](https://app.codecov.io/gh/apache/superset/pull/44616/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.70% <0.00%> (ø)` | |
   | [python](https://app.codecov.io/gh/apache/superset/pull/44616/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.84% <0.00%> (-29.54%)` | :arrow_down: |
   | [sqlite](https://app.codecov.io/gh/apache/superset/pull/44616/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.71% <0.00%> (ø)` | |
   | [unit](https://app.codecov.io/gh/apache/superset/pull/44616/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   </details>
   
   [:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/44616?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [PR] fix(db_engine_specs): raise SupersetGenericDBErrorException instead of bare-reraising JSONDecodeError [superset]

Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.

bito-code-review[bot] commented on PR #44616:
URL: https://github.com/apache/superset/pull/44616#issuecomment-5818430241

   <!-- Bito Reply -->
   The flagged issue is correct. While the current PR updates `BaseEngineSpec` to wrap `JSONDecodeError` in `SupersetGenericDBErrorException`, engine-specific overrides (like those for MySQL, Databricks, and Snowflake) that implement their own `get_extra_params` or `update_params_from_encrypted_extra` methods without calling the base implementation will still raise the raw `JSONDecodeError`.
   
   To resolve this, you should identify engine-specific classes that override these methods and apply a similar `try-except` block to catch `json.JSONDecodeError` and re-raise it as `SupersetGenericDBErrorException`.
   
   Would you like me to fetch all other comments on this PR to validate them and implement fixes for the remaining issues?
   
   **superset/db_engine_specs/base.py**
   ```
   except json.JSONDecodeError as ex:
                   logger.error(ex, exc_info=True)
                   raise SupersetGenericDBErrorException(message=str(ex)) from ex
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [PR] fix(db_engine_specs): raise SupersetGenericDBErrorException instead of bare-reraising JSONDecodeError [superset]

Posted by "eschutho (via GitHub)" <gi...@apache.org>.

eschutho merged PR #44616:
URL: https://github.com/apache/superset/pull/44616


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org