You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "rusackas (via GitHub)" <gi...@apache.org> on 2026/09/23 00:37:52 UTC
[PR] fix(charts): reject query_context missing datasource/queries on save [superset]
rusackas opened a new pull request, #44549:
URL: https://github.com/apache/superset/pull/44549
### SUMMARY
`QueryContextFactory.create()` requires `datasource` and `queries` as keyword-only arguments, but `ChartPostSchema`/`ChartPutSchema` only checked that a chart's `query_context` was well-formed JSON, not that it carried those keys. A chart saved with an incomplete `query_context` (a buggy integration, hand-edited metadata, a partial migration) fails every subsequent read with a raw `TypeError`, which surfaces to users as a `"Error loading chart datasources"` toast (422) with no indication of the actual cause.
Two earlier attempts at this fix (#35798, #36076) went stale without merging. This is a fresh implementation, informed by the review issues those hit along the way — a required-fields check that could be bypassed by a falsy *parsed* JSON value (`"false"`, `"0"`, `"null"` as `query_context`), and duplicated JSON-decode logic alongside the existing `validate_json`.
**Changes:**
- `superset/utils/schema.py`: `validate_query_context_metadata()`, composed on top of the existing `validate_json` parsing rather than duplicating it, checks the parsed payload is a JSON object with non-empty `datasource` and `queries`. Wired into `ChartPostSchema`/`ChartPutSchema` in place of the plain JSON-syntax check.
- `superset/models/slice.py`: `Slice.get_query_context()` now also treats a stored `query_context` missing either field as unreadable (returns `None`, logs an error) rather than letting the `TypeError` propagate — the same "honest-fail" pattern it already used for malformed JSON just above. This matters because the schema fix alone only prevents *new* bad saves; it can't retroactively fix rows saved before this validation existed (exactly the situation the issue reporter is in).
Unrelated but touched in passing: `slice.py` had a pre-existing ruff finding (`S704`, unsafe `Markup` f-string interpolation) on a line this diff's context otherwise includes — fixed via `Markup(...).format(...)`, which auto-escapes non-`Markup` arguments; verified byte-identical output for the existing case.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A — backend validation change, no UI.
### TESTING INSTRUCTIONS
```bash
pytest tests/unit_tests/utils/test_schema.py
pytest tests/unit_tests/models/slice_test.py -k query_context
pytest tests/unit_tests/charts/test_schemas.py -k query_context
```
Manual: `POST /api/v1/chart/` with `query_context` set to `'{"a": 1}'` (or any object missing `datasource`/`queries`) now returns a 422 validation error instead of saving. A chart whose `query_context` predates this fix (missing those keys) no longer throws on read — `Slice.get_query_context()` returns `None` and logs the reason instead.
### ADDITIONAL INFORMATION
- [x] Has associated issue: Fixes #35774
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on code in PR #44549:
URL: https://github.com/apache/superset/pull/44549#discussion_r4113885130
##########
tests/unit_tests/utils/test_schema.py:
##########
@@ -0,0 +1,129 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Tests for ``superset/utils/schema.py``."""
+
+from __future__ import annotations
+
+import pytest
+from marshmallow import ValidationError
+
+from superset.utils.schema import (
+ is_query_context_metadata_complete,
+ validate_query_context_metadata,
+)
+
+VALID_QUERY_CONTEXT = (
+ '{"datasource": {"id": 1, "type": "table"}, "queries": [{"metrics": ["count"]}]}'
+)
+
+
+def test_validate_query_context_metadata_accepts_complete_payload() -> None:
+ """A query_context with both 'datasource' and 'queries' passes."""
+ validate_query_context_metadata(VALID_QUERY_CONTEXT)
+
+
+def test_validate_query_context_metadata_rejects_invalid_json() -> None:
+ """Malformed JSON is still rejected (parity with the plain validate_json)."""
+ with pytest.raises(ValidationError):
+ validate_query_context_metadata("{not valid json")
+
+
+@pytest.mark.parametrize(
+ "payload",
+ [
+ '{"queries": [{"metrics": ["count"]}]}', # missing datasource
+ '{"datasource": {"id": 1, "type": "table"}}', # missing queries
+ '{"datasource": null, "queries": [{"metrics": ["count"]}]}', # null datasource
+ "{}", # neither
+ ],
+)
+def test_validate_query_context_metadata_rejects_missing_fields(
+ payload: str,
+) -> None:
+ """apache/superset#35774: QueryContextFactory.create() requires 'datasource'
+ and 'queries' as keyword-only arguments; a saved query_context missing
+ either fails every read with a raw TypeError instead of a clear error at
+ save time. 'datasource' must be present and non-empty; 'queries' must be
+ present as a list (see the empty-queries acceptance test below)."""
+ with pytest.raises(ValidationError) as exc_info:
+ validate_query_context_metadata(payload)
+ assert "query_context" in str(exc_info.value).lower()
+
+
+def test_validate_query_context_metadata_accepts_empty_queries() -> None:
+ """An empty 'queries' list is a legitimate, complete query_context -- it
+ round-trips through QueryContextFactory.create() without error, so it
+ must not be rejected as 'missing' (see e.g. semantic view chart saves)."""
+ validate_query_context_metadata(
+ '{"datasource": {"id": 1, "type": "table"}, "queries": []}'
+ )
+
+
+@pytest.mark.parametrize(
+ "payload",
+ ["false", "0", "null", "[]", '"a string"'],
+)
+def test_validate_query_context_metadata_rejects_non_object_json(
+ payload: str,
+) -> None:
+ """A syntactically valid but non-object JSON value (e.g. a JSON-encoded
+ ``false``/``0``/``null``) must not silently bypass the required-fields
+ check -- it is falsy as a raw string check would miss it, but it is never
+ a valid query_context."""
+ with pytest.raises(ValidationError):
+ validate_query_context_metadata(payload)
+
+
+def test_is_query_context_metadata_complete_accepts_dict_with_both_fields() -> None:
+ assert is_query_context_metadata_complete(
+ {"datasource": {"id": 1, "type": "table"}, "queries": [{}]}
+ )
+
+
+def test_is_query_context_metadata_complete_accepts_empty_queries_list() -> None:
+ """An empty 'queries' list is a legitimate, complete value: 'queries' just
+ needs to be present as a list, not non-empty."""
+ assert is_query_context_metadata_complete(
+ {"datasource": {"id": 1, "type": "table"}, "queries": []}
+ )
+
+
+@pytest.mark.parametrize(
+ "value",
+ [
+ None,
+ False,
+ 0,
+ [],
+ "a string",
+ {},
+ {"datasource": {"id": 1}},
+ {"queries": [{}]},
+ {"datasource": {}, "queries": [{}]},
+ {"datasource": {"id": 1}, "queries": None},
+ {"datasource": {"id": 1}, "queries": "not-a-list"},
+ {"datasource": {"id": 1}, "queries": []}, # datasource missing 'type'
+ {"datasource": {"type": "table"}, "queries": []}, # datasource missing 'id'
+ {"datasource": "table:1", "queries": []}, # datasource not a mapping
+ {
+ "datasource": {"id": 1, "type": "not-a-real-type"},
+ "queries": [],
+ }, # datasource 'type' not a DatasourceType member
+ ],
+)
+def test_is_query_context_metadata_complete_rejects_incomplete_values(value):
Review Comment:
<!-- Bito Reply -->
The suggestion to add `value: object` and `-> None` to the `test_is_query_context_metadata_complete_rejects_incomplete_values` function is correct. Applying this change ensures consistency with the existing test suite and adheres to the repository's typing conventions.
**tests/unit_tests/utils/test_schema.py**
```
@pytest.mark.parametrize(
"value",
[
None,
False,
0,
[],
"a string",
{},
{"datasource": {"id": 1}},
{"queries": [{}]},
{"datasource": {}, "queries": [{}]},
{"datasource": {"id": 1}, "queries": None},
{"datasource": {"id": 1}, "queries": "not-a-list"},
{"datasource": {"id": 1}, "queries": []}, # datasource missing 'type'
{"datasource": {"type": "table"}, "queries": []}, # datasource missing 'id'
{"datasource": "table:1", "queries": []}, # datasource not a mapping
{
"datasource": {"id": 1, "type": "not-a-real-type"},
"queries": [],
}, # datasource 'type' not a DatasourceType member
],
)
def test_is_query_context_metadata_complete_rejects_incomplete_values(value: object) -> None:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #44549:
URL: https://github.com/apache/superset/pull/44549#issuecomment-5788675946
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #44549:
URL: https://github.com/apache/superset/pull/44549#issuecomment-5794482069
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #44549:
URL: https://github.com/apache/superset/pull/44549#discussion_r4263864398
##########
superset/utils/schema.py:
##########
@@ -56,6 +57,61 @@ def validate_json(value: Union[bytes, bytearray, str]) -> None:
raise ValidationError("JSON not valid") from ex
+def is_query_context_metadata_complete(value: Any) -> bool:
+ """
+ Whether a parsed ``query_context`` payload carries the ``datasource`` and
+ ``queries`` keys ``QueryContextFactory.create()`` requires (apache/superset
+ #35774): both are required keyword-only arguments there, and a chart saved
+ without them fails every subsequent read with a raw ``TypeError`` instead
+ of a clear error at save time. ``datasource`` must be a mapping carrying
+ the ``id``/``type`` keys ``DatasourceDict`` requires -- ``_convert_to_model``
+ indexes both directly and raises ``KeyError`` if either is absent, and
+ passes ``type`` straight to ``DatasourceType(...)``, which raises
+ ``ValueError`` for a value outside that enum -- but ``queries`` may
+ legitimately be an empty list (a chart with no query objects yet still
+ round-trips through ``create()`` without error), so only its presence as
+ a list is checked, not its truthiness.
+
+ :param value: the object obtained by JSON-decoding a query_context string
+ """
+ if not isinstance(value, dict) or not isinstance(value.get("queries"), list):
+ return False
Review Comment:
Elements unchecked: `queries: ["x"]`/`[null]` still raises the raw TypeError on read.
```suggestion
if not isinstance(value, dict) or not isinstance(value.get("queries"), list):
return False
if not all(isinstance(query, dict) for query in value["queries"]):
return False
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on code in PR #44549:
URL: https://github.com/apache/superset/pull/44549#discussion_r4080745626
##########
superset/models/slice.py:
##########
@@ -404,12 +405,26 @@ def form_data(self) -> dict[str, Any]:
def get_query_context(self) -> QueryContext | None:
if self.query_context:
try:
- return self.get_query_context_factory().create(
- **{**json.loads(self.query_context), "current_slice": self}
- )
+ query_context_payload = json.loads(self.query_context)
except json.JSONDecodeError as ex:
logger.error("Malformed json in slice's query context", exc_info=True)
logger.exception(ex)
+ return None
+ # ``QueryContextFactory.create()`` requires ``datasource`` and
+ # ``queries`` as keyword-only arguments; a row saved before the
+ # ChartPostSchema/ChartPutSchema validation added in
+ # apache/superset#35774 can still be missing either, which would
+ # otherwise raise a raw TypeError on every read of this chart.
+ if not is_query_context_metadata_complete(query_context_payload):
+ logger.error(
+ "Slice %s query_context is missing required "
+ "'datasource'/'queries' fields",
+ self.id,
+ )
+ return None
Review Comment:
Good catch, tightened `is_query_context_metadata_complete` to require `datasource` carry `id`/`type` instead of just being truthy, since that's what `_convert_to_model` actually indexes.
##########
superset/utils/schema.py:
##########
@@ -56,6 +56,44 @@ def validate_json(value: Union[bytes, bytearray, str]) -> None:
raise ValidationError("JSON not valid") from ex
+def is_query_context_metadata_complete(value: Any) -> bool:
+ """
+ Whether a parsed ``query_context`` payload carries the ``datasource`` and
+ ``queries`` keys ``QueryContextFactory.create()`` requires (apache/superset
+ #35774): both are required keyword-only arguments there, and a chart saved
+ without them fails every subsequent read with a raw ``TypeError`` instead
+ of a clear error at save time.
+
+ :param value: the object obtained by JSON-decoding a query_context string
+ """
+ return (
+ isinstance(value, dict)
+ and bool(value.get("datasource"))
+ and bool(value.get("queries"))
+ )
Review Comment:
Fixed the datasource half, now requires `id`/`type` instead of a truthy check. Left per-query validation alone, query object shape varies too much by viz type to check without risking false rejections on legit payloads.
##########
superset/models/slice.py:
##########
@@ -449,7 +464,7 @@ def slice_link(self) -> Markup:
# SCRIPT_NAME (the application_root). `Slice.url` itself stays router-
# relative so frontend callers can apply ensureAppRoot exactly once.
href = url_for("ExploreView.root", slice_id=self.id)
- return Markup(f'<a href="{href}">{name}</a>')
+ return Markup('<a href="{}">{}</a>').format(href, name)
Review Comment:
That's the current code already, two `{}` placeholders and two args (`href`, `name`). Might be reviewing a stale diff.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on code in PR #44549:
URL: https://github.com/apache/superset/pull/44549#discussion_r4088272905
##########
tests/unit_tests/utils/test_schema.py:
##########
@@ -0,0 +1,129 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Tests for ``superset/utils/schema.py``."""
+
+from __future__ import annotations
+
+import pytest
+from marshmallow import ValidationError
+
+from superset.utils.schema import (
+ is_query_context_metadata_complete,
+ validate_query_context_metadata,
+)
+
+VALID_QUERY_CONTEXT = (
+ '{"datasource": {"id": 1, "type": "table"}, "queries": [{"metrics": ["count"]}]}'
+)
+
+
+def test_validate_query_context_metadata_accepts_complete_payload() -> None:
+ """A query_context with both 'datasource' and 'queries' passes."""
+ validate_query_context_metadata(VALID_QUERY_CONTEXT)
+
+
+def test_validate_query_context_metadata_rejects_invalid_json() -> None:
+ """Malformed JSON is still rejected (parity with the plain validate_json)."""
+ with pytest.raises(ValidationError):
+ validate_query_context_metadata("{not valid json")
+
+
+@pytest.mark.parametrize(
+ "payload",
+ [
+ '{"queries": [{"metrics": ["count"]}]}', # missing datasource
+ '{"datasource": {"id": 1, "type": "table"}}', # missing queries
+ '{"datasource": null, "queries": [{"metrics": ["count"]}]}', # null datasource
+ "{}", # neither
+ ],
+)
+def test_validate_query_context_metadata_rejects_missing_fields(
+ payload: str,
+) -> None:
+ """apache/superset#35774: QueryContextFactory.create() requires 'datasource'
+ and 'queries' as keyword-only arguments; a saved query_context missing
+ either fails every read with a raw TypeError instead of a clear error at
+ save time. 'datasource' must be present and non-empty; 'queries' must be
+ present as a list (see the empty-queries acceptance test below)."""
+ with pytest.raises(ValidationError) as exc_info:
+ validate_query_context_metadata(payload)
+ assert "query_context" in str(exc_info.value).lower()
+
+
+def test_validate_query_context_metadata_accepts_empty_queries() -> None:
+ """An empty 'queries' list is a legitimate, complete query_context -- it
+ round-trips through QueryContextFactory.create() without error, so it
+ must not be rejected as 'missing' (see e.g. semantic view chart saves)."""
+ validate_query_context_metadata(
+ '{"datasource": {"id": 1, "type": "table"}, "queries": []}'
+ )
+
+
+@pytest.mark.parametrize(
+ "payload",
+ ["false", "0", "null", "[]", '"a string"'],
+)
+def test_validate_query_context_metadata_rejects_non_object_json(
+ payload: str,
+) -> None:
+ """A syntactically valid but non-object JSON value (e.g. a JSON-encoded
+ ``false``/``0``/``null``) must not silently bypass the required-fields
+ check -- it is falsy as a raw string check would miss it, but it is never
+ a valid query_context."""
+ with pytest.raises(ValidationError):
+ validate_query_context_metadata(payload)
+
+
+def test_is_query_context_metadata_complete_accepts_dict_with_both_fields() -> None:
+ assert is_query_context_metadata_complete(
+ {"datasource": {"id": 1, "type": "table"}, "queries": [{}]}
+ )
+
+
+def test_is_query_context_metadata_complete_accepts_empty_queries_list() -> None:
+ """An empty 'queries' list is a legitimate, complete value: 'queries' just
+ needs to be present as a list, not non-empty."""
+ assert is_query_context_metadata_complete(
+ {"datasource": {"id": 1, "type": "table"}, "queries": []}
+ )
+
+
+@pytest.mark.parametrize(
+ "value",
+ [
+ None,
+ False,
+ 0,
+ [],
+ "a string",
+ {},
+ {"datasource": {"id": 1}},
+ {"queries": [{}]},
+ {"datasource": {}, "queries": [{}]},
+ {"datasource": {"id": 1}, "queries": None},
+ {"datasource": {"id": 1}, "queries": "not-a-list"},
+ {"datasource": {"id": 1}, "queries": []}, # datasource missing 'type'
+ {"datasource": {"type": "table"}, "queries": []}, # datasource missing 'id'
+ {"datasource": "table:1", "queries": []}, # datasource not a mapping
+ {
+ "datasource": {"id": 1, "type": "not-a-real-type"},
+ "queries": [],
+ }, # datasource 'type' not a DatasourceType member
+ ],
+)
+def test_is_query_context_metadata_complete_rejects_incomplete_values(value):
Review Comment:
<div>
<div id="suggestion">
<div id="issue"><b>Missing test type annotations</b></div>
<div id="fix">
This is the only test function in the new file without annotations — siblings like `test_validate_query_context_metadata_rejects_missing_fields(payload: str) -> None` are fully typed. This violates the repo's typing convention for tests (BITO.md rule 7819) and is internally inconsistent within the same diff. Annotate `value: object` and `-> None`.
</div>
</div>
<small><i>Code Review Run #f0f1ba</i></small>
</div>
---
Should Bito avoid suggestions like this for future reviews? (<a href=https://alpha.bito.ai/home/ai-agents/review-rules>Manage Rules</a>)
- [ ] Yes, avoid them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #44549:
URL: https://github.com/apache/superset/pull/44549#issuecomment-5853177046
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on code in PR #44549:
URL: https://github.com/apache/superset/pull/44549#discussion_r4271674023
##########
superset/utils/schema.py:
##########
@@ -56,6 +57,63 @@ def validate_json(value: Union[bytes, bytearray, str]) -> None:
raise ValidationError("JSON not valid") from ex
+def is_query_context_metadata_complete(value: Any) -> bool:
+ """
+ Whether a parsed ``query_context`` payload carries the ``datasource`` and
+ ``queries`` keys ``QueryContextFactory.create()`` requires (apache/superset
+ #35774): both are required keyword-only arguments there, and a chart saved
+ without them fails every subsequent read with a raw ``TypeError`` instead
+ of a clear error at save time. ``datasource`` must be a mapping carrying
+ the ``id``/``type`` keys ``DatasourceDict`` requires -- ``_convert_to_model``
+ indexes both directly and raises ``KeyError`` if either is absent, and
+ passes ``type`` straight to ``DatasourceType(...)``, which raises
+ ``ValueError`` for a value outside that enum -- but ``queries`` may
+ legitimately be an empty list (a chart with no query objects yet still
+ round-trips through ``create()`` without error), so it is only required
+ to be a list of mappings, not to be non-empty.
+
+ :param value: the object obtained by JSON-decoding a query_context string
+ """
+ if not isinstance(value, dict) or not isinstance(value.get("queries"), list):
+ return False
+ if not all(isinstance(query, dict) for query in value["queries"]):
+ return False
+ datasource = value.get("datasource")
+ if not isinstance(datasource, dict) or not datasource.get("id"):
+ return False
+ datasource_type = datasource.get("type")
+ if not isinstance(datasource_type, str):
+ return False
+ try:
+ DatasourceType(datasource_type)
+ except ValueError:
+ return False
+ return True
+
+
+def validate_query_context_metadata(value: Union[bytes, bytearray, str]) -> None:
+ """
+ Validator for a chart's ``query_context`` field: beyond being well-formed
+ JSON (see ``validate_json``), it must be a JSON object carrying a
+ non-empty ``datasource`` key and a ``queries`` key that is a list
+ (apache/superset#35774).
+
+ :raises ValidationError: if value is not valid JSON, or is valid JSON that
+ is not an object with a non-empty ``datasource`` key and a ``queries``
+ key holding a list
+ :param value: an object that should be parseable to a query_context JSON object
+ """
+ try:
+ parsed = json.loads(value)
+ except json.JSONDecodeError as ex:
+ raise ValidationError("JSON not valid") from ex
Review Comment:
Good catch, applied: empty string is skipped again, with a test.
##########
superset/utils/schema.py:
##########
@@ -56,6 +57,63 @@ def validate_json(value: Union[bytes, bytearray, str]) -> None:
raise ValidationError("JSON not valid") from ex
+def is_query_context_metadata_complete(value: Any) -> bool:
+ """
+ Whether a parsed ``query_context`` payload carries the ``datasource`` and
+ ``queries`` keys ``QueryContextFactory.create()`` requires (apache/superset
+ #35774): both are required keyword-only arguments there, and a chart saved
+ without them fails every subsequent read with a raw ``TypeError`` instead
+ of a clear error at save time. ``datasource`` must be a mapping carrying
+ the ``id``/``type`` keys ``DatasourceDict`` requires -- ``_convert_to_model``
+ indexes both directly and raises ``KeyError`` if either is absent, and
+ passes ``type`` straight to ``DatasourceType(...)``, which raises
+ ``ValueError`` for a value outside that enum -- but ``queries`` may
+ legitimately be an empty list (a chart with no query objects yet still
+ round-trips through ``create()`` without error), so it is only required
+ to be a list of mappings, not to be non-empty.
+
+ :param value: the object obtained by JSON-decoding a query_context string
+ """
+ if not isinstance(value, dict) or not isinstance(value.get("queries"), list):
+ return False
+ if not all(isinstance(query, dict) for query in value["queries"]):
+ return False
+ datasource = value.get("datasource")
+ if not isinstance(datasource, dict) or not datasource.get("id"):
+ return False
+ datasource_type = datasource.get("type")
+ if not isinstance(datasource_type, str):
+ return False
+ try:
+ DatasourceType(datasource_type)
+ except ValueError:
+ return False
+ return True
+
+
+def validate_query_context_metadata(value: Union[bytes, bytearray, str]) -> None:
+ """
+ Validator for a chart's ``query_context`` field: beyond being well-formed
+ JSON (see ``validate_json``), it must be a JSON object carrying a
+ non-empty ``datasource`` key and a ``queries`` key that is a list
+ (apache/superset#35774).
+
+ :raises ValidationError: if value is not valid JSON, or is valid JSON that
+ is not an object with a non-empty ``datasource`` key and a ``queries``
+ key holding a list
+ :param value: an object that should be parseable to a query_context JSON object
+ """
+ try:
+ parsed = json.loads(value)
+ except json.JSONDecodeError as ex:
+ raise ValidationError("JSON not valid") from ex
+ if not is_query_context_metadata_complete(parsed):
+ raise ValidationError(
+ "query_context must be a JSON object with a non-empty "
+ "'datasource' key and a 'queries' key holding a list"
+ )
Review Comment:
Applied, the error now names the `id`/`type` and list-of-objects requirements.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on code in PR #44549:
URL: https://github.com/apache/superset/pull/44549#discussion_r4082354504
##########
superset/utils/schema.py:
##########
@@ -56,6 +56,54 @@ def validate_json(value: Union[bytes, bytearray, str]) -> None:
raise ValidationError("JSON not valid") from ex
+def is_query_context_metadata_complete(value: Any) -> bool:
+ """
+ Whether a parsed ``query_context`` payload carries the ``datasource`` and
+ ``queries`` keys ``QueryContextFactory.create()`` requires (apache/superset
+ #35774): both are required keyword-only arguments there, and a chart saved
+ without them fails every subsequent read with a raw ``TypeError`` instead
+ of a clear error at save time. ``datasource`` must be a mapping carrying
+ the ``id``/``type`` keys ``DatasourceDict`` requires -- ``_convert_to_model``
+ indexes both directly and raises ``KeyError`` if either is absent -- but
+ ``queries`` may legitimately be an empty list (a chart with no query
+ objects yet still round-trips through ``create()`` without error), so only
+ its presence as a list is checked, not its truthiness.
+
+ :param value: the object obtained by JSON-decoding a query_context string
+ """
+ if not isinstance(value, dict) or not isinstance(value.get("queries"), list):
+ return False
+ datasource = value.get("datasource")
+ return (
+ isinstance(datasource, dict)
+ and bool(datasource.get("id"))
+ and bool(datasource.get("type"))
Review Comment:
<div>
<div id="suggestion">
<div id="issue"><b>invalid datasource type unvalidated</b></div>
<div id="fix">
`bool(datasource.get("type"))` only checks presence, but `QueryContextFactory._convert_to_model` calls `DatasourceType(datasource["type"])`, which raises `ValueError` for any non-member string. A chart saved with an invalid `type` passes this check yet still raises a raw `ValueError` on every read via `Slice.get_query_context` — the same failure class the docstring says this guard prevents. Consider validating the value against `DatasourceType` members.
</div>
</div>
<small><i>Code Review Run #df4558</i></small>
</div>
---
Should Bito avoid suggestions like this for future reviews? (<a href=https://alpha.bito.ai/home/ai-agents/review-rules>Manage Rules</a>)
- [ ] Yes, avoid them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #44549:
URL: https://github.com/apache/superset/pull/44549#issuecomment-5786966337
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on PR #44549:
URL: https://github.com/apache/superset/pull/44549#issuecomment-6112126099
## CodeAnt PR Risk: Medium Risk
- The PR needs attention before merging because malformed stored query contexts can still reach the factory.
- Validation checks datasource metadata and that queries is a list, but not whether individual query entries are valid; malformed legacy entries may still fail during reads.
**Assessed commit:** `41d83878b7b3`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on code in PR #44549:
URL: https://github.com/apache/superset/pull/44549#discussion_r4113884799
##########
tests/unit_tests/utils/test_schema.py:
##########
@@ -0,0 +1,129 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Tests for ``superset/utils/schema.py``."""
+
+from __future__ import annotations
+
+import pytest
+from marshmallow import ValidationError
+
+from superset.utils.schema import (
+ is_query_context_metadata_complete,
+ validate_query_context_metadata,
+)
+
+VALID_QUERY_CONTEXT = (
+ '{"datasource": {"id": 1, "type": "table"}, "queries": [{"metrics": ["count"]}]}'
+)
+
+
+def test_validate_query_context_metadata_accepts_complete_payload() -> None:
+ """A query_context with both 'datasource' and 'queries' passes."""
+ validate_query_context_metadata(VALID_QUERY_CONTEXT)
+
+
+def test_validate_query_context_metadata_rejects_invalid_json() -> None:
+ """Malformed JSON is still rejected (parity with the plain validate_json)."""
+ with pytest.raises(ValidationError):
+ validate_query_context_metadata("{not valid json")
+
+
+@pytest.mark.parametrize(
+ "payload",
+ [
+ '{"queries": [{"metrics": ["count"]}]}', # missing datasource
+ '{"datasource": {"id": 1, "type": "table"}}', # missing queries
+ '{"datasource": null, "queries": [{"metrics": ["count"]}]}', # null datasource
+ "{}", # neither
+ ],
+)
+def test_validate_query_context_metadata_rejects_missing_fields(
+ payload: str,
+) -> None:
+ """apache/superset#35774: QueryContextFactory.create() requires 'datasource'
+ and 'queries' as keyword-only arguments; a saved query_context missing
+ either fails every read with a raw TypeError instead of a clear error at
+ save time. 'datasource' must be present and non-empty; 'queries' must be
+ present as a list (see the empty-queries acceptance test below)."""
+ with pytest.raises(ValidationError) as exc_info:
+ validate_query_context_metadata(payload)
+ assert "query_context" in str(exc_info.value).lower()
+
+
+def test_validate_query_context_metadata_accepts_empty_queries() -> None:
+ """An empty 'queries' list is a legitimate, complete query_context -- it
+ round-trips through QueryContextFactory.create() without error, so it
+ must not be rejected as 'missing' (see e.g. semantic view chart saves)."""
+ validate_query_context_metadata(
+ '{"datasource": {"id": 1, "type": "table"}, "queries": []}'
+ )
+
+
+@pytest.mark.parametrize(
+ "payload",
+ ["false", "0", "null", "[]", '"a string"'],
+)
+def test_validate_query_context_metadata_rejects_non_object_json(
+ payload: str,
+) -> None:
+ """A syntactically valid but non-object JSON value (e.g. a JSON-encoded
+ ``false``/``0``/``null``) must not silently bypass the required-fields
+ check -- it is falsy as a raw string check would miss it, but it is never
+ a valid query_context."""
+ with pytest.raises(ValidationError):
+ validate_query_context_metadata(payload)
+
+
+def test_is_query_context_metadata_complete_accepts_dict_with_both_fields() -> None:
+ assert is_query_context_metadata_complete(
+ {"datasource": {"id": 1, "type": "table"}, "queries": [{}]}
+ )
+
+
+def test_is_query_context_metadata_complete_accepts_empty_queries_list() -> None:
+ """An empty 'queries' list is a legitimate, complete value: 'queries' just
+ needs to be present as a list, not non-empty."""
+ assert is_query_context_metadata_complete(
+ {"datasource": {"id": 1, "type": "table"}, "queries": []}
+ )
+
+
+@pytest.mark.parametrize(
+ "value",
+ [
+ None,
+ False,
+ 0,
+ [],
+ "a string",
+ {},
+ {"datasource": {"id": 1}},
+ {"queries": [{}]},
+ {"datasource": {}, "queries": [{}]},
+ {"datasource": {"id": 1}, "queries": None},
+ {"datasource": {"id": 1}, "queries": "not-a-list"},
+ {"datasource": {"id": 1}, "queries": []}, # datasource missing 'type'
+ {"datasource": {"type": "table"}, "queries": []}, # datasource missing 'id'
+ {"datasource": "table:1", "queries": []}, # datasource not a mapping
+ {
+ "datasource": {"id": 1, "type": "not-a-real-type"},
+ "queries": [],
+ }, # datasource 'type' not a DatasourceType member
+ ],
+)
+def test_is_query_context_metadata_complete_rejects_incomplete_values(value):
Review Comment:
Added the missing annotations (`value: object`, `-> None`) to match the other parametrized tests in the file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #44549:
URL: https://github.com/apache/superset/pull/44549#discussion_r4078001986
##########
superset/models/slice.py:
##########
@@ -404,12 +405,26 @@ def form_data(self) -> dict[str, Any]:
def get_query_context(self) -> QueryContext | None:
if self.query_context:
try:
- return self.get_query_context_factory().create(
- **{**json.loads(self.query_context), "current_slice": self}
- )
+ query_context_payload = json.loads(self.query_context)
except json.JSONDecodeError as ex:
logger.error("Malformed json in slice's query context", exc_info=True)
logger.exception(ex)
+ return None
+ # ``QueryContextFactory.create()`` requires ``datasource`` and
+ # ``queries`` as keyword-only arguments; a row saved before the
+ # ChartPostSchema/ChartPutSchema validation added in
+ # apache/superset#35774 can still be missing either, which would
+ # otherwise raise a raw TypeError on every read of this chart.
+ if not is_query_context_metadata_complete(query_context_payload):
+ logger.error(
+ "Slice %s query_context is missing required "
+ "'datasource'/'queries' fields",
+ self.id,
+ )
+ return None
Review Comment:
**Suggestion:** Truthiness-only metadata validation lets legacy contexts with malformed non-empty datasource or query values reach the factory, so reads can still raise instead of returning `None`.
**Assessment:** 🟠 `Major` · 🔁 `Occurrence: Rarely` · 🏷️ `Error handling`
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=1c90314ecaf84186a387d3baeb00ba6e&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=1c90314ecaf84186a387d3baeb00ba6e&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/models/slice.py
**Line:** 418:424
**Comment:**
*Error Handling: Truthiness-only metadata validation lets legacy contexts with malformed non-empty datasource or query values reach the factory, so reads can still raise instead of returning `None`.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F44549&comment_hash=ca87bc7a398823d8aeac25c99bb0d27b0b0baf56395e93d5a49aa134d75fa7be&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F44549&comment_hash=ca87bc7a398823d8aeac25c99bb0d27b0b0baf56395e93d5a49aa134d75fa7be&reaction=dislike'>👎</a>
##########
superset/utils/schema.py:
##########
@@ -56,6 +56,44 @@ def validate_json(value: Union[bytes, bytearray, str]) -> None:
raise ValidationError("JSON not valid") from ex
+def is_query_context_metadata_complete(value: Any) -> bool:
+ """
+ Whether a parsed ``query_context`` payload carries the ``datasource`` and
+ ``queries`` keys ``QueryContextFactory.create()`` requires (apache/superset
+ #35774): both are required keyword-only arguments there, and a chart saved
+ without them fails every subsequent read with a raw ``TypeError`` instead
+ of a clear error at save time.
+
+ :param value: the object obtained by JSON-decoding a query_context string
+ """
+ return (
+ isinstance(value, dict)
+ and bool(value.get("datasource"))
+ and bool(value.get("queries"))
+ )
Review Comment:
**Suggestion:** `datasource` and each query are only checked for truthiness, so malformed values such as missing datasource `type` still pass validation and later raise `KeyError` or `TypeError`.
**Assessment:** 🟠 `Major` · 🔁 `Occurrence: Rarely` · 🏷️ `Type error`
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=d1399fab3d5c44458d5b9dc95613ad71&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=d1399fab3d5c44458d5b9dc95613ad71&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/utils/schema.py
**Line:** 69:73
**Comment:**
*Type Error: `datasource` and each query are only checked for truthiness, so malformed values such as missing datasource `type` still pass validation and later raise `KeyError` or `TypeError`.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F44549&comment_hash=c351c2f22d212f388e82287a7c919a3285a14b3564d80bcb96180a8711d95ca0&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F44549&comment_hash=c351c2f22d212f388e82287a7c919a3285a14b3564d80bcb96180a8711d95ca0&reaction=dislike'>👎</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #44549:
URL: https://github.com/apache/superset/pull/44549#issuecomment-5788637138
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/44549?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 77.36%. Comparing base ([`7a9fe06`](https://app.codecov.io/gh/apache/superset/commit/7a9fe06762b45d104ad098779116afdfee50bf30?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`8b596b5`](https://app.codecov.io/gh/apache/superset/commit/8b596b5b37f403ccffdfc8e1fbe6f3c841a4c482?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #44549 +/- ##
===========================================
+ Coverage 66.19% 77.36% +11.17%
===========================================
Files 2949 2949
Lines 177374 177389 +15
Branches 41102 41104 +2
===========================================
+ Hits 117411 137244 +19833
+ Misses 57460 37309 -20151
- Partials 2503 2836 +333
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/44549/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/44549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `36.92% <17.64%> (-0.01%)` | :arrow_down: |
| [mysql](https://app.codecov.io/gh/apache/superset/pull/44549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [postgres](https://app.codecov.io/gh/apache/superset/pull/44549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [presto](https://app.codecov.io/gh/apache/superset/pull/44549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.86% <17.64%> (-0.01%)` | :arrow_down: |
| [python](https://app.codecov.io/gh/apache/superset/pull/44549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `78.34% <100.00%> (+21.86%)` | :arrow_up: |
| [sqlite](https://app.codecov.io/gh/apache/superset/pull/44549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unit](https://app.codecov.io/gh/apache/superset/pull/44549/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `77.48% <100.00%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/44549?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "netlify[bot] (via GitHub)" <gi...@apache.org>.
netlify[bot] commented on PR #44549:
URL: https://github.com/apache/superset/pull/44549#issuecomment-5788449732
### <span aria-hidden="true">✅</span> Deploy Preview for *superset-docs-preview* ready!
| Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | 8b596b5b37f403ccffdfc8e1fbe6f3c841a4c482 |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/projects/superset-docs-preview/deploys/6ab34656a8f45c00080c29a1 |
|<span aria-hidden="true">😎</span> Deploy Preview | [https://deploy-preview-44549--superset-docs-preview.netlify.app](https://deploy-preview-44549--superset-docs-preview.netlify.app) |
|<span aria-hidden="true">📱</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br /><br /><br />_Use your smartphone camera to open QR code link._</details> |
|<span aria-hidden="true">🤖</span> Make changes | [Run an agent on this branch](https://app.netlify.com/projects/superset-docs-preview/agent-runs#DcrJDYAgEADAXvYtERXw6MAyliuSKCBBgzH2Lr95zAsxhSNmWAAakAm92qqtK-3Ax5GR8zLpISr4bEomN-5OY3bB161N3MOz6voFyoEJLnCyjCtK6URVP2MH3w8) |
---
<!-- [superset-docs-preview Preview](https://deploy-preview-44549--superset-docs-preview.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify project configuration](https://app.netlify.com/projects/superset-docs-preview/configuration/notifications#deploy-notifications)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #44549:
URL: https://github.com/apache/superset/pull/44549#issuecomment-5804396121
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on code in PR #44549:
URL: https://github.com/apache/superset/pull/44549#discussion_r4078038604
##########
superset/models/slice.py:
##########
@@ -449,7 +464,7 @@ def slice_link(self) -> Markup:
# SCRIPT_NAME (the application_root). `Slice.url` itself stays router-
# relative so frontend callers can apply ensureAppRoot exactly once.
href = url_for("ExploreView.root", slice_id=self.id)
- return Markup(f'<a href="{href}">{name}</a>')
+ return Markup('<a href="{}">{}</a>').format(href, name)
Review Comment:
<div>
<div id="suggestion">
<div id="issue"><b>IndexError in slice_link format</b></div>
<div id="fix">
`Markup('<a href="{}{}">{}</a>').format(href, name)` has 3 placeholders but only 2 args, so every access of `slice_link` raises `IndexError: tuple index out of range` (confirmed: `Markup(...).format('http://x','name')` fails). The prior f-string had 2 placeholders. This crashes the FAB chart list view that renders this raw HTML. Fix: `Markup('<a href="{}">{}</a>').format(href, name)`.
</div>
</div>
<small><i>Code Review Run #1c1531</i></small>
</div>
---
Should Bito avoid suggestions like this for future reviews? (<a href=https://alpha.bito.ai/home/ai-agents/review-rules>Manage Rules</a>)
- [ ] Yes, avoid them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas merged PR #44549:
URL: https://github.com/apache/superset/pull/44549
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #44549:
URL: https://github.com/apache/superset/pull/44549#issuecomment-5798747178
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on code in PR #44549:
URL: https://github.com/apache/superset/pull/44549#discussion_r4083026104
##########
superset/utils/schema.py:
##########
@@ -56,6 +56,54 @@ def validate_json(value: Union[bytes, bytearray, str]) -> None:
raise ValidationError("JSON not valid") from ex
+def is_query_context_metadata_complete(value: Any) -> bool:
+ """
+ Whether a parsed ``query_context`` payload carries the ``datasource`` and
+ ``queries`` keys ``QueryContextFactory.create()`` requires (apache/superset
+ #35774): both are required keyword-only arguments there, and a chart saved
+ without them fails every subsequent read with a raw ``TypeError`` instead
+ of a clear error at save time. ``datasource`` must be a mapping carrying
+ the ``id``/``type`` keys ``DatasourceDict`` requires -- ``_convert_to_model``
+ indexes both directly and raises ``KeyError`` if either is absent -- but
+ ``queries`` may legitimately be an empty list (a chart with no query
+ objects yet still round-trips through ``create()`` without error), so only
+ its presence as a list is checked, not its truthiness.
+
+ :param value: the object obtained by JSON-decoding a query_context string
+ """
+ if not isinstance(value, dict) or not isinstance(value.get("queries"), list):
+ return False
+ datasource = value.get("datasource")
+ return (
+ isinstance(datasource, dict)
+ and bool(datasource.get("id"))
+ and bool(datasource.get("type"))
Review Comment:
<!-- Bito Reply -->
The update correctly addresses the concern by validating the `type` field against the `DatasourceType` members. This ensures that only valid datasource types are accepted, preventing potential runtime errors during model conversion.
**superset/utils/schema.py**
```
datasource = value.get("datasource")
return (
isinstance(datasource, dict)
and bool(datasource.get("id"))
and datasource.get("type") in {t.value for t in DatasourceType}
)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on code in PR #44549:
URL: https://github.com/apache/superset/pull/44549#discussion_r4083024377
##########
superset/utils/schema.py:
##########
@@ -56,6 +56,54 @@ def validate_json(value: Union[bytes, bytearray, str]) -> None:
raise ValidationError("JSON not valid") from ex
+def is_query_context_metadata_complete(value: Any) -> bool:
+ """
+ Whether a parsed ``query_context`` payload carries the ``datasource`` and
+ ``queries`` keys ``QueryContextFactory.create()`` requires (apache/superset
+ #35774): both are required keyword-only arguments there, and a chart saved
+ without them fails every subsequent read with a raw ``TypeError`` instead
+ of a clear error at save time. ``datasource`` must be a mapping carrying
+ the ``id``/``type`` keys ``DatasourceDict`` requires -- ``_convert_to_model``
+ indexes both directly and raises ``KeyError`` if either is absent -- but
+ ``queries`` may legitimately be an empty list (a chart with no query
+ objects yet still round-trips through ``create()`` without error), so only
+ its presence as a list is checked, not its truthiness.
+
+ :param value: the object obtained by JSON-decoding a query_context string
+ """
+ if not isinstance(value, dict) or not isinstance(value.get("queries"), list):
+ return False
+ datasource = value.get("datasource")
+ return (
+ isinstance(datasource, dict)
+ and bool(datasource.get("id"))
+ and bool(datasource.get("type"))
Review Comment:
Good catch, added a check that `type` is a valid `DatasourceType` member instead of just present.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #44549:
URL: https://github.com/apache/superset/pull/44549#discussion_r4270451394
##########
superset/utils/schema.py:
##########
@@ -56,6 +57,63 @@ def validate_json(value: Union[bytes, bytearray, str]) -> None:
raise ValidationError("JSON not valid") from ex
+def is_query_context_metadata_complete(value: Any) -> bool:
+ """
+ Whether a parsed ``query_context`` payload carries the ``datasource`` and
+ ``queries`` keys ``QueryContextFactory.create()`` requires (apache/superset
+ #35774): both are required keyword-only arguments there, and a chart saved
+ without them fails every subsequent read with a raw ``TypeError`` instead
+ of a clear error at save time. ``datasource`` must be a mapping carrying
+ the ``id``/``type`` keys ``DatasourceDict`` requires -- ``_convert_to_model``
+ indexes both directly and raises ``KeyError`` if either is absent, and
+ passes ``type`` straight to ``DatasourceType(...)``, which raises
+ ``ValueError`` for a value outside that enum -- but ``queries`` may
+ legitimately be an empty list (a chart with no query objects yet still
+ round-trips through ``create()`` without error), so it is only required
+ to be a list of mappings, not to be non-empty.
+
+ :param value: the object obtained by JSON-decoding a query_context string
+ """
+ if not isinstance(value, dict) or not isinstance(value.get("queries"), list):
+ return False
+ if not all(isinstance(query, dict) for query in value["queries"]):
+ return False
+ datasource = value.get("datasource")
+ if not isinstance(datasource, dict) or not datasource.get("id"):
+ return False
+ datasource_type = datasource.get("type")
+ if not isinstance(datasource_type, str):
+ return False
+ try:
+ DatasourceType(datasource_type)
+ except ValueError:
+ return False
+ return True
+
+
+def validate_query_context_metadata(value: Union[bytes, bytearray, str]) -> None:
+ """
+ Validator for a chart's ``query_context`` field: beyond being well-formed
+ JSON (see ``validate_json``), it must be a JSON object carrying a
+ non-empty ``datasource`` key and a ``queries`` key that is a list
+ (apache/superset#35774).
+
+ :raises ValidationError: if value is not valid JSON, or is valid JSON that
+ is not an object with a non-empty ``datasource`` key and a ``queries``
+ key holding a list
+ :param value: an object that should be parseable to a query_context JSON object
+ """
+ try:
+ parsed = json.loads(value)
+ except json.JSONDecodeError as ex:
+ raise ValidationError("JSON not valid") from ex
+ if not is_query_context_metadata_complete(parsed):
+ raise ValidationError(
+ "query_context must be a JSON object with a non-empty "
+ "'datasource' key and a 'queries' key holding a list"
+ )
Review Comment:
Nit: `{"queries": ["x"]}`, or a datasource without `id` or with an unknown `type`, is rejected with "...a 'queries' key holding a list", which that payload already satisfies, so the caller cannot tell what to fix.
```suggestion
raise ValidationError(
"query_context must be a JSON object with a 'datasource' object "
"carrying a valid 'id' and 'type', and a 'queries' list of objects"
)
```
##########
superset/utils/schema.py:
##########
@@ -56,6 +57,63 @@ def validate_json(value: Union[bytes, bytearray, str]) -> None:
raise ValidationError("JSON not valid") from ex
+def is_query_context_metadata_complete(value: Any) -> bool:
+ """
+ Whether a parsed ``query_context`` payload carries the ``datasource`` and
+ ``queries`` keys ``QueryContextFactory.create()`` requires (apache/superset
+ #35774): both are required keyword-only arguments there, and a chart saved
+ without them fails every subsequent read with a raw ``TypeError`` instead
+ of a clear error at save time. ``datasource`` must be a mapping carrying
+ the ``id``/``type`` keys ``DatasourceDict`` requires -- ``_convert_to_model``
+ indexes both directly and raises ``KeyError`` if either is absent, and
+ passes ``type`` straight to ``DatasourceType(...)``, which raises
+ ``ValueError`` for a value outside that enum -- but ``queries`` may
+ legitimately be an empty list (a chart with no query objects yet still
+ round-trips through ``create()`` without error), so it is only required
+ to be a list of mappings, not to be non-empty.
+
+ :param value: the object obtained by JSON-decoding a query_context string
+ """
+ if not isinstance(value, dict) or not isinstance(value.get("queries"), list):
+ return False
+ if not all(isinstance(query, dict) for query in value["queries"]):
+ return False
+ datasource = value.get("datasource")
+ if not isinstance(datasource, dict) or not datasource.get("id"):
+ return False
+ datasource_type = datasource.get("type")
+ if not isinstance(datasource_type, str):
+ return False
+ try:
+ DatasourceType(datasource_type)
+ except ValueError:
+ return False
+ return True
+
+
+def validate_query_context_metadata(value: Union[bytes, bytearray, str]) -> None:
+ """
+ Validator for a chart's ``query_context`` field: beyond being well-formed
+ JSON (see ``validate_json``), it must be a JSON object carrying a
+ non-empty ``datasource`` key and a ``queries`` key that is a list
+ (apache/superset#35774).
+
+ :raises ValidationError: if value is not valid JSON, or is valid JSON that
+ is not an object with a non-empty ``datasource`` key and a ``queries``
+ key holding a list
+ :param value: an object that should be parseable to a query_context JSON object
+ """
+ try:
+ parsed = json.loads(value)
+ except json.JSONDecodeError as ex:
+ raise ValidationError("JSON not valid") from ex
Review Comment:
`validate_json` let `""` through (falsy skip), so `query_context: ""` was accepted on POST/PUT and read back as absent. This re-parses it and now 400s with "JSON not valid", e.g. when re-posting a chart whose stored context is `""`.
```suggestion
if not value:
return
try:
parsed = json.loads(value)
except json.JSONDecodeError as ex:
raise ValidationError("JSON not valid") from ex
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(charts): reject query_context missing datasource/queries on save [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on code in PR #44549:
URL: https://github.com/apache/superset/pull/44549#discussion_r4269375297
##########
superset/utils/schema.py:
##########
@@ -56,6 +57,61 @@ def validate_json(value: Union[bytes, bytearray, str]) -> None:
raise ValidationError("JSON not valid") from ex
+def is_query_context_metadata_complete(value: Any) -> bool:
+ """
+ Whether a parsed ``query_context`` payload carries the ``datasource`` and
+ ``queries`` keys ``QueryContextFactory.create()`` requires (apache/superset
+ #35774): both are required keyword-only arguments there, and a chart saved
+ without them fails every subsequent read with a raw ``TypeError`` instead
+ of a clear error at save time. ``datasource`` must be a mapping carrying
+ the ``id``/``type`` keys ``DatasourceDict`` requires -- ``_convert_to_model``
+ indexes both directly and raises ``KeyError`` if either is absent, and
+ passes ``type`` straight to ``DatasourceType(...)``, which raises
+ ``ValueError`` for a value outside that enum -- but ``queries`` may
+ legitimately be an empty list (a chart with no query objects yet still
+ round-trips through ``create()`` without error), so only its presence as
+ a list is checked, not its truthiness.
+
+ :param value: the object obtained by JSON-decoding a query_context string
+ """
+ if not isinstance(value, dict) or not isinstance(value.get("queries"), list):
+ return False
Review Comment:
Good catch @aminghadersohi. `queries` entries must now be objects, so `["x"]` and `[null]` are rejected at save. Added both as test cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org