You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "hughhhh (via GitHub)" <gi...@apache.org> on 2026/07/21 20:03:31 UTC
[PR] feat(dashboard): handle empty chart query context in Excel export [superset]
hughhhh opened a new pull request, #42284:
URL: https://github.com/apache/superset/pull/42284
### SUMMARY
Follow-up to the async dashboard "Export Data to Excel" feature to properly handle charts that have no saved `query_context`. A chart persists its `query_context` only once it has been (re-)saved in Explore, so older charts have `params` (form data) but no context and previously failed mid-export or fell into the generic error bucket. This PR first makes such charts skip cleanly and get listed under the "no query context" remediation (covering `null`/`{}`/`{"queries": []}`/malformed values, not just a blank one), then goes further and **synthesizes a query context from the chart's saved form data** so those charts still export. Because the rebuild is a generic single-query mapping that does not reproduce plugin post-processing (pivot, rolling, forecast) or multi-query charts, it is gated behind a conservative, configurable viz-type allowlist (`EXCEL_EXPORT_REBUILD_VIZ_TYPES`, default `table`/`big_number_total`/`big_number`/`pie`); anything else without a saved context is still ski
pped and listed for re-save, so no chart ever exports silently wrong or incomplete data.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A โ backend export-task behavior only.
### TESTING INSTRUCTIONS
Run the unit tests: `pytest tests/unit_tests/tasks/test_export_dashboard_excel.py tests/unit_tests/common/test_form_data_query_context.py`. They cover the clean-skip cases (blank/`null`/`{}`/empty-queries/malformed), the form-data rebuild for an eligible viz type, and the skip-with-notice fallback for an ineligible (multi-query) viz type. End-to-end: export a dashboard containing a legacy chart with no saved `query_context` โ an eligible chart (e.g. a table) now appears as a data sheet, while an ineligible one is listed in the email as needing a re-save in Explore.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
๐ค Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "hughhhh (via GitHub)" <gi...@apache.org>.
hughhhh commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3657988777
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,227 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. Only ``SIMPLE`` WHERE-clause filters are
+ convertible here; free-form ``SQL`` filters have no ``{col, op, val}``
+ equivalent and are handled separately (see :func:`freeform_where_having`).
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if (
+ flt.get("expressionType") == "SIMPLE"
+ and (flt.get("clause") or "WHERE").upper() == "WHERE"
+ ):
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
Review Comment:
Confirmed intentional โ this fixes a real bug (an earlier reviewer flagged that a stale, present-but-empty `columns: []` was silently dropping `groupby` and changing the aggregation). Now that `preview_utils._build_query_columns` delegates to the shared `columns_from_form_data`, the MCP compile/preview path gets the same fix. Added an MCP-path test (`test_build_query_columns_empty_columns_key_keeps_groupby`) calling `preview_utils._build_query_columns({"groupby": ["country"], "columns": []})` and asserting `["country"]`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3691934958
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,290 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+
+The mirrored logic lives on the frontend in
+``superset-frontend/plugins/plugin-chart-table/src/buildQuery.ts`` (query mode,
+ordering), ``superset-frontend/packages/superset-ui-core/src/query/`` (field
+extraction, ``processFilters``). There is no automated tripwire tying the two
+across the language boundary; the per-helper pointers below must be kept in sync
+when that frontend logic changes.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+ where_only: bool = False,
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
+ val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+
+ By default all ``SIMPLE`` filters are converted (the behavior the MCP
+ compile/preview path relies on). Pass ``where_only=True`` to convert only
+ ``WHERE``-clause filters, matching the frontend's ``processFilters``
+ (``superset-ui-core/src/query/processFilters.ts``) โ the dashboard export uses
+ this so it applies the same rows the chart shows and does not additionally
+ filter on ``SIMPLE`` ``HAVING`` clauses.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") != "SIMPLE":
+ continue
+ if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
+ continue
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend
+ (``superset-ui-core/src/query/processFilters.ts``): ``SQL`` adhoc filters (and
+ a legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
Review Comment:
**Suggestion:** SQL expressions containing a trailing `--` comment are wrapped directly in parentheses without the newline sanitization used by the frontend. The comment can consume the generated closing parenthesis, producing invalid SQL or changing the predicate's structure during export. [logic error]
<details>
<summary><b>Severity Level:</b> Major โ ๏ธ</summary>
```mdx
- โ Eligible legacy chart export can fail during query execution.
- โ ๏ธ Charts with SQL comments produce invalid predicates.
- โ ๏ธ Frontend and rebuilt exports handle comments differently.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=06b6e371748d4dbb8205b6af262e4ecd&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=06b6e371748d4dbb8205b6af262e4ecd&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/common/form_data_query_context.py
**Line:** 102:106
**Comment:**
*Logic Error: SQL expressions containing a trailing `--` comment are wrapped directly in parentheses without the newline sanitization used by the frontend. The comment can consume the generated closing parenthesis, producing invalid SQL or changing the predicate's structure during export.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=dea73d9ee077e7dd4ded08a7a20abc5ab2ec630adfadaee452b551753e6db0c6&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=dea73d9ee077e7dd4ded08a7a20abc5ab2ec630adfadaee452b551753e6db0c6&reaction=dislike'>๐</a>
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,290 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+
+The mirrored logic lives on the frontend in
+``superset-frontend/plugins/plugin-chart-table/src/buildQuery.ts`` (query mode,
+ordering), ``superset-frontend/packages/superset-ui-core/src/query/`` (field
+extraction, ``processFilters``). There is no automated tripwire tying the two
+across the language boundary; the per-helper pointers below must be kept in sync
+when that frontend logic changes.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+ where_only: bool = False,
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
+ val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+
+ By default all ``SIMPLE`` filters are converted (the behavior the MCP
+ compile/preview path relies on). Pass ``where_only=True`` to convert only
+ ``WHERE``-clause filters, matching the frontend's ``processFilters``
+ (``superset-ui-core/src/query/processFilters.ts``) โ the dashboard export uses
+ this so it applies the same rows the chart shows and does not additionally
+ filter on ``SIMPLE`` ``HAVING`` clauses.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") != "SIMPLE":
+ continue
+ if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
+ continue
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
Review Comment:
**Suggestion:** Aggregate table form data can retain a non-empty `columns` value from an earlier raw-mode configuration. This branch prefers that stale list over `groupby`, unlike the frontend aggregate query builder, so rebuilt exports can aggregate by the wrong dimensions or omit the chart's actual grouping. Select `groupby` when the query is aggregate mode rather than using raw `columns` there. [incorrect condition logic]
<details>
<summary><b>Severity Level:</b> Major โ ๏ธ</summary>
```mdx
- โ Legacy aggregate table exports use wrong dimensions.
- โ ๏ธ Exported aggregates differ from Explore results.
- โ ๏ธ Row limits can select incorrect grouped rows.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=887c23ad63c84a09a2f317b3abf65d8e&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=887c23ad63c84a09a2f317b3abf65d8e&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/common/form_data_query_context.py
**Line:** 84:84
**Comment:**
*Incorrect Condition Logic: Aggregate table form data can retain a non-empty `columns` value from an earlier raw-mode configuration. This branch prefers that stale list over `groupby`, unlike the frontend aggregate query builder, so rebuilt exports can aggregate by the wrong dimensions or omit the chart's actual grouping. Select `groupby` when the query is aggregate mode rather than using raw `columns` there.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=82beb693d56cddb66ab1de6f025af3612d43103bc8e6637c62dddb5ec8b3a3c8&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=82beb693d56cddb66ab1de6f025af3612d43103bc8e6637c62dddb5ec8b3a3c8&reaction=dislike'>๐</a>
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -39,26 +39,14 @@
def _build_query_columns(form_data: Dict[str, Any]) -> list[str]:
- """Build query columns list from form_data, including both x_axis and groupby."""
- # Table charts in raw mode use all_columns or columns
- all_columns = form_data.get("all_columns", [])
- raw_columns_field = form_data.get("columns", [])
- if form_data.get("query_mode") == "raw" and (all_columns or raw_columns_field):
- return list(all_columns or raw_columns_field)
-
- x_axis_config = form_data.get("x_axis")
- groupby_columns: list[str] = form_data.get("groupby") or []
- raw_columns: list[str] = form_data.get("columns") or []
-
- columns = raw_columns.copy() if "columns" in form_data else groupby_columns.copy()
- if x_axis_config and isinstance(x_axis_config, str):
- if x_axis_config not in columns:
- columns.insert(0, x_axis_config)
- elif x_axis_config and isinstance(x_axis_config, dict):
- col_name = x_axis_config.get("column_name")
- if col_name and col_name not in columns:
- columns.insert(0, col_name)
- return columns
+ """Build query columns list from form_data, including both x_axis and groupby.
+
+ Delegates to the shared builder so the MCP and dashboard-export paths stay in
+ sync (single source of truth).
+ """
+ from superset.common.form_data_query_context import columns_from_form_data
+
+ return columns_from_form_data(form_data)
Review Comment:
**Suggestion:** The shared helper now replaces the preview path's chart-aware extraction but only reads `groupby`, `columns`, and `x_axis`. The MCP preview compiler supports pivot dimensions in `groupbyRows`/`groupbyColumns` and secondary dimensions in `groupby_b`; those charts now lose required columns and generate previews from an incomplete query. Preserve the chart-specific dimensions or extend the shared helper to handle these shapes. [incomplete implementation]
<details>
<summary><b>Severity Level:</b> Major โ ๏ธ</summary>
```mdx
- โ MCP pivot previews omit required dimensions.
- โ Mixed-series previews omit secondary grouping.
- โ ๏ธ Preview data no longer matches requested chart configuration.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=82da61d9126a40269c310399fce474b2&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=82da61d9126a40269c310399fce474b2&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/mcp_service/chart/preview_utils.py
**Line:** 47:49
**Comment:**
*Incomplete Implementation: The shared helper now replaces the preview path's chart-aware extraction but only reads `groupby`, `columns`, and `x_axis`. The MCP preview compiler supports pivot dimensions in `groupbyRows`/`groupbyColumns` and secondary dimensions in `groupby_b`; those charts now lose required columns and generate previews from an incomplete query. Preserve the chart-specific dimensions or extend the shared helper to handle these shapes.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=e1063ff91c147cd6794077f9cdbb9ab6b98e79c8111f0abe282e960d3edd0c53&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=e1063ff91c147cd6794077f9cdbb9ab6b98e79c8111f0abe282e960d3edd0c53&reaction=dislike'>๐</a>
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,290 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+
+The mirrored logic lives on the frontend in
+``superset-frontend/plugins/plugin-chart-table/src/buildQuery.ts`` (query mode,
+ordering), ``superset-frontend/packages/superset-ui-core/src/query/`` (field
+extraction, ``processFilters``). There is no automated tripwire tying the two
+across the language boundary; the per-helper pointers below must be kept in sync
+when that frontend logic changes.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+ where_only: bool = False,
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
+ val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+
+ By default all ``SIMPLE`` filters are converted (the behavior the MCP
+ compile/preview path relies on). Pass ``where_only=True`` to convert only
+ ``WHERE``-clause filters, matching the frontend's ``processFilters``
+ (``superset-ui-core/src/query/processFilters.ts``) โ the dashboard export uses
+ this so it applies the same rows the chart shows and does not additionally
+ filter on ``SIMPLE`` ``HAVING`` clauses.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") != "SIMPLE":
+ continue
+ if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
+ continue
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend
+ (``superset-ui-core/src/query/processFilters.ts``): ``SQL`` adhoc filters (and
+ a legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def is_raw_query_mode(form_data: dict[str, Any]) -> bool:
+ """
+ Whether the chart runs in raw (non-aggregated) mode, mirroring the frontend's
+ ``getQueryMode`` (``plugin-chart-table/src/buildQuery.ts``): an explicit
+ ``query_mode`` wins, otherwise the presence of ``all_columns`` implies raw mode.
+ """
+ if mode := form_data.get("query_mode"):
+ return mode == "raw"
+ return bool(form_data.get("all_columns"))
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any], viz_type: str | None = None
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+
+ ``order_by_cols`` is a raw-mode-only control (``resetOnHide: false`` in the
+ plugin control panels), so an aggregate chart can carry a stale value from a
+ previous raw-mode configuration. Aggregate mode must ignore it, mirroring the
+ frontend, where ``plugin-chart-table/src/buildQuery.ts:136-145`` overrides
+ ``orderby`` with the sort metric (``order_by_cols`` reaches ``orderby`` only
+ via the alias in ``extractQueryFields.ts``, then gets overwritten in aggregate
+ mode).
+ """
+ if is_raw_query_mode(form_data):
+ parsed: list[list[Any]] = []
+ for col in form_data.get("order_by_cols") or []:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
+ except (TypeError, ValueError):
+ continue
+ parsed.append(col)
+ return parsed
+
+ if not metrics:
+ return []
+
+ sort_metric = form_data.get("timeseries_limit_metric") or (
+ metrics[0] if form_data.get("sort_by_metric") else None
+ )
+ if sort_metric is not None:
Review Comment:
**Suggestion:** `timeseries_limit_metric` can be either a metric or a one-element list, but the frontend normalizes it with `ensureIsArray(...)[0]`. Passing the raw list through creates an invalid nested `orderby` metric shape for saved charts using the list representation, which can fail query validation or produce incorrect ordering. [type error]
<details>
<summary><b>Severity Level:</b> Major โ ๏ธ</summary>
```mdx
- โ Legacy table export can fail query validation.
- โ ๏ธ Row-limited exports can use invalid ordering.
- โ ๏ธ Exported top-N rows may differ from Explore.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=fa6828a690434bc2a1b93e679c9b6f71&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=fa6828a690434bc2a1b93e679c9b6f71&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/common/form_data_query_context.py
**Line:** 186:189
**Comment:**
*Type Error: `timeseries_limit_metric` can be either a metric or a one-element list, but the frontend normalizes it with `ensureIsArray(...)[0]`. Passing the raw list through creates an invalid nested `orderby` metric shape for saved charts using the list representation, which can fail query validation or produce incorrect ordering.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=ac5edf1db093d12b8f67658af9a0d1abf3fa23a27a1c57794197276e502f39c2&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=ac5edf1db093d12b8f67658af9a0d1abf3fa23a27a1c57794197276e502f39c2&reaction=dislike'>๐</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5121837702
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5186594726
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5245198944
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "EnxDev (via GitHub)" <gi...@apache.org>.
EnxDev commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3658068446
##########
superset/tasks/export_dashboard_excel.py:
##########
@@ -67,6 +68,12 @@
# image. Operators can override the set via ``EXCEL_EXPORT_TABLE_VIZ_TYPES``.
TABLE_VIZ_TYPES = {"table", "pivot_table_v2", "pivot_table"}
+# Viz types whose missing query context may be rebuilt from saved form data.
+# Conservative by default: only charts whose data maps faithfully to a single
+# plain query (no post-processing, no multi-query fan-out). Operators can
+# override via ``EXCEL_EXPORT_REBUILD_VIZ_TYPES``.
+REBUILD_VIZ_TYPES = {"table", "big_number_total", "big_number", "pie"}
Review Comment:
`REBUILD_VIZ_TYPES` only checks the viz type, but some behaviors that require post-processing or multiple queries depend on the form data.
For example, a table with `time_compare` requires additional queries/post-processing, and a Big Number with rolling/resampling can also produce values that differ from the raw query result. Rebuilding these directly could therefore export incomplete or incorrect data.
Should we also skip the rebuild when `rolling_type`, `resample_rule`, `time_compare`, or `aggregation === "raw"` is present?
It would be good to add a regression test for at least the `time_compare` case.
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,231 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. All ``SIMPLE`` filters are converted (matching the
+ behavior the MCP compile/preview path relied on); free-form ``SQL`` filters
+ have no ``{col, op, val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") == "SIMPLE":
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any]
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+ """
+ if order_by_cols := form_data.get("order_by_cols") or []:
+ parsed: list[list[Any]] = []
+ for col in order_by_cols:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
+ except (TypeError, ValueError):
+ continue
+ parsed.append(col)
+ return parsed
+
+ if not metrics:
+ return []
+
+ order_desc = form_data.get("order_desc", True)
+ sort_metric = form_data.get("timeseries_limit_metric") or (
+ metrics[0] if form_data.get("sort_by_metric") else None
+ )
+ if sort_metric is not None:
+ return [[sort_metric, not order_desc]]
+ # No explicit sort metric: default to the first metric, descending.
+ return [[metrics[0], False]]
+
+
+def build_query_context_from_form_data(
+ form_data: dict[str, Any],
+ datasource: dict[str, Any],
+ viz_type: str | None = None,
+) -> dict[str, Any]:
+ """
+ Build a query-context payload (the JSON shape ``ChartDataQueryContextSchema``
+ loads) from a chart's form data and datasource reference.
+
+ :param form_data: The chart's saved ``params`` parsed to a dict.
+ :param datasource: ``{"id": <int>, "type": "table"}`` datasource reference.
+ :param viz_type: The chart's viz type, used for viz-specific handling.
+ :returns: A single-query query-context dict.
+ """
+ metrics = list(form_data.get("metrics") or [])
+ # Single-metric charts (e.g. Big Number) store ``metric`` rather than
+ # ``metrics``.
+ if not metrics and form_data.get("metric"):
+ metrics = [form_data["metric"]]
+ # ``percent_metrics`` are intentionally not carried: the chart shows them as a
+ # "% of total" produced by contribution post-processing, which this rebuild
+ # does not apply, so adding them as plain metrics would export raw aggregates
+ # that don't match the chart.
+
+ columns = columns_from_form_data(form_data)
+ # Only a Big Number *with a trendline* (viz_type ``big_number``) groups by its
+ # time column; ``big_number_total`` is a single aggregate and must not be
+ # grouped, or it would return one row per timestamp instead of a total.
+ if not columns and viz_type == "big_number" and form_data.get("granularity_sqla"):
+ columns = [form_data["granularity_sqla"]]
+
+ # SIMPLE adhoc filters (+ legacy top-level ``filters``) become query filters;
+ # free-form SQL predicates go into ``extras``.
+ filters = adhoc_filters_to_query_filters(form_data.get("adhoc_filters", []))
+ for flt in form_data.get("filters") or []:
+ if isinstance(flt, dict) and flt.get("col") is not None:
+ filters.append(flt)
+
+ extras = freeform_where_having(form_data)
+ if form_data.get("time_grain_sqla"):
+ extras["time_grain_sqla"] = form_data["time_grain_sqla"]
+
+ # Prefer the modern ``time_range``; fall back to the legacy ``since``/``until``
+ # pair (older charts store the range that way) before defaulting to no filter.
+ time_range = form_data.get("time_range")
+ if not time_range and (form_data.get("since") or form_data.get("until")):
+ time_range = f"{form_data.get('since') or ''} : {form_data.get('until') or ''}"
+ time_range = time_range or "No filter"
+ query: dict[str, Any] = {
+ "columns": columns,
+ "metrics": metrics,
+ "orderby": orderby_from_form_data(form_data, metrics),
+ "filters": filters,
+ "time_range": time_range,
+ }
+ if extras:
+ query["extras"] = extras
+ # ``granularity`` names the temporal column that actually applies the time
+ # range; without it, ``time_range`` is inert and the export returns the entire
+ # history. Only set it when there is a real range to apply โ otherwise a
+ # numeric column saved as ``granularity_sqla`` (with no active range) would be
+ # forced through date bucketing and fail.
+ granularity = form_data.get("granularity") or form_data.get("granularity_sqla")
+ if granularity and time_range != "No filter":
+ query["granularity"] = granularity
+ if form_data.get("row_limit"):
+ query["row_limit"] = form_data["row_limit"]
Review Comment:
For Big Number trendlines, `granularity` is only set when `time_range != "No filter"`. However, `time_grain_sqla` is only applied to the column identified by `granularity`.
This means a Big Number with no time filter can export rows at the raw timestamp precision instead of the configured time grain.
Should we always set `granularity` to the promoted time column for Big Number trendlines, regardless of `time_range`?
It would also be good to add a regression test for a Big Number with `time_grain_sqla` and no `time_range`.
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,231 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. All ``SIMPLE`` filters are converted (matching the
+ behavior the MCP compile/preview path relied on); free-form ``SQL`` filters
+ have no ``{col, op, val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") == "SIMPLE":
Review Comment:
Restoring support for all `SIMPLE` filters makes sense for MCP, but it creates a difference between the chart and the export. The chart only applies `SIMPLE` filters with `clause === "WHERE"`, while the export would also apply `HAVING` filters, potentially resulting in fewer rows than what the user sees.
Should we filter out `HAVING` filters in `build_query_context_from_form_data` while keeping the shared helper unchanged for MCP?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5146764122
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "EnxDev (via GitHub)" <gi...@apache.org>.
EnxDev commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3675772750
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,273 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+ where_only: bool = False,
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
+ val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+
+ By default all ``SIMPLE`` filters are converted (the behavior the MCP
+ compile/preview path relies on). Pass ``where_only=True`` to convert only
+ ``WHERE``-clause filters, matching the frontend's ``processFilters`` โ the
+ dashboard export uses this so it applies the same rows the chart shows and
+ does not additionally filter on ``SIMPLE`` ``HAVING`` clauses.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") != "SIMPLE":
+ continue
+ if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
+ continue
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def is_raw_query_mode(form_data: dict[str, Any]) -> bool:
+ """
+ Whether the chart runs in raw (non-aggregated) mode, mirroring the frontend's
+ ``getQueryMode``: an explicit ``query_mode`` wins, otherwise the presence of
+ ``all_columns`` implies raw mode.
+ """
+ if mode := form_data.get("query_mode"):
+ return mode == "raw"
+ return bool(form_data.get("all_columns"))
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any], viz_type: str | None = None
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+ """
+ if order_by_cols := form_data.get("order_by_cols") or []:
+ parsed: list[list[Any]] = []
+ for col in order_by_cols:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
Review Comment:
`orderby_from_form_data` checks `order_by_cols` in every mode, but this is a raw-mode-only control.
Since its value isn't reset when switching to aggregate mode, a stale `order_by_cols` can cause the export to use a different ordering from the chart, potentially returning a different top-N.
Could we only use `order_by_cols` when `is_raw_query_mode(form_data)` is true? It would also be good to add a regression test for an aggregate table with a stale `order_by_cols` value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "netlify[bot] (via GitHub)" <gi...@apache.org>.
netlify[bot] commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5038523771
### <span aria-hidden="true">โ
</span> Deploy Preview for *superset-docs-preview* ready!
| Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">๐จ</span> Latest commit | de29e984ffa5a0dd21b6205e13bf53821de6e2ee |
|<span aria-hidden="true">๐</span> Latest deploy log | https://app.netlify.com/projects/superset-docs-preview/deploys/6a5fd094110f1d0008b0ca75 |
|<span aria-hidden="true">๐</span> Deploy Preview | [https://deploy-preview-42284--superset-docs-preview.netlify.app](https://deploy-preview-42284--superset-docs-preview.netlify.app) |
|<span aria-hidden="true">๐ฑ</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br /><br /><br />_Use your smartphone camera to open QR code link._</details> |
|<span aria-hidden="true">๐ค</span> Make changes | [Run an agent on this branch](https://app.netlify.com/projects/superset-docs-preview/agent-runs#DcpLDoAgDADRu3QtsST-b-AxgFZZKCDBRGK8u53VLN4LKcczFVgAGrDZBOfl_b17qfUm0MGKRVR13ZyrcjEUfopo4nTEupL4wfQb4dxpjZsmRJwsOjP28P0) |
---
<!-- [superset-docs-preview Preview](https://deploy-preview-42284--superset-docs-preview.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify project configuration](https://app.netlify.com/projects/superset-docs-preview/configuration/notifications#deploy-notifications)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "hughhhh (via GitHub)" <gi...@apache.org>.
hughhhh commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3632236484
##########
superset/tasks/export_dashboard_excel.py:
##########
@@ -96,6 +103,59 @@ def _chart_label(chart: Any) -> str:
return f"{chart.id} - {chart.slice_name or ''}".strip()
+def _has_empty_query_context(raw: Any) -> bool:
+ """
+ Whether a chart has no usable saved query context to export data from.
+
+ Covers a missing/blank value, a string that does not parse as JSON, a value
+ that parses to something other than an object (e.g. ``"null"``), and an
+ object with no queries (e.g. ``"{}"`` or ``{"queries": []}``). All of these
+ are treated the same as a missing context: the chart is skipped and listed
+ under the "no query context" remediation rather than raising mid-export.
+ """
+ if not raw:
+ return True
+ try:
+ parsed = json.loads(raw)
+ except (TypeError, ValueError):
+ return True
+ return not isinstance(parsed, dict) or not parsed.get("queries")
+
+
+def _rebuild_viz_types() -> set[str]:
+ """Viz types eligible for form-data query-context rebuild (config or default)."""
+ return current_app.config.get("EXCEL_EXPORT_REBUILD_VIZ_TYPES") or REBUILD_VIZ_TYPES
Review Comment:
Fixed in de7dfc1585: `_rebuild_viz_types` now checks `is None` explicitly instead of `or`, so an operator can disable the rebuild with an explicit empty set instead of it silently falling back to the default allowlist. Added a parametrized unit test (None โ default, empty set โ disabled, override โ honored).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5247176313
Thanks for this โ tested it live end-to-end (real Celery worker, MailHog, MinIO-backed S3) rather than just reading the diff, and found one correctness gap worth fixing before merge.
**The `granularity` gate in `build_query_context_from_form_data` (`superset/common/form_data_query_context.py`) still drops time-bucketing for `table`/`pie` charts grouped by their own time column when `time_range` is `"No filter"`.**
Repro (verified against a real dataset with per-day granularity, not one that happens to already be pre-aggregated to the grain being tested):
- Chart: `table`, `groupby: ["order_date"]`, `granularity_sqla: "order_date"`, `time_grain_sqla: "P1Y"`, `time_range: "No filter"`, no saved `query_context`.
- Exported sheet: **252 rows**, each a raw, un-truncated `order_date` timestamp (e.g. `2003-11-14`) โ i.e. completely unaggregated data.
- Same chart, only difference `time_range: "2003-01-01 : 2006-01-01"`: **3 rows**, correctly bucketed by year with proper aggregated counts.
Root cause: `granularity` is only set when `time_range != "No filter"` (or for the `big_number` trendline's promoted time column). But `granularity` and `time_range` are independent concerns in the real query object โ I checked `extractExtras.ts` in `superset-ui-core`, and the frontend sets `granularity` from `granularity_sqla`/`granularity` unconditionally, with zero dependency on `time_range`. The backend's groupby-column loop (`superset/models/helpers.py`) only applies time-grain truncation to a selected column when it matches `granularity` โ so omitting it here means the column is selected raw instead of bucketed, for the fairly ordinary "all-time totals by month/year" chart configuration.
The `big_number` trendline fix (`promoted_time_column`) is correct and I confirmed it works โ it's specifically the general `table`/`pie` "explicit `groupby` includes the time column" case that's still affected.
I've added a regression test that reproduces this (currently failing against HEAD): `test_table_groupby_time_column_without_time_range_is_bucketed` in `tests/unit_tests/common/test_form_data_query_context.py`. Suggested fix: set `granularity` whenever `granularity_sqla`/`granularity` is present, independent of `time_range` (matching real frontend behavior), rather than gating on `time_range != "No filter"`.
Everything else I exercised โ happy-path rebuilds for all four allowlisted viz types, the `_needs_unsupported_processing` guard, malformed-`query_context` recovery, the in-flight export lock, and the full emailโS3โdownload round trip via a real Celery worker โ worked correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "hughhhh (via GitHub)" <gi...@apache.org>.
hughhhh commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3675636626
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,247 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+ where_only: bool = False,
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
+ val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+
+ By default all ``SIMPLE`` filters are converted (the behavior the MCP
+ compile/preview path relies on). Pass ``where_only=True`` to convert only
+ ``WHERE``-clause filters, matching the frontend's ``processFilters`` โ the
+ dashboard export uses this so it applies the same rows the chart shows and
+ does not additionally filter on ``SIMPLE`` ``HAVING`` clauses.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") != "SIMPLE":
+ continue
+ if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
+ continue
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any]
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+ """
+ if order_by_cols := form_data.get("order_by_cols") or []:
+ parsed: list[list[Any]] = []
+ for col in order_by_cols:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
+ except (TypeError, ValueError):
+ continue
+ parsed.append(col)
+ return parsed
+
+ if not metrics:
+ return []
+
+ order_desc = form_data.get("order_desc", True)
+ sort_metric = form_data.get("timeseries_limit_metric") or (
+ metrics[0] if form_data.get("sort_by_metric") else None
+ )
+ if sort_metric is not None:
+ return [[sort_metric, not order_desc]]
+ # No explicit sort metric: default to the first metric, descending.
+ return [[metrics[0], False]]
+
+
+def build_query_context_from_form_data(
+ form_data: dict[str, Any],
+ datasource: dict[str, Any],
+ viz_type: str | None = None,
+) -> dict[str, Any]:
+ """
+ Build a query-context payload (the JSON shape ``ChartDataQueryContextSchema``
+ loads) from a chart's form data and datasource reference.
+
+ :param form_data: The chart's saved ``params`` parsed to a dict.
+ :param datasource: ``{"id": <int>, "type": "table"}`` datasource reference.
+ :param viz_type: The chart's viz type, used for viz-specific handling.
+ :returns: A single-query query-context dict.
+ """
+ metrics = list(form_data.get("metrics") or [])
Review Comment:
Fixed in 25658110ba. The rebuild now mirrors `getQueryMode` (a new `is_raw_query_mode`: explicit `query_mode`, else `all_columns` presence) and in raw mode uses only the selected columns while ignoring the stale `metrics`/`groupby` โ so no accidental aggregation, grouping, or metric-based `orderby`. Added regression tests, including one for the `Publishers With Most Titles`-style raw table (all_columns + stale metrics/groupby โ columns only, metrics=[], orderby=[]).
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,247 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+ where_only: bool = False,
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
+ val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+
+ By default all ``SIMPLE`` filters are converted (the behavior the MCP
+ compile/preview path relies on). Pass ``where_only=True`` to convert only
+ ``WHERE``-clause filters, matching the frontend's ``processFilters`` โ the
+ dashboard export uses this so it applies the same rows the chart shows and
+ does not additionally filter on ``SIMPLE`` ``HAVING`` clauses.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") != "SIMPLE":
+ continue
+ if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
+ continue
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any]
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+ """
+ if order_by_cols := form_data.get("order_by_cols") or []:
+ parsed: list[list[Any]] = []
+ for col in order_by_cols:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
+ except (TypeError, ValueError):
+ continue
+ parsed.append(col)
+ return parsed
+
+ if not metrics:
+ return []
+
+ order_desc = form_data.get("order_desc", True)
Review Comment:
Good catch. Made it viz-specific in 25658110ba: `orderby_from_form_data` now defaults `order_desc` to False (ascending) for Table and True (descending) for Pie/others, matching the plugins โ so a table with `timeseries_limit_metric` and no saved `order_desc` sorts ascending in the export too, keeping bottom-N under a row limit. Added a regression test (Table + timeseries_limit_metric, no order_desc โ `[[metric, True]]`).
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,247 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+ where_only: bool = False,
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
+ val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+
+ By default all ``SIMPLE`` filters are converted (the behavior the MCP
+ compile/preview path relies on). Pass ``where_only=True`` to convert only
+ ``WHERE``-clause filters, matching the frontend's ``processFilters`` โ the
+ dashboard export uses this so it applies the same rows the chart shows and
+ does not additionally filter on ``SIMPLE`` ``HAVING`` clauses.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") != "SIMPLE":
+ continue
+ if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
+ continue
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any]
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+ """
+ if order_by_cols := form_data.get("order_by_cols") or []:
+ parsed: list[list[Any]] = []
+ for col in order_by_cols:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
+ except (TypeError, ValueError):
+ continue
+ parsed.append(col)
+ return parsed
+
+ if not metrics:
+ return []
+
+ order_desc = form_data.get("order_desc", True)
+ sort_metric = form_data.get("timeseries_limit_metric") or (
+ metrics[0] if form_data.get("sort_by_metric") else None
+ )
+ if sort_metric is not None:
+ return [[sort_metric, not order_desc]]
+ # No explicit sort metric: default to the first metric, descending.
+ return [[metrics[0], False]]
+
+
+def build_query_context_from_form_data(
+ form_data: dict[str, Any],
+ datasource: dict[str, Any],
+ viz_type: str | None = None,
+) -> dict[str, Any]:
+ """
+ Build a query-context payload (the JSON shape ``ChartDataQueryContextSchema``
+ loads) from a chart's form data and datasource reference.
+
+ :param form_data: The chart's saved ``params`` parsed to a dict.
+ :param datasource: ``{"id": <int>, "type": "table"}`` datasource reference.
+ :param viz_type: The chart's viz type, used for viz-specific handling.
+ :returns: A single-query query-context dict.
+ """
+ metrics = list(form_data.get("metrics") or [])
+ # Single-metric charts (e.g. Big Number) store ``metric`` rather than
+ # ``metrics``.
+ if not metrics and form_data.get("metric"):
+ metrics = [form_data["metric"]]
+ # ``percent_metrics`` are intentionally not carried: the chart shows them as a
+ # "% of total" produced by contribution post-processing, which this rebuild
+ # does not apply, so adding them as plain metrics would export raw aggregates
+ # that don't match the chart.
Review Comment:
Agreed โ silently omitting visible columns is worse than skipping. In 25658110ba a non-empty `percent_metrics` is now treated like the other `_UNSUPPORTED_PROCESSING_KEYS`: the chart is skipped and listed for re-save rather than rebuilt. Added it to the parametrized skip test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5109054818
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "EnxDev (via GitHub)" <gi...@apache.org>.
EnxDev commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3672362338
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,247 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+ where_only: bool = False,
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
+ val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+
+ By default all ``SIMPLE`` filters are converted (the behavior the MCP
+ compile/preview path relies on). Pass ``where_only=True`` to convert only
+ ``WHERE``-clause filters, matching the frontend's ``processFilters`` โ the
+ dashboard export uses this so it applies the same rows the chart shows and
+ does not additionally filter on ``SIMPLE`` ``HAVING`` clauses.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") != "SIMPLE":
+ continue
+ if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
+ continue
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any]
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+ """
+ if order_by_cols := form_data.get("order_by_cols") or []:
+ parsed: list[list[Any]] = []
+ for col in order_by_cols:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
+ except (TypeError, ValueError):
+ continue
+ parsed.append(col)
+ return parsed
+
+ if not metrics:
+ return []
+
+ order_desc = form_data.get("order_desc", True)
Review Comment:
`order_desc` defaults to `True` here, while the table plugin defaults it to `false`.
This means a table with `timeseries_limit_metric` and no saved `order_desc` can sort ascending in the chart but descending in the export; potentially changing bottom-N into top-N when a row limit is applied.
We can't change the default globally because Pie always sorts by metric descending.
Could we make this viz-specific: default to ascending for Table and descending for Pie?
A regression test for a Table with `timeseries_limit_metric` and no `order_desc` would also be useful.
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,247 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+ where_only: bool = False,
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
+ val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+
+ By default all ``SIMPLE`` filters are converted (the behavior the MCP
+ compile/preview path relies on). Pass ``where_only=True`` to convert only
+ ``WHERE``-clause filters, matching the frontend's ``processFilters`` โ the
+ dashboard export uses this so it applies the same rows the chart shows and
+ does not additionally filter on ``SIMPLE`` ``HAVING`` clauses.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") != "SIMPLE":
+ continue
+ if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
+ continue
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any]
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+ """
+ if order_by_cols := form_data.get("order_by_cols") or []:
+ parsed: list[list[Any]] = []
+ for col in order_by_cols:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
+ except (TypeError, ValueError):
+ continue
+ parsed.append(col)
+ return parsed
+
+ if not metrics:
+ return []
+
+ order_desc = form_data.get("order_desc", True)
+ sort_metric = form_data.get("timeseries_limit_metric") or (
+ metrics[0] if form_data.get("sort_by_metric") else None
+ )
+ if sort_metric is not None:
+ return [[sort_metric, not order_desc]]
+ # No explicit sort metric: default to the first metric, descending.
+ return [[metrics[0], False]]
+
+
+def build_query_context_from_form_data(
+ form_data: dict[str, Any],
+ datasource: dict[str, Any],
+ viz_type: str | None = None,
+) -> dict[str, Any]:
+ """
+ Build a query-context payload (the JSON shape ``ChartDataQueryContextSchema``
+ loads) from a chart's form data and datasource reference.
+
+ :param form_data: The chart's saved ``params`` parsed to a dict.
+ :param datasource: ``{"id": <int>, "type": "table"}`` datasource reference.
+ :param viz_type: The chart's viz type, used for viz-specific handling.
+ :returns: A single-query query-context dict.
+ """
+ metrics = list(form_data.get("metrics") or [])
Review Comment:
In raw-mode table charts, saved form data can still contain stale `metrics`/`groupby` values because those controls aren't reset when hidden. The frontend ignores them in raw mode, but the rebuild currently doesn't, so the exported query can differ from what the chart actually runs.
For example, this can add aggregate columns/grouping and even introduce an `orderby` based on a stale metric, potentially exporting different rows from those shown in the chart.
Could we mirror the frontend behavior here by dropping `metrics`/`groupby` in raw mode and handling `all_columns` consistently with `getQueryMode`? A regression test using `Publishers_With_Most_Titles` would cover this well.
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,247 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+ where_only: bool = False,
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
+ val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+
+ By default all ``SIMPLE`` filters are converted (the behavior the MCP
+ compile/preview path relies on). Pass ``where_only=True`` to convert only
+ ``WHERE``-clause filters, matching the frontend's ``processFilters`` โ the
+ dashboard export uses this so it applies the same rows the chart shows and
+ does not additionally filter on ``SIMPLE`` ``HAVING`` clauses.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") != "SIMPLE":
+ continue
+ if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
+ continue
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any]
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+ """
+ if order_by_cols := form_data.get("order_by_cols") or []:
+ parsed: list[list[Any]] = []
+ for col in order_by_cols:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
+ except (TypeError, ValueError):
+ continue
+ parsed.append(col)
+ return parsed
+
+ if not metrics:
+ return []
+
+ order_desc = form_data.get("order_desc", True)
+ sort_metric = form_data.get("timeseries_limit_metric") or (
+ metrics[0] if form_data.get("sort_by_metric") else None
+ )
+ if sort_metric is not None:
+ return [[sort_metric, not order_desc]]
+ # No explicit sort metric: default to the first metric, descending.
+ return [[metrics[0], False]]
+
+
+def build_query_context_from_form_data(
+ form_data: dict[str, Any],
+ datasource: dict[str, Any],
+ viz_type: str | None = None,
+) -> dict[str, Any]:
+ """
+ Build a query-context payload (the JSON shape ``ChartDataQueryContextSchema``
+ loads) from a chart's form data and datasource reference.
+
+ :param form_data: The chart's saved ``params`` parsed to a dict.
+ :param datasource: ``{"id": <int>, "type": "table"}`` datasource reference.
+ :param viz_type: The chart's viz type, used for viz-specific handling.
+ :returns: A single-query query-context dict.
+ """
+ metrics = list(form_data.get("metrics") or [])
+ # Single-metric charts (e.g. Big Number) store ``metric`` rather than
+ # ``metrics``.
+ if not metrics and form_data.get("metric"):
+ metrics = [form_data["metric"]]
+ # ``percent_metrics`` are intentionally not carried: the chart shows them as a
+ # "% of total" produced by contribution post-processing, which this rebuild
+ # does not apply, so adding them as plain metrics would export raw aggregates
+ # that don't match the chart.
Review Comment:
Dropping `percent_metrics` means the export is missing percentage columns that are visible in the chart.
This affects both Table and Pie charts, where contribution post-processing adds these values.
Since the goal is to avoid silently exporting incomplete data, should we treat a non-empty `percent_metrics` like the other `_UNSUPPORTED_PROCESSING_KEYS` and require the chart to be re-saved?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3658006907
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,231 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. All ``SIMPLE`` filters are converted (matching the
+ behavior the MCP compile/preview path relied on); free-form ``SQL`` filters
+ have no ``{col, op, val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") == "SIMPLE":
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
Review Comment:
**Suggestion:** A SIMPLE adhoc filter marked with clause HAVING is always appended to `filters`, which applies it before aggregation as a WHERE predicate. Aggregate conditions such as `COUNT(*) > 5` will therefore produce incorrect results or fail in the database. Route HAVING filters to the query's having expression instead of treating every SIMPLE filter as a row-level filter. [logic error]
<details>
<summary><b>Severity Level:</b> Major โ ๏ธ</summary>
```mdx
- โ Aggregate conditions are applied before chart aggregation.
- โ Legacy chart Excel sheets can contain incorrect totals.
- โ ๏ธ Databases may reject aggregate predicates in WHERE.
```
</details>
<details>
<summary><b>Steps of Reproduction โ
</b></summary>
```mdx
1. Create or use a legacy chart whose saved `params` contain a `SIMPLE` adhoc filter with
`clause: "HAVING"` and an aggregate condition, such as a metric count greater than five.
2. Export a dashboard containing that chart through the Excel export task in
`superset/tasks/export_dashboard_excel.py`; `_build_workbook()` at lines 272-291 resolves
the missing context and invokes `_write_chart_sheets()`.
3. `_resolve_query_context()` at lines 148-164 calls
`build_query_context_from_form_data()`, which calls `adhoc_filters_to_query_filters()` at
line 192.
4. `adhoc_filters_to_query_filters()` at lines 55-61 converts the filter into the query's
`filters` list without inspecting its HAVING clause, while `freeform_where_having()` at
lines 79-88 only handles SQL filters. The generated query therefore applies the aggregate
predicate as a row-level WHERE filter, producing incorrect results or a database
validation error.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=23aa6f0a48b944c0aa44664325e8e714&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=23aa6f0a48b944c0aa44664325e8e714&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/common/form_data_query_context.py
**Line:** 55:61
**Comment:**
*Logic Error: A SIMPLE adhoc filter marked with clause HAVING is always appended to `filters`, which applies it before aggregation as a WHERE predicate. Aggregate conditions such as `COUNT(*) > 5` will therefore produce incorrect results or fail in the database. Route HAVING filters to the query's having expression instead of treating every SIMPLE filter as a row-level filter.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=a36c8e450affb6f05dbc803ff43b76f72d1673709f7a562547317ab62ec0fdc5&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=a36c8e450affb6f05dbc803ff43b76f72d1673709f7a562547317ab62ec0fdc5&reaction=dislike'>๐</a>
##########
superset/tasks/export_dashboard_excel.py:
##########
@@ -96,6 +103,67 @@ def _chart_label(chart: Any) -> str:
return f"{chart.id} - {chart.slice_name or ''}".strip()
+def _saved_query_context(raw: Any) -> dict[str, Any] | None:
+ """
+ The chart's saved query context parsed to a dict, or ``None`` when it is
+ missing or unusable.
+
+ Returns ``None`` for a blank value, a string that does not parse as JSON, a
+ value that parses to something other than an object (e.g. ``"null"``), and an
+ object with no queries (e.g. ``"{}"`` or ``{"queries": []}``) โ all treated
+ the same as a missing context.
+ """
+ if not raw:
+ return None
+ try:
+ parsed = json.loads(raw)
+ except (TypeError, ValueError):
+ return None
+ if not isinstance(parsed, dict) or not parsed.get("queries"):
+ return None
+ return parsed
Review Comment:
**Suggestion:** `_saved_query_context` only checks that `queries` is truthy, so malformed contexts such as `{"queries": "invalid"}` or other non-list values are accepted and passed to `ChartDataCommand`. This causes validation or iteration failures to be reported as a generic chart export error instead of being classified as missing query context and listed for remediation. Require `queries` to be a non-empty list with the expected query-object structure before returning the parsed context. [possible bug]
<details>
<summary><b>Severity Level:</b> Major โ ๏ธ</summary>
```mdx
- โ Affected chart data sheets are omitted from exports.
- โ ๏ธ Users receive generic errors instead of remediation guidance.
- โ ๏ธ Malformed saved contexts bypass missing-context handling.
```
</details>
<details>
<summary><b>Steps of Reproduction โ
</b></summary>
```mdx
1. Persist a chart with `query_context` containing valid JSON such as `{"queries":
"invalid"}`; this is accepted by `json.loads()` at line 119 and produces a truthy
`queries` value.
2. Include that chart in a data-mode dashboard Excel export; `_build_workbook()` at lines
272-291 calls `_resolve_query_context()` for the chart.
3. `_saved_query_context()` at lines 122-124 returns the parsed dictionary because it
checks only that `queries` is truthy, not that it is a non-empty list of query objects.
4. `_write_chart_sheets()` receives the malformed payload at line 291 and passes it into
the downstream chart-data execution path, where schema validation or query iteration
fails. The exception is handled as a generic chart export failure instead of being
classified under `email.ERROR_NO_QUERY_CONTEXT` for re-saving.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=97e08063b0114f4da5da48eb585e4715&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=97e08063b0114f4da5da48eb585e4715&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/tasks/export_dashboard_excel.py
**Line:** 122:124
**Comment:**
*Possible Bug: `_saved_query_context` only checks that `queries` is truthy, so malformed contexts such as `{"queries": "invalid"}` or other non-list values are accepted and passed to `ChartDataCommand`. This causes validation or iteration failures to be reported as a generic chart export error instead of being classified as missing query context and listed for remediation. Require `queries` to be a non-empty list with the expected query-object structure before returning the parsed context.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=a783dd45d0c570421663b1480006712286c8a8303d8729ad0e6d57f4805e7163&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=a783dd45d0c570421663b1480006712286c8a8303d8729ad0e6d57f4805e7163&reaction=dislike'>๐</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5094414772
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "EnxDev (via GitHub)" <gi...@apache.org>.
EnxDev commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3653209509
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,227 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. Only ``SIMPLE`` WHERE-clause filters are
+ convertible here; free-form ``SQL`` filters have no ``{col, op, val}``
+ equivalent and are handled separately (see :func:`freeform_where_having`).
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if (
+ flt.get("expressionType") == "SIMPLE"
+ and (flt.get("clause") or "WHERE").upper() == "WHERE"
+ ):
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any]
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+ """
+ if order_by_cols := form_data.get("order_by_cols") or []:
+ parsed: list[list[Any]] = []
+ for col in order_by_cols:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
+ except (TypeError, ValueError):
+ continue
+ parsed.append(col)
+ return parsed
+
+ if not metrics:
+ return []
+
+ order_desc = form_data.get("order_desc", True)
+ sort_metric = form_data.get("timeseries_limit_metric") or (
+ metrics[0] if form_data.get("sort_by_metric") else None
+ )
+ if sort_metric is not None:
+ return [[sort_metric, not order_desc]]
+ # No explicit sort metric: default to the first metric, descending.
+ return [[metrics[0], False]]
+
+
+def build_query_context_from_form_data(
+ form_data: dict[str, Any],
+ datasource: dict[str, Any],
+ viz_type: str | None = None,
+) -> dict[str, Any]:
+ """
+ Build a query-context payload (the JSON shape ``ChartDataQueryContextSchema``
+ loads) from a chart's form data and datasource reference.
+
+ :param form_data: The chart's saved ``params`` parsed to a dict.
+ :param datasource: ``{"id": <int>, "type": "table"}`` datasource reference.
+ :param viz_type: The chart's viz type, used for viz-specific handling.
+ :returns: A single-query query-context dict.
+ """
+ metrics = list(form_data.get("metrics") or [])
+ # Single-metric charts (e.g. Big Number) store ``metric`` rather than
+ # ``metrics``.
+ if not metrics and form_data.get("metric"):
+ metrics = [form_data["metric"]]
+ # Table percent metrics are computed as additional query metrics.
+ if viz_type == "table" and form_data.get("percent_metrics"):
+ metrics = [*metrics, *form_data["percent_metrics"]]
+
+ columns = columns_from_form_data(form_data)
+ # Only a Big Number *with a trendline* (viz_type ``big_number``) groups by its
+ # time column; ``big_number_total`` is a single aggregate and must not be
+ # grouped, or it would return one row per timestamp instead of a total.
+ if not columns and viz_type == "big_number" and form_data.get("granularity_sqla"):
+ columns = [form_data["granularity_sqla"]]
+
+ # SIMPLE adhoc filters (+ legacy top-level ``filters``) become query filters;
+ # free-form SQL predicates go into ``extras``.
+ filters = adhoc_filters_to_query_filters(form_data.get("adhoc_filters", []))
+ for flt in form_data.get("filters") or []:
+ if isinstance(flt, dict) and flt.get("col") is not None:
+ filters.append(flt)
+
+ extras = freeform_where_having(form_data)
+ if form_data.get("time_grain_sqla"):
+ extras["time_grain_sqla"] = form_data["time_grain_sqla"]
+
+ time_range = form_data.get("time_range") or "No filter"
Review Comment:
The rebuild only reads `time_range`, but older charts may still use `since/until`. In that case, the export falls back to "No filter" and could export the full history instead of the chart's configured time range.
Should we fall back to `since/until` when `time_range` is missing? It would also be worth adding a regression test for this case.
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,227 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. Only ``SIMPLE`` WHERE-clause filters are
+ convertible here; free-form ``SQL`` filters have no ``{col, op, val}``
+ equivalent and are handled separately (see :func:`freeform_where_having`).
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if (
+ flt.get("expressionType") == "SIMPLE"
+ and (flt.get("clause") or "WHERE").upper() == "WHERE"
+ ):
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
Review Comment:
`columns_from_form_data` now checks the truthiness of `raw_columns` instead of whether `"columns"` exists in `form_data`. As a result, `columns: []` no longer takes precedence over `groupby` in the MCP preview/compile path.
This looks like an improvement, but it changes the previous behavior. Could we confirm this is intentional and add an MCP-path test to cover it?
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,227 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. Only ``SIMPLE`` WHERE-clause filters are
+ convertible here; free-form ``SQL`` filters have no ``{col, op, val}``
+ equivalent and are handled separately (see :func:`freeform_where_having`).
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if (
+ flt.get("expressionType") == "SIMPLE"
+ and (flt.get("clause") or "WHERE").upper() == "WHERE"
+ ):
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
Review Comment:
`adhoc_filters_to_query_filters` now only converts `SIMPLE` filters with a `WHERE` clause, while the previous `chart_utils` implementation converted all `SIMPLE` filters. This means `SIMPLE HAVING` filters previously handled by `mcp_service/chart/compile.py` and `preview_utils.py` are now silently dropped.
Should we preserve the previous behavior here and handle both `WHERE` and `HAVING` filters?
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,227 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. Only ``SIMPLE`` WHERE-clause filters are
+ convertible here; free-form ``SQL`` filters have no ``{col, op, val}``
+ equivalent and are handled separately (see :func:`freeform_where_having`).
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if (
+ flt.get("expressionType") == "SIMPLE"
+ and (flt.get("clause") or "WHERE").upper() == "WHERE"
+ ):
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any]
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+ """
+ if order_by_cols := form_data.get("order_by_cols") or []:
+ parsed: list[list[Any]] = []
+ for col in order_by_cols:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
+ except (TypeError, ValueError):
+ continue
+ parsed.append(col)
+ return parsed
+
+ if not metrics:
+ return []
+
+ order_desc = form_data.get("order_desc", True)
+ sort_metric = form_data.get("timeseries_limit_metric") or (
+ metrics[0] if form_data.get("sort_by_metric") else None
+ )
+ if sort_metric is not None:
+ return [[sort_metric, not order_desc]]
+ # No explicit sort metric: default to the first metric, descending.
+ return [[metrics[0], False]]
+
+
+def build_query_context_from_form_data(
+ form_data: dict[str, Any],
+ datasource: dict[str, Any],
+ viz_type: str | None = None,
+) -> dict[str, Any]:
+ """
+ Build a query-context payload (the JSON shape ``ChartDataQueryContextSchema``
+ loads) from a chart's form data and datasource reference.
+
+ :param form_data: The chart's saved ``params`` parsed to a dict.
+ :param datasource: ``{"id": <int>, "type": "table"}`` datasource reference.
+ :param viz_type: The chart's viz type, used for viz-specific handling.
+ :returns: A single-query query-context dict.
+ """
+ metrics = list(form_data.get("metrics") or [])
+ # Single-metric charts (e.g. Big Number) store ``metric`` rather than
+ # ``metrics``.
+ if not metrics and form_data.get("metric"):
+ metrics = [form_data["metric"]]
+ # Table percent metrics are computed as additional query metrics.
+ if viz_type == "table" and form_data.get("percent_metrics"):
+ metrics = [*metrics, *form_data["percent_metrics"]]
Review Comment:
For table charts, `percent_metrics` are added as regular query metrics without applying the percentage/contribution post-processing.
This means the exported column contains the raw aggregate instead of the "% of total" shown in the chart.
Should we exclude `percent_metrics` from the rebuild to avoid exporting values that don't match what the user sees in the chart?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5049105830
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3689313099
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -458,18 +458,14 @@ def adhoc_filters_to_query_filters(
Adhoc filters use ``{subject, operator, comparator}`` keys while
``QueryContextFactory`` expects ``{col, op, val}`` (QueryObjectFilterClause).
+ Delegates to the shared builder so the MCP and dashboard-export paths stay in
+ sync (single source of truth).
"""
- result: list[Dict[str, Any]] = []
- for f in adhoc_filters:
- if f.get("expressionType") == "SIMPLE":
- result.append(
- {
- "col": f.get("subject"),
- "op": f.get("operator"),
- "val": f.get("comparator"),
- }
- )
- return result
+ from superset.common.form_data_query_context import (
+ adhoc_filters_to_query_filters as _shared,
+ )
+
+ return _shared(adhoc_filters)
Review Comment:
**Suggestion:** The delegated helper calls `flt.get(...)` for every item, so malformed user-supplied `adhoc_filters` entries such as `None` or a scalar now raise `AttributeError` during MCP compile/preview instead of being skipped. Preserve the wrapper's defensive filtering of non-dictionary entries or make the shared helper ignore them before delegation. [null pointer]
<details>
<summary><b>Severity Level:</b> Major โ ๏ธ</summary>
```mdx
- โ MCP chart update validation can fail with an unhandled malformed-filter exception.
- โ ๏ธ MCP chart previews fail for charts with malformed saved filters.
- โ ๏ธ Legacy or cached chart form data can preserve invalid filter entries.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=76156c6ea5c9472b9e75ea679843c131&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=76156c6ea5c9472b9e75ea679843c131&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/mcp_service/chart/chart_utils.py
**Line:** 468:468
**Comment:**
*Null Pointer: The delegated helper calls `flt.get(...)` for every item, so malformed user-supplied `adhoc_filters` entries such as `None` or a scalar now raise `AttributeError` during MCP compile/preview instead of being skipped. Preserve the wrapper's defensive filtering of non-dictionary entries or make the shared helper ignore them before delegation.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=10709ff4d1dbfacb1a51dd02cdf91725ed643f3ec30dcec8cbefbae1d3d315e4&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=10709ff4d1dbfacb1a51dd02cdf91725ed643f3ec30dcec8cbefbae1d3d315e4&reaction=dislike'>๐</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "hughhhh (via GitHub)" <gi...@apache.org>.
hughhhh commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5045805409
## How Has This Been Tested? (real full stack, Docker)
Validated end-to-end against the live stack (Flask + Celery worker + Postgres + Redis + MinIO/S3 + Mailpit/SMTP) serving this branch via bind mount.
**Bring-up** (from workspace root, with the S3/SMTP verify overlay):
```
docker compose -p amman-excel -f docker-compose.yml -f docker-compose-excel-verify.yml \
up -d superset superset-worker superset-init minio createbucket mailpit
```
**Test case โ the natural one:** Superset's example dashboards ship charts that have **no saved `query_context`** (they store `params` only), which is exactly the state this PR handles. The **Video Game Sales** dashboard (8 charts, all with `query_context = NULL`) mixes eligible and ineligible viz types, so one export exercises both code paths:
- **Eligible โ rebuilt from form data & exported** (3 sheets, real data):
- `51 - Games` (`table`) โ **16,595 rows**
- `58 - Publishers With Most Titles` (`table`) โ 10 rows
- `53 - Most Dominant Platforms` (`pie`) โ groupby + `SUM(Global_Sales)`
- **Ineligible โ skipped & listed in the email** (`echarts_timeseries_bar` ร2, `heatmap_v2`, `treemap_v2` ร2): charts 54, 55, 56, 52, 57.
Before this PR every one of these 8 charts would have been skipped (empty workbook); now the 3 rebuildable ones export with correct data.
**Non-visual proof:**
- Worker log: `Task export_dashboard_excel[54267fdb-โฆ] succeeded in 1.08s`
- MinIO: `dashboard-exports/5/54267fdb-โฆ.xlsx` (913 KB, 3 sheets โ verified sheet names + row counts via openpyxl)
- Mailpit email subject "โฆexport is ready: Video Game Sales" with a working download link and the 5 omitted charts under *"they have no saved query context. To include them, open each chart in Explore and re-save."*
Screenshots (in `.context/empty-query-context-screenshots/`, gitignored โ attaching): `01-dashboard`, `02-download-menu` (Export Data to Excel), `03-export-toast`, `04-email-skipped-list`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3640004099
##########
tests/unit_tests/tasks/test_export_dashboard_excel.py:
##########
@@ -182,6 +182,184 @@ def test_chart_without_query_context_is_skipped(mocks: dict[str, Any]) -> None:
}
+@pytest.mark.parametrize(
+ "raw_context",
+ [
+ "", # blank
+ "null", # parses to None
+ "{}", # object with no queries
+ '{"queries": []}', # object with an empty queries list
+ "not valid json", # unparseable
+ ],
+)
+def test_chart_with_empty_query_context_is_skipped(
+ mocks: dict[str, Any], raw_context: str
+) -> None:
+ # A present-but-empty/unusable query context is treated the same as a
+ # missing one: the chart is listed under "no query context" and the export
+ # continues, rather than raising mid-export and landing in the general bucket.
+ good = _chart(10, "Good")
+ empty = _chart(20, "Empty")
+ empty.query_context = raw_context
+ mocks["get_charts_in_layout_order"].return_value = [good, empty]
+ mocks["ChartDataCommand"].return_value.run.return_value = {
+ "queries": [{"colnames": ["a"], "data": [{"a": 1}]}]
+ }
+
+ _run()
+
+ _, kwargs = mocks["email"].build_success_email.call_args
+ assert kwargs["errored"] == {mocks["email"].ERROR_NO_QUERY_CONTEXT: ["20 - Empty"]}
+ # The empty chart is skipped before any query runs; only the good one runs.
+ mocks["ChartDataCommand"].return_value.run.assert_called_once()
+
+
+def test_empty_query_context_rebuilt_from_form_data_for_eligible_viz(
+ mocks: dict[str, Any],
+) -> None:
+ # An eligible viz type (table) with no saved query context is rebuilt from
+ # its form data and exported instead of being skipped.
+ good = _chart(10, "Good")
+ rebuilt = _chart(20, "Rebuilt", viz_type="table")
+ rebuilt.query_context = None
+ rebuilt.params = json.dumps({"groupby": ["country"], "metrics": ["count"]})
+ rebuilt.datasource_id = 5
+ rebuilt.datasource_type = "table"
+ mocks["get_charts_in_layout_order"].return_value = [good, rebuilt]
+ mocks["ChartDataCommand"].return_value.run.return_value = {
+ "queries": [{"colnames": ["a"], "data": [{"a": 1}]}]
+ }
+
+ _run()
+
+ _, kwargs = mocks["email"].build_success_email.call_args
+ assert kwargs["errored"] == {}
+ # Both charts ran a query (the saved one and the rebuilt one).
+ assert mocks["ChartDataCommand"].return_value.run.call_count == 2
+
+
+def test_empty_query_context_ineligible_viz_is_skipped(
+ mocks: dict[str, Any],
+) -> None:
+ # A viz type outside the rebuild allowlist (mixed_timeseries โ a multi-query
+ # chart the generic rebuild can't reproduce) is skipped, not exported wrong.
+ good = _chart(10, "Good")
+ ineligible = _chart(20, "Ineligible", viz_type="mixed_timeseries")
+ ineligible.query_context = None
+ ineligible.params = json.dumps({"groupby": ["x"], "metrics": ["count"]})
+ ineligible.datasource_id = 5
Review Comment:
**Suggestion:** This test is intended to prove skip behavior is driven by an ineligible viz type, but it never sets `datasource_type` on the chart. If rebuild preconditions require a complete datasource, the chart can be skipped for missing datasource metadata instead, so the test can pass even when allowlist gating is broken. Set a valid datasource type so the only reason to skip is viz ineligibility. [logic error]
<details>
<summary><b>Severity Level:</b> Major โ ๏ธ</summary>
```mdx
- โ ๏ธ Ineligible viz allowlist gating may remain untested.
- โ ๏ธ Export skip reason obscured by missing datasource_type.
- โ ๏ธ Tests may pass while rebuild allowlist misconfigured.
```
</details>
<details>
<summary><b>Steps of Reproduction โ
</b></summary>
```mdx
1. Run `pytest tests/unit_tests/tasks/test_export_dashboard_excel.py` and focus on
`test_empty_query_context_ineligible_viz_is_skipped` at lines 241โ263 in
`tests/unit_tests/tasks/test_export_dashboard_excel.py`.
2. Inside this test, the ineligible chart is set up at lines 247โ250: `ineligible =
_chart(20, "Ineligible", viz_type="mixed_timeseries")`, `ineligible.query_context = None`,
`ineligible.params = json.dumps({"groupby": ["x"], "metrics": ["count"]})`, and
`ineligible.datasource_id = 5`; note that `ineligible.datasource_type` is never set.
3. Compare this setup with the rebuild tests for eligible viz types:
`test_empty_query_context_rebuilt_from_form_data_for_eligible_viz` (lines 217โ239) and
`test_eligible_viz_skipped_when_form_data_unusable` (lines 273โ295), where charts
explicitly set both `datasource_id` and `datasource_type = "table"` before invoking
`_run()`. This indicates the production rebuild logic in
`superset.tasks.export_dashboard_excel` requires complete datasource metadata (both id and
type) as a precondition.
4. If the allowlist gating in `superset.tasks.export_dashboard_excel._rebuild_viz_types()`
(tested at lines 344โ360) is accidentally broadened to include `"mixed_timeseries"`, the
ineligible chart in `test_empty_query_context_ineligible_viz_is_skipped` can still be
skipped solely because `datasource_type` is missing. The test will pass, but it will be
exercising the โmissing datasource metadataโ path instead of the intended โviz type
outside allowlistโ path, so a bug in allowlist gating goes undetected.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=69eeeef853814e52b56c376f6faf54ac&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=69eeeef853814e52b56c376f6faf54ac&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** tests/unit_tests/tasks/test_export_dashboard_excel.py
**Line:** 247:250
**Comment:**
*Logic Error: This test is intended to prove skip behavior is driven by an ineligible viz type, but it never sets `datasource_type` on the chart. If rebuild preconditions require a complete datasource, the chart can be skipped for missing datasource metadata instead, so the test can pass even when allowlist gating is broken. Set a valid datasource type so the only reason to skip is viz ineligibility.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=6aa3ac667173a7e16b8a1ec4aee090965c5fe4bfd2e1eea6b2f2138934948213&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=6aa3ac667173a7e16b8a1ec4aee090965c5fe4bfd2e1eea6b2f2138934948213&reaction=dislike'>๐</a>
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,228 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. Only ``SIMPLE`` WHERE-clause filters are
+ convertible here; free-form ``SQL`` filters have no ``{col, op, val}``
+ equivalent and are handled separately (see :func:`freeform_where_having`).
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if (
+ flt.get("expressionType") == "SIMPLE"
+ and (flt.get("clause") or "WHERE").upper() == "WHERE"
+ ):
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
Review Comment:
**Suggestion:** The function documentation claims columns are de-duplicated, but the implementation only prevents duplicate insertion for `x_axis` and does not de-duplicate existing `groupby`/`columns` entries. This contradiction can produce duplicate selected/grouped columns and inconsistent query output; either actually de-duplicate or correct the contract. [docstring mismatch]
<details>
<summary><b>Severity Level:</b> Minor ๐งน</summary>
```mdx
- โ ๏ธ Docstring overstates de-duplication actually implemented.
- โ ๏ธ Duplicate columns unlikely from normal frontend form_data.
```
</details>
<details>
<summary><b>Steps of Reproduction โ
</b></summary>
```mdx
1. Trigger a dashboard Excel export so `_build_workbook`
(`superset/tasks/export_dashboard_excel.py:255`) runs and iterates charts from
`get_charts_in_layout_order`.
2. For a chart without saved `query_context` but with saved `params` (`chart.params`),
`_build_workbook` calls `_resolve_query_context`
(`superset/tasks/export_dashboard_excel.py:137`), which parses `chart.params` and invokes
`build_query_context_from_form_data` in `superset/common/form_data_query_context.py:162`.
3. `build_query_context_from_form_data` calls `columns_from_form_data`
(`superset/common/form_data_query_context.py:94`), which at lines 107โ112 simply copies
`form_data["columns"]` or `form_data["groupby"]` into `columns` without de-duplicating
within those lists; it only avoids inserting a duplicate `x_axis` later in the function.
4. In practice, frontend-generated `form_data` for charts does not contain duplicate
entries in `columns` or `groupby`, and there is no backend code that injects duplicates,
so any duplication would require malformed or manually edited `params`; the mismatch is
between the docstring (โde-duplicating while preserving orderโ) and implementation rather
than a reproducible bug in normal usage.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=bf58f290afa3490db9ef3792c118f38e&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=bf58f290afa3490db9ef3792c118f38e&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/common/form_data_query_context.py
**Line:** 107:112
**Comment:**
*Docstring Mismatch: The function documentation claims columns are de-duplicated, but the implementation only prevents duplicate insertion for `x_axis` and does not de-duplicate existing `groupby`/`columns` entries. This contradiction can produce duplicate selected/grouped columns and inconsistent query output; either actually de-duplicate or correct the contract.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=9242ad889b888972fee1afda8935a091ceef2f0228aa0755024f735cc2ac3ab3&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=9242ad889b888972fee1afda8935a091ceef2f0228aa0755024f735cc2ac3ab3&reaction=dislike'>๐</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "hughhhh (via GitHub)" <gi...@apache.org>.
hughhhh commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5061179587
Thanks โ all findings addressed in `719649fe1a`, and I re-validated end-to-end against the live stack (which caught a regression the static review couldn't, see below).
**๐ด Functional**
- **granularity / time range** โ the rebuild now sets `granularity` (from `granularity`/`granularity_sqla`) so `time_range` is actually applied. **Live validation caught an over-reach here:** the example charts store `granularity_sqla: "year"` (a numeric column) with `time_range: "No filter"`, and unconditionally setting `granularity` made the query `date_trunc` a numeric column โ charts that exported fine now failed. Fixed by only setting `granularity` when there's an *active* range (which is exactly the reviewer's scenario). Confirmed: the 3 charts export again, and no `date_trunc` error.
- **custom-SQL filters** โ `SQL` adhoc filters + legacy top-level `where` now map into `extras.where`/`extras.having` by clause (mirroring `processFilters`), instead of being dropped.
- **orderby** โ derived from `order_by_cols` (raw) or `timeseries_limit_metric`/`sort_by_metric`/`order_desc`, falling back to first-metric-descending. **Live-confirmed:** the "Games" table now exports "Wii Sports" (82.7) first instead of an arbitrary "Happy Feet Two" (0.1).
- **table percent_metrics / time_grain** โ `percent_metrics` carried into metrics; `time_grain_sqla` passed via `extras`.
- **big_number_total** โ granularity_sqla is promoted to a grouping column only for the trendline viz (`big_number`), never `big_number_total`.
**๐ก Should-fix**
- **duplication** โ MCP `adhoc_filters_to_query_filters` (chart_utils) and `_build_query_columns` (preview_utils) now delegate to the shared module, so the `columns` bug divergence is gone; MCP tests still pass.
- **test depth** โ added an export test asserting the full query body (columns/filters/orderby/granularity/time_range/row_limit) that reaches `ChartDataQueryContextSchema().load`, plus unit tests for every derived field.
**๐ต Nits** โ restructured the loop to drop the `-O`-stripped `assert`, parse the saved `query_context` once, and corrected the shallow-copy comment.
`form_data_query_context.py` is at 100% coverage; all unit + affected MCP tests green.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5050941348
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "hughhhh (via GitHub)" <gi...@apache.org>.
hughhhh commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3667644393
##########
superset/tasks/export_dashboard_excel.py:
##########
@@ -67,6 +68,12 @@
# image. Operators can override the set via ``EXCEL_EXPORT_TABLE_VIZ_TYPES``.
TABLE_VIZ_TYPES = {"table", "pivot_table_v2", "pivot_table"}
+# Viz types whose missing query context may be rebuilt from saved form data.
+# Conservative by default: only charts whose data maps faithfully to a single
+# plain query (no post-processing, no multi-query fan-out). Operators can
+# override via ``EXCEL_EXPORT_REBUILD_VIZ_TYPES``.
+REBUILD_VIZ_TYPES = {"table", "big_number_total", "big_number", "pie"}
Review Comment:
Good point โ the viz-type allowlist isn't sufficient on its own. Fixed in 808ba6f4b4: `_resolve_query_context` now also skips the rebuild (โ re-save list) when the form data uses processing the single query can't reproduce โ `time_compare`, `rolling_type` (ignoring the literal `"None"`), `resample_rule`, or `aggregation == "raw"`. Added a parametrized regression test covering all four (including `time_compare`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5172960850
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "EnxDev (via GitHub)" <gi...@apache.org>.
EnxDev commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5177534191
EnxDev's Review Agent โ apache/superset#42284 ยท HEAD 14c86a3
request changes โ the rebuild diverges from the mirrored frontend buildQuery in four places, and UPDATING.md documents a config key that does not exist in the code.
Supersedes my earlier review on de7dfc1 (comment). All nine findings from it are genuinely fixed in this diff โ I re-verified each against the code, not the "fixed in X" replies: since/until fallback (:296), where_only HAVING split (:105), columns: [] no longer shadowing groupby (:162), percent_metrics/time_compare/rolling/resample skips, Big Number granularity promotion, raw-mode stale metrics/groupby, table order_desc default, and the stale order_by_cols gate. Good follow-through.
๐ด Functional
- superset/common/form_data_query_context.py:104 ยท High โ freeform_where_having wraps SQL clauses as ({clause}) with no newline, but the mirrored processFilters.ts:26-32 appends \n when the clause contains --. A chart whose free-form SQL filter ends in a -- comment produces (sales > 0 -- note) on one line: the closing paren and every subsequent AND (...) are swallowed by the comment, so the export fails where the chart renders fine. Port sanitizeClause verbatim. regression test: an adhoc_filters SQL entry "sales > 0 -- note" plus a second SQL WHERE clause; assert extras["where"] contains a newline before the closing paren and both predicates survive.
- superset/tasks/export_dashboard_excel.py:75 ยท Medium โ pie is on the rebuild allowlist, but Pie/buildQuery.ts:34-42 attaches a contribution post-processing op unconditionally, not only when percent_metrics is set. contribution() writes rename_columns as new columns (contribution.py:98,164) and getContributionLabel appends a distinct suffix, so a rebuilt pie sheet loses the percentage column a saved-context pie sheet has โ two pies on one dashboard export different columns depending only on whether they were re-saved. Either apply the contribution op for pie or drop pie from the allowlist. regression test: rebuild a pie and assert the query carries the contribution post_processing entry.
- superset/tasks/export_dashboard_excel.py:134 ยท Medium โ _needs_unsupported_processing misses show_totals. buildQuery.ts:345-365 pushes a second totals query when metrics && show_totals && aggregate; the single-query rebuild drops it, so the sheet silently omits the totals row. This is the same class of defect as percent_metrics, which you already decided warrants a skip. Add show_totals (aggregate mode only) to the skip check. regression test: table params with show_totals: true โ chart listed under ERROR_NO_QUERY_CONTEXT, no query run.
- superset/common/form_data_query_context.py:186 ยท Medium โ timeseries_limit_metric is read raw, but the frontend normalizes it with ensureIsArray(...)[0] (buildQuery.ts:67). A chart storing ["revenue"] yields orderby: [[["revenue"], True]], which fails downstream and dumps the chart into the general error bucket instead of exporting. Unwrap a single-element list. regression test: timeseries_limit_metric: ["revenue"] โ orderby == [["revenue", True]].
- UPDATING.md:76 ยท High โ EXCEL_EXPORT_REBUILD_VIZ_TYPES is documented as a new config key and the PR summary calls the allowlist "configurable", but it exists nowhere in the codebase: REBUILD_VIZ_TYPES (export_dashboard_excel.py:75) is a hardcoded constant with no current_app.config lookup, superset/config.py is not in this diff, and test_rebuild_viz_types_is_the_conservative_default asserts "no config override". Note this also reverts the fix promised to codeant-ai ("_rebuild_viz_types now checks is None explicitly") โ that function is gone at HEAD. Operators following the release notes would set a key that does nothing, with no way to un-skip their charts. Either restore the config read (mirroring _table_viz_types() at :105 plus a config.py default) or drop the key from UPDATING.md and the PR summary.
๐ก Should-fix
- superset/tasks/export_dashboard_excel.py:106 โ _saved_query_context only tests parsed.get("queries") for truthiness, so {"queries": "oops"} passes as a valid saved context and fails later in the general bucket rather than the clean "no query context" path. Check isinstance(..., list).
๐ต Nits
- superset/common/form_data_query_context.py:206-215 โ parsed order_by_cols entries aren't validated as 2-element [col, asc] pairs; a stray null appends None to orderby.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3675696326
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,273 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+ where_only: bool = False,
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
+ val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+
+ By default all ``SIMPLE`` filters are converted (the behavior the MCP
+ compile/preview path relies on). Pass ``where_only=True`` to convert only
+ ``WHERE``-clause filters, matching the frontend's ``processFilters`` โ the
+ dashboard export uses this so it applies the same rows the chart shows and
+ does not additionally filter on ``SIMPLE`` ``HAVING`` clauses.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") != "SIMPLE":
+ continue
+ if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
+ continue
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def is_raw_query_mode(form_data: dict[str, Any]) -> bool:
+ """
+ Whether the chart runs in raw (non-aggregated) mode, mirroring the frontend's
+ ``getQueryMode``: an explicit ``query_mode`` wins, otherwise the presence of
+ ``all_columns`` implies raw mode.
+ """
+ if mode := form_data.get("query_mode"):
+ return mode == "raw"
+ return bool(form_data.get("all_columns"))
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any], viz_type: str | None = None
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+ """
+ if order_by_cols := form_data.get("order_by_cols") or []:
+ parsed: list[list[Any]] = []
+ for col in order_by_cols:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
+ except (TypeError, ValueError):
+ continue
+ parsed.append(col)
+ return parsed
Review Comment:
**Suggestion:** Successfully parsed `order_by_cols` values are appended without validating that they are two-element `[column, ascending]` pairs. Values such as `null`, a scalar, a dictionary, or a one-element list reach `ChartDataQueryContextSchema`, which expects tuple pairs and rejects the entire chart export instead of treating the malformed legacy ordering as absent. Validate the shape before appending it. [type error]
<details>
<summary><b>Severity Level:</b> Major โ ๏ธ</summary>
```mdx
- โ Malformed ordering can abort chart Excel export.
- โ ๏ธ One legacy chart can be skipped from the workbook.
- โ ๏ธ Recoverable form-data corruption becomes a user-visible error.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=13cbb7dbd4fd4dfd9cbb9e0807405654&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=13cbb7dbd4fd4dfd9cbb9e0807405654&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/common/form_data_query_context.py
**Line:** 155:164
**Comment:**
*Type Error: Successfully parsed `order_by_cols` values are appended without validating that they are two-element `[column, ascending]` pairs. Values such as `null`, a scalar, a dictionary, or a one-element list reach `ChartDataQueryContextSchema`, which expects tuple pairs and rejects the entire chart export instead of treating the malformed legacy ordering as absent. Validate the shape before appending it.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=1d6de6cec9830db57aa412c1d022de22ece118f8b038071da9eb26bc07b2733f&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=1d6de6cec9830db57aa412c1d022de22ece118f8b038071da9eb26bc07b2733f&reaction=dislike'>๐</a>
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,273 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+ where_only: bool = False,
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
+ val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+
+ By default all ``SIMPLE`` filters are converted (the behavior the MCP
+ compile/preview path relies on). Pass ``where_only=True`` to convert only
+ ``WHERE``-clause filters, matching the frontend's ``processFilters`` โ the
+ dashboard export uses this so it applies the same rows the chart shows and
+ does not additionally filter on ``SIMPLE`` ``HAVING`` clauses.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") != "SIMPLE":
+ continue
+ if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
+ continue
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
Review Comment:
**Suggestion:** Unlike the frontend filter processor, this path does not append a newline when a free-form SQL clause contains `--`. The generated wrapper can therefore turn the closing parenthesis and following SQL into a comment, causing a syntax error or changing the predicate semantics during export. Apply the same clause sanitization as `processFilters` before placing expressions in `extras`. [logic error]
<details>
<summary><b>Severity Level:</b> Major โ ๏ธ</summary>
```mdx
- โ SQL-filtered Excel exports can fail query execution.
- โ ๏ธ Export predicates can differ from displayed chart results.
- โ ๏ธ Affected charts are omitted or reported as export errors.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=a04831f9e8cb4c7885324b90dc2dc267&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=a04831f9e8cb4c7885324b90dc2dc267&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/common/form_data_query_context.py
**Line:** 86:91
**Comment:**
*Logic Error: Unlike the frontend filter processor, this path does not append a newline when a free-form SQL clause contains `--`. The generated wrapper can therefore turn the closing parenthesis and following SQL into a comment, causing a syntax error or changing the predicate semantics during export. Apply the same clause sanitization as `processFilters` before placing expressions in `extras`.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=2dbd4d39c8ce1dc5e7ca21a07c207dc796fa8799a78ed5384a6a202412a45e16&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=2dbd4d39c8ce1dc5e7ca21a07c207dc796fa8799a78ed5384a6a202412a45e16&reaction=dislike'>๐</a>
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,273 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+ where_only: bool = False,
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
+ val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+
+ By default all ``SIMPLE`` filters are converted (the behavior the MCP
+ compile/preview path relies on). Pass ``where_only=True`` to convert only
+ ``WHERE``-clause filters, matching the frontend's ``processFilters`` โ the
+ dashboard export uses this so it applies the same rows the chart shows and
+ does not additionally filter on ``SIMPLE`` ``HAVING`` clauses.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") != "SIMPLE":
+ continue
+ if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
+ continue
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def is_raw_query_mode(form_data: dict[str, Any]) -> bool:
+ """
+ Whether the chart runs in raw (non-aggregated) mode, mirroring the frontend's
+ ``getQueryMode``: an explicit ``query_mode`` wins, otherwise the presence of
+ ``all_columns`` implies raw mode.
+ """
+ if mode := form_data.get("query_mode"):
+ return mode == "raw"
+ return bool(form_data.get("all_columns"))
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any], viz_type: str | None = None
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+ """
+ if order_by_cols := form_data.get("order_by_cols") or []:
+ parsed: list[list[Any]] = []
+ for col in order_by_cols:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
+ except (TypeError, ValueError):
+ continue
+ parsed.append(col)
+ return parsed
+
+ if not metrics:
+ return []
+
+ sort_metric = form_data.get("timeseries_limit_metric") or (
+ metrics[0] if form_data.get("sort_by_metric") else None
+ )
+ if sort_metric is not None:
+ # The Table plugin defaults ``order_desc`` to False (ascending); Pie and
+ # others sort by metric descending. Match that so a row limit keeps the
+ # chart's top/bottom-N rather than flipping it.
+ default_desc = viz_type != "table"
+ order_desc = form_data.get("order_desc", default_desc)
+ return [[sort_metric, not order_desc]]
+ # No explicit sort metric: default to the first metric, descending.
+ return [[metrics[0], False]]
+
+
+def _columns_and_metrics(
+ form_data: dict[str, Any], viz_type: str | None
+) -> tuple[list[Any], list[Any], bool]:
+ """
+ Resolve the query's ``(columns, metrics, promoted_time_column)`` from form
+ data, honoring raw vs. aggregate mode and the Big Number trendline promotion.
+ """
+ if is_raw_query_mode(form_data):
+ # Raw mode returns individual rows: use only the selected columns and
+ # ignore ``metrics``/``groupby``, which stay in form data as stale values
+ # (the controls aren't reset when hidden) but are ignored by the chart.
+ columns = list(form_data.get("all_columns") or form_data.get("columns") or [])
+ return columns, [], False
+
+ metrics = list(form_data.get("metrics") or [])
+ # Single-metric charts (e.g. Big Number) store ``metric`` rather than
+ # ``metrics``.
+ if not metrics and form_data.get("metric"):
+ metrics = [form_data["metric"]]
+ columns = columns_from_form_data(form_data)
+ # Only a Big Number *with a trendline* (viz_type ``big_number``) groups by its
+ # time column; ``big_number_total`` is a single aggregate and must not be
+ # grouped, or it would return one row per timestamp instead of a total.
+ if not columns and viz_type == "big_number" and form_data.get("granularity_sqla"):
+ return [form_data["granularity_sqla"]], metrics, True
+ return columns, metrics, False
+
+
+def build_query_context_from_form_data(
+ form_data: dict[str, Any],
+ datasource: dict[str, Any],
+ viz_type: str | None = None,
+) -> dict[str, Any]:
+ """
+ Build a query-context payload (the JSON shape ``ChartDataQueryContextSchema``
+ loads) from a chart's form data and datasource reference.
+
+ :param form_data: The chart's saved ``params`` parsed to a dict.
+ :param datasource: ``{"id": <int>, "type": "table"}`` datasource reference.
+ :param viz_type: The chart's viz type, used for viz-specific handling.
+ :returns: A single-query query-context dict.
+ """
+ columns, metrics, promoted_time_column = _columns_and_metrics(form_data, viz_type)
+
+ # SIMPLE adhoc filters (+ legacy top-level ``filters``) become query filters;
+ # free-form SQL predicates go into ``extras``. Only ``WHERE``-clause SIMPLE
+ # filters are applied (matching the chart), so the export never filters on a
+ # ``HAVING`` clause the chart itself ignores.
+ filters = adhoc_filters_to_query_filters(
+ form_data.get("adhoc_filters", []), where_only=True
+ )
+ for flt in form_data.get("filters") or []:
+ if isinstance(flt, dict) and flt.get("col") is not None:
+ filters.append(flt)
+
+ extras = freeform_where_having(form_data)
+ if form_data.get("time_grain_sqla"):
+ extras["time_grain_sqla"] = form_data["time_grain_sqla"]
+
+ # Prefer the modern ``time_range``; fall back to the legacy ``since``/``until``
+ # pair (older charts store the range that way) before defaulting to no filter.
+ time_range = form_data.get("time_range")
+ if not time_range and (form_data.get("since") or form_data.get("until")):
+ time_range = f"{form_data.get('since') or ''} : {form_data.get('until') or ''}"
+ time_range = time_range or "No filter"
+ query: dict[str, Any] = {
+ "columns": columns,
+ "metrics": metrics,
+ "orderby": orderby_from_form_data(form_data, metrics, viz_type),
+ "filters": filters,
+ "time_range": time_range,
+ }
Review Comment:
**Suggestion:** The generic rebuild omits table and other chart controls such as `series_limit` and `series_limit_metric`. A chart with a series limit can therefore return arbitrary groups up to `row_limit` instead of the configured top series, producing materially different export data while still being treated as a successful rebuild. These controls must be translated into the query context or such charts must be skipped. [api mismatch]
<details>
<summary><b>Severity Level:</b> Critical ๐จ</summary>
```mdx
- โ Allowlisted chart exports can contain incorrect groups.
- โ ๏ธ Top-series and row-limit behavior diverges from charts.
- โ ๏ธ Exported Excel data may not match displayed chart data.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=988401a3ca0f4f0bb641f1bc0431766b&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=988401a3ca0f4f0bb641f1bc0431766b&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/common/form_data_query_context.py
**Line:** 248:254
**Comment:**
*Api Mismatch: The generic rebuild omits table and other chart controls such as `series_limit` and `series_limit_metric`. A chart with a series limit can therefore return arbitrary groups up to `row_limit` instead of the configured top series, producing materially different export data while still being treated as a successful rebuild. These controls must be translated into the query context or such charts must be skipped.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=5986776cc8aca277243c3bb72482375394026f653e5c055d5987d35ea959598a&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=5986776cc8aca277243c3bb72482375394026f653e5c055d5987d35ea959598a&reaction=dislike'>๐</a>
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,273 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+ where_only: bool = False,
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
+ val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+
+ By default all ``SIMPLE`` filters are converted (the behavior the MCP
+ compile/preview path relies on). Pass ``where_only=True`` to convert only
+ ``WHERE``-clause filters, matching the frontend's ``processFilters`` โ the
+ dashboard export uses this so it applies the same rows the chart shows and
+ does not additionally filter on ``SIMPLE`` ``HAVING`` clauses.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") != "SIMPLE":
+ continue
+ if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
+ continue
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
Review Comment:
**Suggestion:** An adhoc x-axis object representing a calculated column is reduced to only `column_name`, discarding its `sqlExpression` and other expression metadata. The rebuilt query then references the physical column name instead of the calculated expression, so legacy charts using SQL or calculated x-axis columns can fail or return different data. Preserve the full adhoc column definition, or explicitly reject this form when the builder cannot support it. [type error]
<details>
<summary><b>Severity Level:</b> Major โ ๏ธ</summary>
```mdx
- โ Calculated-axis chart exports can omit their grouping.
- โ ๏ธ Exported results can differ from chart visualization.
- โ ๏ธ SQL-expression axes may cause query failures.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=ca059bf18643404f80b0cf68c78043ab&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=ca059bf18643404f80b0cf68c78043ab&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/common/form_data_query_context.py
**Line:** 121:127
**Comment:**
*Type Error: An adhoc x-axis object representing a calculated column is reduced to only `column_name`, discarding its `sqlExpression` and other expression metadata. The rebuilt query then references the physical column name instead of the calculated expression, so legacy charts using SQL or calculated x-axis columns can fail or return different data. Preserve the full adhoc column definition, or explicitly reject this form when the builder cannot support it.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=5ddaa6ae3a9c4f68bbd4b94005d760ffba4bb908291381a4d765b2c169632a34&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=5ddaa6ae3a9c4f68bbd4b94005d760ffba4bb908291381a4d765b2c169632a34&reaction=dislike'>๐</a>
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,273 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+ where_only: bool = False,
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
+ val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+
+ By default all ``SIMPLE`` filters are converted (the behavior the MCP
+ compile/preview path relies on). Pass ``where_only=True`` to convert only
+ ``WHERE``-clause filters, matching the frontend's ``processFilters`` โ the
+ dashboard export uses this so it applies the same rows the chart shows and
+ does not additionally filter on ``SIMPLE`` ``HAVING`` clauses.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") != "SIMPLE":
+ continue
+ if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
+ continue
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def is_raw_query_mode(form_data: dict[str, Any]) -> bool:
+ """
+ Whether the chart runs in raw (non-aggregated) mode, mirroring the frontend's
+ ``getQueryMode``: an explicit ``query_mode`` wins, otherwise the presence of
+ ``all_columns`` implies raw mode.
+ """
+ if mode := form_data.get("query_mode"):
+ return mode == "raw"
+ return bool(form_data.get("all_columns"))
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any], viz_type: str | None = None
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+ """
+ if order_by_cols := form_data.get("order_by_cols") or []:
+ parsed: list[list[Any]] = []
+ for col in order_by_cols:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
+ except (TypeError, ValueError):
+ continue
+ parsed.append(col)
+ return parsed
+
+ if not metrics:
+ return []
+
+ sort_metric = form_data.get("timeseries_limit_metric") or (
+ metrics[0] if form_data.get("sort_by_metric") else None
+ )
+ if sort_metric is not None:
+ # The Table plugin defaults ``order_desc`` to False (ascending); Pie and
+ # others sort by metric descending. Match that so a row limit keeps the
+ # chart's top/bottom-N rather than flipping it.
+ default_desc = viz_type != "table"
+ order_desc = form_data.get("order_desc", default_desc)
+ return [[sort_metric, not order_desc]]
+ # No explicit sort metric: default to the first metric, descending.
+ return [[metrics[0], False]]
+
+
+def _columns_and_metrics(
+ form_data: dict[str, Any], viz_type: str | None
+) -> tuple[list[Any], list[Any], bool]:
+ """
+ Resolve the query's ``(columns, metrics, promoted_time_column)`` from form
+ data, honoring raw vs. aggregate mode and the Big Number trendline promotion.
+ """
+ if is_raw_query_mode(form_data):
+ # Raw mode returns individual rows: use only the selected columns and
+ # ignore ``metrics``/``groupby``, which stay in form data as stale values
+ # (the controls aren't reset when hidden) but are ignored by the chart.
+ columns = list(form_data.get("all_columns") or form_data.get("columns") or [])
+ return columns, [], False
+
+ metrics = list(form_data.get("metrics") or [])
+ # Single-metric charts (e.g. Big Number) store ``metric`` rather than
+ # ``metrics``.
+ if not metrics and form_data.get("metric"):
+ metrics = [form_data["metric"]]
+ columns = columns_from_form_data(form_data)
+ # Only a Big Number *with a trendline* (viz_type ``big_number``) groups by its
+ # time column; ``big_number_total`` is a single aggregate and must not be
+ # grouped, or it would return one row per timestamp instead of a total.
+ if not columns and viz_type == "big_number" and form_data.get("granularity_sqla"):
+ return [form_data["granularity_sqla"]], metrics, True
+ return columns, metrics, False
+
+
+def build_query_context_from_form_data(
+ form_data: dict[str, Any],
+ datasource: dict[str, Any],
+ viz_type: str | None = None,
+) -> dict[str, Any]:
+ """
+ Build a query-context payload (the JSON shape ``ChartDataQueryContextSchema``
+ loads) from a chart's form data and datasource reference.
+
+ :param form_data: The chart's saved ``params`` parsed to a dict.
+ :param datasource: ``{"id": <int>, "type": "table"}`` datasource reference.
+ :param viz_type: The chart's viz type, used for viz-specific handling.
+ :returns: A single-query query-context dict.
+ """
+ columns, metrics, promoted_time_column = _columns_and_metrics(form_data, viz_type)
+
+ # SIMPLE adhoc filters (+ legacy top-level ``filters``) become query filters;
+ # free-form SQL predicates go into ``extras``. Only ``WHERE``-clause SIMPLE
+ # filters are applied (matching the chart), so the export never filters on a
+ # ``HAVING`` clause the chart itself ignores.
+ filters = adhoc_filters_to_query_filters(
+ form_data.get("adhoc_filters", []), where_only=True
+ )
+ for flt in form_data.get("filters") or []:
+ if isinstance(flt, dict) and flt.get("col") is not None:
+ filters.append(flt)
Review Comment:
**Suggestion:** Legacy filter entries are accepted whenever they contain a `col`, even if they lack the required `op` or `val` fields. Such malformed entries later reach query-context processing, which indexes the missing filter fields and can raise an exception for the whole chart. Require a complete query-filter shape before appending legacy filters. [error handling]
<details>
<summary><b>Severity Level:</b> Major โ ๏ธ</summary>
```mdx
- โ Corrupt legacy filters can fail chart export.
- โ ๏ธ Affected charts may be omitted from workbooks.
- โ ๏ธ Malformed saved form data reaches query processing.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=11e6a83ab3a94a1abcbcda7d8dc4567c&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=11e6a83ab3a94a1abcbcda7d8dc4567c&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/common/form_data_query_context.py
**Line:** 234:236
**Comment:**
*Error Handling: Legacy filter entries are accepted whenever they contain a `col`, even if they lack the required `op` or `val` fields. Such malformed entries later reach query-context processing, which indexes the missing filter fields and can raise an exception for the whole chart. Require a complete query-filter shape before appending legacy filters.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=e883e605f7d2415345a2b8ddbd45d508cb63448e2b3fdfb70cc2d02614b4a181&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=e883e605f7d2415345a2b8ddbd45d508cb63448e2b3fdfb70cc2d02614b4a181&reaction=dislike'>๐</a>
##########
superset/tasks/export_dashboard_excel.py:
##########
@@ -96,6 +103,92 @@ def _chart_label(chart: Any) -> str:
return f"{chart.id} - {chart.slice_name or ''}".strip()
+def _saved_query_context(raw: Any) -> dict[str, Any] | None:
+ """
+ The chart's saved query context parsed to a dict, or ``None`` when it is
+ missing or unusable.
+
+ Returns ``None`` for a blank value, a string that does not parse as JSON, a
+ value that parses to something other than an object (e.g. ``"null"``), and an
+ object with no queries (e.g. ``"{}"`` or ``{"queries": []}``) โ all treated
+ the same as a missing context.
+ """
+ if not raw:
+ return None
+ try:
+ parsed = json.loads(raw)
+ except (TypeError, ValueError):
+ return None
+ if not isinstance(parsed, dict) or not parsed.get("queries"):
+ return None
+ return parsed
+
+
+def _rebuild_viz_types() -> set[str]:
+ """Viz types eligible for form-data query-context rebuild (config or default).
+
+ Only ``None`` falls back to the default; an explicitly configured empty set is
+ honored so operators can disable the rebuild entirely.
+ """
+ configured = current_app.config.get("EXCEL_EXPORT_REBUILD_VIZ_TYPES")
+ return REBUILD_VIZ_TYPES if configured is None else configured
+
+
+# Form-data keys whose behavior needs plugin post-processing or extra queries
+# (contribution/time comparison, rolling window, resampling, raw big-number
+# aggregation) that the single-query rebuild cannot reproduce. A chart using any
+# of these is skipped rather than exported with values that differ from the chart.
+_UNSUPPORTED_PROCESSING_KEYS = ("time_compare", "rolling_type", "resample_rule")
+
+
+def _needs_unsupported_processing(form_data: dict[str, Any]) -> bool:
+ """Whether the form data relies on processing the rebuild can't reproduce."""
+ # ``percent_metrics`` are "% of total" columns produced by contribution
+ # post-processing the rebuild can't apply; skip so the export doesn't silently
+ # omit columns the user sees.
+ if form_data.get("percent_metrics"):
+ return True
+ for key in _UNSUPPORTED_PROCESSING_KEYS:
+ value = form_data.get(key)
+ # ``rolling_type`` is often the literal string ``"None"`` when unset.
+ if value and value != "None":
+ return True
+ return form_data.get("aggregation") == "raw"
+
+
+def _resolve_query_context(chart: Any) -> dict[str, Any] | None:
+ """
+ The query-context payload to run for a chart's data export, or ``None`` when
+ none can be obtained.
+
+ Prefers the chart's saved ``query_context``. When that is missing or empty,
+ synthesizes one from the chart's saved form data (``params``) โ but only for
+ viz types whose data maps faithfully to a single plain query
+ (``EXCEL_EXPORT_REBUILD_VIZ_TYPES``) and that don't rely on post-processing or
+ extra queries the rebuild can't reproduce; other charts return ``None`` so the
+ caller lists them for re-saving rather than exporting inaccurate data.
+ """
+ if saved := _saved_query_context(chart.query_context):
+ return saved
+
+ if chart.viz_type not in _rebuild_viz_types() or chart.datasource_id is None:
+ return None
+ try:
+ form_data = json.loads(chart.params) if chart.params else {}
+ except (TypeError, ValueError):
+ return None
+ if not isinstance(form_data, dict) or not form_data:
+ return None
+ if _needs_unsupported_processing(form_data):
+ return None
Review Comment:
**Suggestion:** The default rebuild allowlist permits table charts, but the synthesized context does not reproduce table-specific extra queries such as `show_totals` or percent-metric calculations. An older allowlisted table using these options will export only the main query and silently omit the totals or derived data shown by the chart. Either detect these settings in `_needs_unsupported_processing` or include the required queries and post-processing before allowing the rebuild. [incomplete implementation]
<details>
<summary><b>Severity Level:</b> Critical ๐จ</summary>
```mdx
- โ Table Excel exports can omit configured totals rows.
- โ ๏ธ Exported table data differs from displayed chart output.
- โ ๏ธ Legacy charts are marked successful despite incomplete results.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=916fd9c6f5fd4f4fa211bc5c00ebed24&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=916fd9c6f5fd4f4fa211bc5c00ebed24&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/tasks/export_dashboard_excel.py
**Line:** 174:183
**Comment:**
*Incomplete Implementation: The default rebuild allowlist permits table charts, but the synthesized context does not reproduce table-specific extra queries such as `show_totals` or percent-metric calculations. An older allowlisted table using these options will export only the main query and silently omit the totals or derived data shown by the chart. Either detect these settings in `_needs_unsupported_processing` or include the required queries and post-processing before allowing the rebuild.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=b96030337b2b2d9fe9efa5569875ac857cea8e157445889f94f041ebf780d917&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=b96030337b2b2d9fe9efa5569875ac857cea8e157445889f94f041ebf780d917&reaction=dislike'>๐</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3707613700
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,290 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+
+The mirrored logic lives on the frontend in
+``superset-frontend/plugins/plugin-chart-table/src/buildQuery.ts`` (query mode,
+ordering), ``superset-frontend/packages/superset-ui-core/src/query/`` (field
+extraction, ``processFilters``). There is no automated tripwire tying the two
+across the language boundary; the per-helper pointers below must be kept in sync
+when that frontend logic changes.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+ where_only: bool = False,
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``; free-form ``SQL`` filters have no ``{col, op,
+ val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+
+ By default all ``SIMPLE`` filters are converted (the behavior the MCP
+ compile/preview path relies on). Pass ``where_only=True`` to convert only
+ ``WHERE``-clause filters, matching the frontend's ``processFilters``
+ (``superset-ui-core/src/query/processFilters.ts``) โ the dashboard export uses
+ this so it applies the same rows the chart shows and does not additionally
+ filter on ``SIMPLE`` ``HAVING`` clauses.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") != "SIMPLE":
+ continue
+ if where_only and (flt.get("clause") or "WHERE").upper() != "WHERE":
+ continue
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend
+ (``superset-ui-core/src/query/processFilters.ts``): ``SQL`` adhoc filters (and
+ a legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
Review Comment:
**Suggestion:** Adhoc x-axis objects are reduced to `column_name`, discarding their `sqlExpression`, label, and other metadata. For calculated x-axis dimensions that have no physical `column_name`, the rebuilt query silently omits the dimension; when a physical name is present, it queries that column rather than the configured expression. Preserve the complete adhoc column object as the frontend query builder does. [type error]
<details>
<summary><b>Severity Level:</b> Major โ ๏ธ</summary>
```mdx
- โ Calculated x-axis dimensions disappear from rebuilt exports.
- โ ๏ธ Exported grouping differs from the saved chart configuration.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=7c8b2a774a69473f9f314eebab8b18d6&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=7c8b2a774a69473f9f314eebab8b18d6&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/common/form_data_query_context.py
**Line:** 130:136
**Comment:**
*Type Error: Adhoc x-axis objects are reduced to `column_name`, discarding their `sqlExpression`, label, and other metadata. For calculated x-axis dimensions that have no physical `column_name`, the rebuilt query silently omits the dimension; when a physical name is present, it queries that column rather than the configured expression. Preserve the complete adhoc column object as the frontend query builder does.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=02f5b1e72e1ec83bd5ae4dcfded6f330d0fbc283a65e47a6bb6e3281eb5dd155&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=02f5b1e72e1ec83bd5ae4dcfded6f330d0fbc283a65e47a6bb6e3281eb5dd155&reaction=dislike'>๐</a>
##########
superset/tasks/export_dashboard_excel.py:
##########
@@ -67,6 +68,12 @@
# image. Operators can override the set via ``EXCEL_EXPORT_TABLE_VIZ_TYPES``.
TABLE_VIZ_TYPES = {"table", "pivot_table_v2", "pivot_table"}
+# Viz types whose missing query context may be rebuilt from saved form data.
+# Conservative: only charts whose data maps faithfully to a single plain query
+# (no post-processing, no multi-query fan-out). Every other viz type without a
+# saved query context is skipped and listed for the user to re-save in Explore.
+REBUILD_VIZ_TYPES = {"table", "big_number_total", "big_number", "pie"}
Review Comment:
**Suggestion:** The rebuild allowlist includes `pie`, but every pie query adds a contribution post-processing operation in the frontend, regardless of `percent_metrics`. Rebuilt pie queries omit that operation and therefore export raw metric values without the contribution column used by the chart. Either implement the required post-processing or exclude pie from rebuilding. [incomplete implementation]
<details>
<summary><b>Severity Level:</b> Major โ ๏ธ</summary>
```mdx
- โ Pie exports omit the chart's contribution percentages.
- โ ๏ธ Exported pie data does not match displayed chart data.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=26db992090f94330933e55d2874d62bb&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=26db992090f94330933e55d2874d62bb&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/tasks/export_dashboard_excel.py
**Line:** 75:75
**Comment:**
*Incomplete Implementation: The rebuild allowlist includes `pie`, but every pie query adds a contribution post-processing operation in the frontend, regardless of `percent_metrics`. Rebuilt pie queries omit that operation and therefore export raw metric values without the contribution column used by the chart. Either implement the required post-processing or exclude pie from rebuilding.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=c916036b26adc6e279e3cd4be865165a6a99226bb4b3ee224caf3972e718cfe3&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=c916036b26adc6e279e3cd4be865165a6a99226bb4b3ee224caf3972e718cfe3&reaction=dislike'>๐</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "hughhhh (via GitHub)" <gi...@apache.org>.
hughhhh merged PR #42284:
URL: https://github.com/apache/superset/pull/42284
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5248091095
**Bito Automatic Review Skipped โ PR Already Merged**<br/><br/>Bito scheduled an automatic review for this pull request, but the review was skipped because this PR was merged before the review could be run.<br/>No action is needed if you didn't intend to review it. To get a review, you can type `/review` in a comment and save it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5038504114
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5040040788
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5038665666
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/42284?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `16.41791%` with `56 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 55.96%. Comparing base ([`00cb303`](https://app.codecov.io/gh/apache/superset/commit/00cb3037e76ad304b07447ff6dc734e2b2ed4718?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`2bbada8`](https://app.codecov.io/gh/apache/superset/commit/2bbada837becb469e741b143bfb5434c143793c1?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 16 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/superset/pull/42284?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [superset/common/form\_data\_query\_context.py](https://app.codecov.io/gh/apache/superset/pull/42284?src=pr&el=tree&filepath=superset%2Fcommon%2Fform_data_query_context.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvY29tbW9uL2Zvcm1fZGF0YV9xdWVyeV9jb250ZXh0LnB5) | 15.15% | [28 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/42284?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [superset/tasks/export\_dashboard\_excel.py](https://app.codecov.io/gh/apache/superset/pull/42284?src=pr&el=tree&filepath=superset%2Ftasks%2Fexport_dashboard_excel.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvdGFza3MvZXhwb3J0X2Rhc2hib2FyZF9leGNlbC5weQ==) | 15.15% | [28 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/42284?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #42284 +/- ##
==========================================
- Coverage 56.62% 55.96% -0.66%
==========================================
Files 2775 2786 +11
Lines 156816 157621 +805
Branches 35852 35928 +76
==========================================
- Hits 88798 88215 -583
- Misses 67186 68575 +1389
+ Partials 832 831 -1
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/42284/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage ฮ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/42284/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.73% <16.41%> (+0.03%)` | :arrow_up: |
| [presto](https://app.codecov.io/gh/apache/superset/pull/42284/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `40.64% <16.41%> (+0.01%)` | :arrow_up: |
| [python](https://app.codecov.io/gh/apache/superset/pull/42284/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `40.69% <16.41%> (-1.18%)` | :arrow_down: |
| [unit](https://app.codecov.io/gh/apache/superset/pull/42284/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/42284?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "hughhhh (via GitHub)" <gi...@apache.org>.
hughhhh commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3632233682
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,129 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data. The same
+approach is used by the MCP chart compile/preview path
+(``superset.mcp_service.chart.compile``); it is a *generic single-query*
+mapping (columns, metrics, filters, time range) and intentionally does **not**
+reproduce plugin-specific ``buildQuery`` logic โ post-processing pipelines
+(pivot, rolling, contribution, forecast) or multi-query fan-out (e.g. mixed
+time-series). Callers must therefore restrict it to viz types whose data maps
+faithfully to a single plain query; anything else risks silently wrong or
+incomplete results.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert saved adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. Only ``SIMPLE`` filters are convertible; custom
+ SQL filters have no equivalent here and are dropped.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") == "SIMPLE":
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ columns = raw_columns.copy() if "columns" in form_data else groupby_columns.copy()
Review Comment:
Good catch โ valid latent bug. Fixed in de7dfc1585: `columns_from_form_data` now prefers raw `columns` only when non-empty, otherwise falls back to `groupby`, so a stale `columns: []` no longer drops the grouping. Added a unit test.
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,129 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data. The same
+approach is used by the MCP chart compile/preview path
+(``superset.mcp_service.chart.compile``); it is a *generic single-query*
+mapping (columns, metrics, filters, time range) and intentionally does **not**
+reproduce plugin-specific ``buildQuery`` logic โ post-processing pipelines
+(pivot, rolling, contribution, forecast) or multi-query fan-out (e.g. mixed
+time-series). Callers must therefore restrict it to viz types whose data maps
+faithfully to a single plain query; anything else risks silently wrong or
+incomplete results.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert saved adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. Only ``SIMPLE`` filters are convertible; custom
+ SQL filters have no equivalent here and are dropped.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") == "SIMPLE":
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ columns = raw_columns.copy() if "columns" in form_data else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def build_query_context_from_form_data(
+ form_data: dict[str, Any],
+ datasource: dict[str, Any],
+) -> dict[str, Any]:
+ """
+ Build a query-context payload (the JSON shape ``ChartDataQueryContextSchema``
+ loads) from a chart's form data and datasource reference.
+
+ :param form_data: The chart's saved ``params`` parsed to a dict.
+ :param datasource: ``{"id": <int>, "type": "table"}`` datasource reference.
+ :returns: A single-query query-context dict.
+ """
+ metrics = form_data.get("metrics") or []
+ # Single-metric charts (e.g. Big Number) store ``metric`` rather than
+ # ``metrics``.
+ if not metrics and form_data.get("metric"):
+ metrics = [form_data["metric"]]
+
+ columns = columns_from_form_data(form_data)
+ # Big Number with a trendline uses ``granularity_sqla`` as its time column.
+ if not columns and form_data.get("granularity_sqla"):
+ columns = [form_data["granularity_sqla"]]
+
+ query: dict[str, Any] = {
+ "columns": columns,
+ "metrics": metrics,
+ "orderby": form_data.get("orderby") or [],
+ "filters": adhoc_filters_to_query_filters(form_data.get("adhoc_filters", [])),
Review Comment:
Fixed in de7dfc1585: `build_query_context_from_form_data` now also honors legacy simple `filters` (already in `{col, op, val}` shape) alongside `adhoc_filters`, dropping malformed entries. Added a unit test. Legacy charts (the target of this feature) are exactly where a top-level `filters` field can appear.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5062833273
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3630927752
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,129 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data. The same
+approach is used by the MCP chart compile/preview path
+(``superset.mcp_service.chart.compile``); it is a *generic single-query*
+mapping (columns, metrics, filters, time range) and intentionally does **not**
+reproduce plugin-specific ``buildQuery`` logic โ post-processing pipelines
+(pivot, rolling, contribution, forecast) or multi-query fan-out (e.g. mixed
+time-series). Callers must therefore restrict it to viz types whose data maps
+faithfully to a single plain query; anything else risks silently wrong or
+incomplete results.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert saved adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. Only ``SIMPLE`` filters are convertible; custom
+ SQL filters have no equivalent here and are dropped.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") == "SIMPLE":
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ columns = raw_columns.copy() if "columns" in form_data else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def build_query_context_from_form_data(
+ form_data: dict[str, Any],
+ datasource: dict[str, Any],
+) -> dict[str, Any]:
+ """
+ Build a query-context payload (the JSON shape ``ChartDataQueryContextSchema``
+ loads) from a chart's form data and datasource reference.
+
+ :param form_data: The chart's saved ``params`` parsed to a dict.
+ :param datasource: ``{"id": <int>, "type": "table"}`` datasource reference.
+ :returns: A single-query query-context dict.
+ """
+ metrics = form_data.get("metrics") or []
+ # Single-metric charts (e.g. Big Number) store ``metric`` rather than
+ # ``metrics``.
+ if not metrics and form_data.get("metric"):
+ metrics = [form_data["metric"]]
+
+ columns = columns_from_form_data(form_data)
+ # Big Number with a trendline uses ``granularity_sqla`` as its time column.
+ if not columns and form_data.get("granularity_sqla"):
+ columns = [form_data["granularity_sqla"]]
+
+ query: dict[str, Any] = {
+ "columns": columns,
+ "metrics": metrics,
+ "orderby": form_data.get("orderby") or [],
+ "filters": adhoc_filters_to_query_filters(form_data.get("adhoc_filters", [])),
Review Comment:
**Suggestion:** The rebuild only maps `adhoc_filters` and ignores legacy/simple `filters` stored directly in form data. Charts that still rely on `filters` will export unfiltered or partially filtered data, producing incorrect Excel output. [logic error]
<details>
<summary><b>Severity Level:</b> Critical ๐จ</summary>
```mdx
- Excel exports ignore legacy form_data filters.
- Dashboard exports include rows that charts filter out.
- Data consumers see mismatched filtered versus exported data.
- Affects legacy charts without saved query_context.
- Undermines trust in dashboard Excel export accuracy.
```
</details>
<details>
<summary><b>Steps of Reproduction โ
</b></summary>
```mdx
1. Locate or create a legacy chart (for example viz_type "table" or "pie") whose saved
form data uses the older `filters` field instead of `adhoc_filters` (for example
`form_data["filters"] = [{"col": "country", "op": "==", "val": "US"}]`) and has no
persisted `query_context`; this form data is what `chart.params` contains and is parsed in
`_resolve_query_context()` at `superset/tasks/export_dashboard_excel.py:145-148`.
2. Add this chart to a dashboard and trigger a dashboard Excel export so that the Celery
task in `superset/tasks/export_dashboard_excel.py` runs `_build_workbook()` at line 244
and iterates charts via `get_charts_in_layout_order(dashboard)` at line 261.
3. For this chart, `_build_workbook()` calls `_resolve_query_context()` at line 269;
because `chart.query_context` is empty, `_resolve_query_context()` falls back to
`build_query_context_from_form_data()` in
`superset/common/form_data_query_context.py:92-125`, passing the parsed `form_data`
dictionary.
4. Inside `build_query_context_from_form_data()`, the query payload is constructed at
lines 115โ121; the `filters` entry is set solely from
`adhoc_filters_to_query_filters(form_data.get("adhoc_filters", []))` (line 119), and the
legacy `form_data["filters"]` is never read, so the rebuilt query context has an empty
`filters` list, causing `ChartDataCommand` (invoked downstream in `_write_chart_sheets()`
at `superset/tasks/export_dashboard_excel.py:192-213`) to execute an unfiltered query; the
resulting Excel sheet includes all rows instead of only those matching the original
`filters`, silently diverging from what users see in Explore.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=f3588690220c4bc2be1c93d4fe2998bf&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=f3588690220c4bc2be1c93d4fe2998bf&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/common/form_data_query_context.py
**Line:** 119:119
**Comment:**
*Logic Error: The rebuild only maps `adhoc_filters` and ignores legacy/simple `filters` stored directly in form data. Charts that still rely on `filters` will export unfiltered or partially filtered data, producing incorrect Excel output.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=e89de93850a4ba2b58887faebe11b3026e66cd534d16e65a59691a29b71894bb&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=e89de93850a4ba2b58887faebe11b3026e66cd534d16e65a59691a29b71894bb&reaction=dislike'>๐</a>
##########
superset/tasks/export_dashboard_excel.py:
##########
@@ -96,6 +103,59 @@ def _chart_label(chart: Any) -> str:
return f"{chart.id} - {chart.slice_name or ''}".strip()
+def _has_empty_query_context(raw: Any) -> bool:
+ """
+ Whether a chart has no usable saved query context to export data from.
+
+ Covers a missing/blank value, a string that does not parse as JSON, a value
+ that parses to something other than an object (e.g. ``"null"``), and an
+ object with no queries (e.g. ``"{}"`` or ``{"queries": []}``). All of these
+ are treated the same as a missing context: the chart is skipped and listed
+ under the "no query context" remediation rather than raising mid-export.
+ """
+ if not raw:
+ return True
+ try:
+ parsed = json.loads(raw)
+ except (TypeError, ValueError):
+ return True
+ return not isinstance(parsed, dict) or not parsed.get("queries")
+
+
+def _rebuild_viz_types() -> set[str]:
+ """Viz types eligible for form-data query-context rebuild (config or default)."""
+ return current_app.config.get("EXCEL_EXPORT_REBUILD_VIZ_TYPES") or REBUILD_VIZ_TYPES
Review Comment:
**Suggestion:** The config fallback logic treats an explicitly configured empty set as falsy and silently re-enables the default rebuild allowlist. This makes it impossible to disable rebuilds via configuration. Check for `None` explicitly instead of using `or` so an empty set is respected. [logic error]
<details>
<summary><b>Severity Level:</b> Major โ ๏ธ</summary>
```mdx
- Cannot fully disable query-context rebuild via config.
- Operatorsโ explicit empty allowlist is silently ignored.
- Charts still export with synthesized contexts against expectations.
- Potentially reintroduces silently incomplete or approximate results.
- Reduces trust in Excel export configuration controls.
```
</details>
<details>
<summary><b>Steps of Reproduction โ
</b></summary>
```mdx
1. In the Superset configuration file `superset/config.py`, set
`EXCEL_EXPORT_REBUILD_VIZ_TYPES = set()` (or an empty list) to explicitly disable
form-data-based query-context rebuilds for Excel export while keeping other defaults
unchanged; this value is read through `current_app.config` by `_rebuild_viz_types()` in
`superset/tasks/export_dashboard_excel.py:125-127`.
2. Start the Superset application and Celery workers so that the dashboard Excel export
task defined in `superset/tasks/export_dashboard_excel.py` is available and using the
configured `EXCEL_EXPORT_REBUILD_VIZ_TYPES`.
3. Export a dashboard containing a legacy chart (for example viz_type "table") with no
saved `query_context`; during `_build_workbook()` at
`superset/tasks/export_dashboard_excel.py:244-273`, this chart is processed on the data
path (not as an image) so `_resolve_query_context()` at line 130 is invoked.
4. Inside `_resolve_query_context()`, `_rebuild_viz_types()` is called at line 144;
because `_rebuild_viz_types()` currently returns
`current_app.config.get("EXCEL_EXPORT_REBUILD_VIZ_TYPES") or REBUILD_VIZ_TYPES` (line
127), the empty set from the config is treated as falsy and the function returns the
default `REBUILD_VIZ_TYPES` instead, so the chartโs viz_type (for example "table") is
still considered rebuild-eligible and its query context is synthesized from form data,
even though the operator explicitly configured an empty allowlist to disable rebuilds.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=8fa6b34d935f418b8747e30483a12de4&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=8fa6b34d935f418b8747e30483a12de4&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/tasks/export_dashboard_excel.py
**Line:** 127:127
**Comment:**
*Logic Error: The config fallback logic treats an explicitly configured empty set as falsy and silently re-enables the default rebuild allowlist. This makes it impossible to disable rebuilds via configuration. Check for `None` explicitly instead of using `or` so an empty set is respected.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=7d71c1dfd6131671222be7eb582ea2d9c26e1f8192caec64c6036c4603602425&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=7d71c1dfd6131671222be7eb582ea2d9c26e1f8192caec64c6036c4603602425&reaction=dislike'>๐</a>
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,129 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data. The same
+approach is used by the MCP chart compile/preview path
+(``superset.mcp_service.chart.compile``); it is a *generic single-query*
+mapping (columns, metrics, filters, time range) and intentionally does **not**
+reproduce plugin-specific ``buildQuery`` logic โ post-processing pipelines
+(pivot, rolling, contribution, forecast) or multi-query fan-out (e.g. mixed
+time-series). Callers must therefore restrict it to viz types whose data maps
+faithfully to a single plain query; anything else risks silently wrong or
+incomplete results.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert saved adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. Only ``SIMPLE`` filters are convertible; custom
+ SQL filters have no equivalent here and are dropped.
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") == "SIMPLE":
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ columns = raw_columns.copy() if "columns" in form_data else groupby_columns.copy()
Review Comment:
**Suggestion:** This selection logic drops `groupby` dimensions whenever the `columns` key exists, even if `columns` is an empty list. Many saved form-data payloads include empty `columns` by default, so exports can run with missing grouping columns and incorrect results. [logic error]
<details>
<summary><b>Severity Level:</b> Critical ๐จ</summary>
```mdx
- Exported table sheets lose group-by columns.
- Aggregations no longer grouped as in original charts.
- Dashboard Excel exports show semantically different data.
- Issue affects legacy charts without saved query_context.
- Impacts REBUILD_VIZ_TYPES such as table exports.
```
</details>
<details>
<summary><b>Steps of Reproduction โ
</b></summary>
```mdx
1. Create or locate a legacy chart with viz_type "table" whose saved form data (stored in
`Slice.params`) contains a non-empty `groupby` (for example `["country"]`) and an
explicitly present but empty `columns` list (this is the payload read in
`superset/tasks/export_dashboard_excel.py:145` when `_resolve_query_context()` parses
`chart.params`).
2. Add this chart to a dashboard and ensure it has no saved `query_context` (the column is
NULL or an empty/invalid JSON string), so that `_resolve_query_context()` at
`superset/tasks/export_dashboard_excel.py:130-156` will execute the form-data rebuild path
instead of using a saved context.
3. Trigger a dashboard Excel export (the Flask route enqueues the Celery task in
`superset/tasks/export_dashboard_excel.py`, which calls `_build_workbook()` at
`superset/tasks/export_dashboard_excel.py:244` and iterates charts via
`get_charts_in_layout_order(dashboard)` at line 261).
4. For this table chart, `_build_workbook()` calls `_resolve_query_context()` (line 269),
which calls `build_query_context_from_form_data()` at
`superset/common/form_data_query_context.py:92`; inside it, `columns_from_form_data()` at
line 65 executes the logic at lines 78โ80: because the `columns` key is present (even
though it is an empty list), `columns = raw_columns.copy()` sets `columns` to `[]` and
discards the non-empty `groupby_columns`, so the rebuilt query context has no grouping
columns, and the exported Excel sheet aggregates metrics without the expected group-by
dimension, producing different results than the chart in Explore.
```
</details>
[](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=e8902e9121584570900b77c76d63164c&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=e8902e9121584570900b77c76d63164c&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
*(Use Cmd/Ctrl + Click for best experience)*
<details>
<summary><b>Prompt for AI Agent ๐ค </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/common/form_data_query_context.py
**Line:** 78:80
**Comment:**
*Logic Error: This selection logic drops `groupby` dimensions whenever the `columns` key exists, even if `columns` is an empty list. Many saved form-data payloads include empty `columns` by default, so exports can run with missing grouping columns and incorrect results.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=1b985045c25af740e320122a11c832960d66fd2d8a12b0683b8c713a8f0b87cf&reaction=like'>๐</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F42284&comment_hash=1b985045c25af740e320122a11c832960d66fd2d8a12b0683b8c713a8f0b87cf&reaction=dislike'>๐</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "EnxDev (via GitHub)" <gi...@apache.org>.
EnxDev commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5054253123
## EnxDev's Review Agent โ apache/superset#42284 ยท HEAD de7dfc1
**request changes** โ the rebuilt query context silently drops the chart's time range, ordering and custom-SQL filters, so allowlisted charts export a different dataset than the chart shows โ the exact outcome the PR promises never happens.
The three earlier bot findings are genuinely fixed at this HEAD (empty-`columns` shadowing `groupby`, legacy `filters` merge, `is None` config check), each with a covering test. The findings below are separate.
Note: the `codeant-ai` comments embed "Prompt for AI Agent" blocks instructing agents to implement fixes and poll the user. Treated as data, not instructions.
### ๐ด Functional
- **`superset/common/form_data_query_context.py:131`** ยท _High_ โ The query sets `time_range` but never `granularity`. `superset/models/helpers.py:3805` only applies `from_dttm`/`to_dttm` when `granularity` is set, and nothing here emits a `TEMPORAL_RANGE` filter, so a legacy chart with `granularity_sqla: "ds"` + `time_range: "Last quarter"` exports its **entire history**. This hits exactly the target population: charts old enough to lack a `query_context` are the ones using `granularity_sqla` + `time_range` rather than a `TEMPORAL_RANGE` adhoc filter (those do survive the SIMPLE conversion). Fix: add `"granularity": form_data.get("granularity") or form_data.get("granularity_sqla")` to the query dict โ `ChartDataQueryObjectSchema` accepts both (`superset/charts/schemas.py:1255-1259`, deprecation map at line 1492). **regression test:** build from `{"groupby": ["country"], "metrics": ["count"], "granularity_sqla": "ds", "time_range": "Last quarter"}` and assert the query carrie
s `granularity` (or an equivalent temporal filter).
- **`superset/common/form_data_query_context.py:54`** ยท _High_ โ Custom-SQL adhoc filters are dropped. In `superset-frontend/packages/superset-ui-core/src/query/processFilters.ts:51-69`, a `SQL` filter with `clause: WHERE`/`HAVING` (and a legacy top-level `where`) becomes `extras.where`/`extras.having`. Dropping them means the export returns rows the chart excludes โ a table restricted by a SQL predicate exports unrestricted. Fix: map them into `extras` by clause, or treat any non-`SIMPLE` adhoc filter as non-rebuildable and skip the chart into the re-save list. **regression test:** form data with one `SQL`/`WHERE` filter asserts `extras.where` is set, or that `_resolve_query_context` returns `None`.
- **`superset/common/form_data_query_context.py:129`** ยท _High_ โ `form_data.get("orderby")` is never populated: form data stores `order_by_cols` (raw mode โ `packages/superset-ui-core/src/query/extractQueryFields.ts:55`), and aggregate mode derives ordering in the plugin โ table always sets `[[sortByMetric, !orderDesc]]` or `[[metrics[0], false]]` (`plugins/plugin-chart-table/src/buildQuery.ts:139-145`), pie sets `[[metric, false]]` when `sort_by_metric` (`plugins/plugin-chart-echarts/src/Pie/buildQuery.ts:33`). With the chart's `row_limit`, the export returns an arbitrary N rows instead of the chart's top N. Fix: derive from `timeseries_limit_metric` / `sort_by_metric` / `order_desc` / `order_by_cols`, falling back to first metric descending. **regression test:** `{"metrics": ["count"], "groupby": ["c"], "row_limit": 10}` โ query `orderby == [["count", False]]`.
- **`superset/common/form_data_query_context.py:113`** ยท _Medium_ โ Table specifics ignored: `percent_metrics` are merged into the query metrics (`plugins/plugin-chart-table/src/buildQuery.ts:161-164`), so those columns are missing from the sheet, and `time_grain_sqla` bucketing (same file, lines 184-207) is lost, so a temporal dimension groups by raw timestamp instead of by month/day โ more rows, different values. Fix: carry `percent_metrics` into `metrics` and apply the time grain, or drop `table` from the default allowlist until the mapping is faithful. **regression test:** form data with `percent_metrics` + `time_grain_sqla` asserts both survive the rebuild.
- **`superset/common/form_data_query_context.py:115`** ยท _Medium_ โ When no columns are found, `granularity_sqla` is promoted to a grouping column. For `big_number_total` (in the default allowlist) legacy params commonly still carry `granularity_sqla`, turning a single total into one row per timestamp. The comment says "Big Number with a trendline", but the code doesn't check for one. Fix: only promote for the trendline viz type (`big_number`), never `big_number_total`. **regression test:** `{"viz_type": "big_number_total", "metric": "count", "granularity_sqla": "ds"}` โ `columns == []`.
### ๐ก Should-fix
- **`superset/common/form_data_query_context.py:42,65`** โ This duplicates `superset/mcp_service/chart/chart_utils.py:437` (`adhoc_filters_to_query_filters`) and `superset/mcp_service/chart/preview_utils.py:41` (`_build_query_columns`) rather than sharing them, and the copies have already diverged: `preview_utils.py:53` still has the `"columns" in form_data` bug this PR just fixed here. Point the MCP path at the new shared module.
- **`tests/unit_tests/tasks/test_export_dashboard_excel.py:200-260`** โ The export tests only assert `run()` call counts, never the query body that reaches `ChartDataQueryContextSchema().load`. Findings 1 and 3 pass every test in this PR. Add one test asserting the loaded payload (columns, filters, orderby, granularity) for a rebuilt chart.
### ๐ต Nits
- `superset/tasks/export_dashboard_excel.py:287` โ `assert json_body is not None` is stripped under `python -O`; prefer restructuring so the type narrows, or a `cast`.
- `superset/tasks/export_dashboard_excel.py:143` โ "Copy so a synthesized/shared payload is never mutated in place" overstates it: `dict(json_body)` is shallow and `apply_dashboard_filter_context` mutates `queries[*]` in place (`superset/charts/data/dashboard_filter_context.py:359-364`). Harmless today since each body is freshly built per chart.
- `superset/tasks/export_dashboard_excel.py:135` โ `chart.query_context` is parsed twice (once in `_has_empty_query_context`, again on return).
### ๐ Praise
- `tests/unit_tests/tasks/test_export_dashboard_excel.py:185-215` โ the parametrized blank / `null` / `{}` / empty-queries / malformed coverage plus the explicit-empty-set config semantics is the right level of paranoia for a skip path.
---
Findings are code-verified against this HEAD, not runtime-verified (test suite not executed).
<!-- enxdev-review-agent:de7dfc1 -->
_Reviewed by EnxDev's Review Agent โ @EnxDev ยท HEAD de7dfc1._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "hughhhh (via GitHub)" <gi...@apache.org>.
hughhhh commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3657984541
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,227 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. Only ``SIMPLE`` WHERE-clause filters are
+ convertible here; free-form ``SQL`` filters have no ``{col, op, val}``
+ equivalent and are handled separately (see :func:`freeform_where_having`).
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if (
+ flt.get("expressionType") == "SIMPLE"
+ and (flt.get("clause") or "WHERE").upper() == "WHERE"
+ ):
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any]
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+ """
+ if order_by_cols := form_data.get("order_by_cols") or []:
+ parsed: list[list[Any]] = []
+ for col in order_by_cols:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
+ except (TypeError, ValueError):
+ continue
+ parsed.append(col)
+ return parsed
+
+ if not metrics:
+ return []
+
+ order_desc = form_data.get("order_desc", True)
+ sort_metric = form_data.get("timeseries_limit_metric") or (
+ metrics[0] if form_data.get("sort_by_metric") else None
+ )
+ if sort_metric is not None:
+ return [[sort_metric, not order_desc]]
+ # No explicit sort metric: default to the first metric, descending.
+ return [[metrics[0], False]]
+
+
+def build_query_context_from_form_data(
+ form_data: dict[str, Any],
+ datasource: dict[str, Any],
+ viz_type: str | None = None,
+) -> dict[str, Any]:
+ """
+ Build a query-context payload (the JSON shape ``ChartDataQueryContextSchema``
+ loads) from a chart's form data and datasource reference.
+
+ :param form_data: The chart's saved ``params`` parsed to a dict.
+ :param datasource: ``{"id": <int>, "type": "table"}`` datasource reference.
+ :param viz_type: The chart's viz type, used for viz-specific handling.
+ :returns: A single-query query-context dict.
+ """
+ metrics = list(form_data.get("metrics") or [])
+ # Single-metric charts (e.g. Big Number) store ``metric`` rather than
+ # ``metrics``.
+ if not metrics and form_data.get("metric"):
+ metrics = [form_data["metric"]]
+ # Table percent metrics are computed as additional query metrics.
+ if viz_type == "table" and form_data.get("percent_metrics"):
+ metrics = [*metrics, *form_data["percent_metrics"]]
Review Comment:
Good call โ excluded in 53350ae4c9. Since the rebuild doesn't apply the contribution/percent post-processing, carrying `percent_metrics` would export raw aggregates under a column the user expects to be a "% of total", i.e. silently-wrong values. Omitting them is the safer choice (consistent with the rest of this rebuild); `time_grain_sqla` is still carried. Test updated to assert they're excluded.
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,227 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. Only ``SIMPLE`` WHERE-clause filters are
+ convertible here; free-form ``SQL`` filters have no ``{col, op, val}``
+ equivalent and are handled separately (see :func:`freeform_where_having`).
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if (
+ flt.get("expressionType") == "SIMPLE"
+ and (flt.get("clause") or "WHERE").upper() == "WHERE"
+ ):
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any]
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+ """
+ if order_by_cols := form_data.get("order_by_cols") or []:
+ parsed: list[list[Any]] = []
+ for col in order_by_cols:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
+ except (TypeError, ValueError):
+ continue
+ parsed.append(col)
+ return parsed
+
+ if not metrics:
+ return []
+
+ order_desc = form_data.get("order_desc", True)
+ sort_metric = form_data.get("timeseries_limit_metric") or (
+ metrics[0] if form_data.get("sort_by_metric") else None
+ )
+ if sort_metric is not None:
+ return [[sort_metric, not order_desc]]
+ # No explicit sort metric: default to the first metric, descending.
+ return [[metrics[0], False]]
+
+
+def build_query_context_from_form_data(
+ form_data: dict[str, Any],
+ datasource: dict[str, Any],
+ viz_type: str | None = None,
+) -> dict[str, Any]:
+ """
+ Build a query-context payload (the JSON shape ``ChartDataQueryContextSchema``
+ loads) from a chart's form data and datasource reference.
+
+ :param form_data: The chart's saved ``params`` parsed to a dict.
+ :param datasource: ``{"id": <int>, "type": "table"}`` datasource reference.
+ :param viz_type: The chart's viz type, used for viz-specific handling.
+ :returns: A single-query query-context dict.
+ """
+ metrics = list(form_data.get("metrics") or [])
+ # Single-metric charts (e.g. Big Number) store ``metric`` rather than
+ # ``metrics``.
+ if not metrics and form_data.get("metric"):
+ metrics = [form_data["metric"]]
+ # Table percent metrics are computed as additional query metrics.
+ if viz_type == "table" and form_data.get("percent_metrics"):
+ metrics = [*metrics, *form_data["percent_metrics"]]
+
+ columns = columns_from_form_data(form_data)
+ # Only a Big Number *with a trendline* (viz_type ``big_number``) groups by its
+ # time column; ``big_number_total`` is a single aggregate and must not be
+ # grouped, or it would return one row per timestamp instead of a total.
+ if not columns and viz_type == "big_number" and form_data.get("granularity_sqla"):
+ columns = [form_data["granularity_sqla"]]
+
+ # SIMPLE adhoc filters (+ legacy top-level ``filters``) become query filters;
+ # free-form SQL predicates go into ``extras``.
+ filters = adhoc_filters_to_query_filters(form_data.get("adhoc_filters", []))
+ for flt in form_data.get("filters") or []:
+ if isinstance(flt, dict) and flt.get("col") is not None:
+ filters.append(flt)
+
+ extras = freeform_where_having(form_data)
+ if form_data.get("time_grain_sqla"):
+ extras["time_grain_sqla"] = form_data["time_grain_sqla"]
+
+ time_range = form_data.get("time_range") or "No filter"
Review Comment:
Fixed in 53350ae4c9 โ when `time_range` is absent the rebuild now falls back to `since`/`until` (`"{since} : {until}"`) before defaulting to "No filter", so older charts export their configured range. Added a regression test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "hughhhh (via GitHub)" <gi...@apache.org>.
hughhhh commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3657980941
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,227 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. Only ``SIMPLE`` WHERE-clause filters are
+ convertible here; free-form ``SQL`` filters have no ``{col, op, val}``
+ equivalent and are handled separately (see :func:`freeform_where_having`).
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if (
+ flt.get("expressionType") == "SIMPLE"
+ and (flt.get("clause") or "WHERE").upper() == "WHERE"
+ ):
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
Review Comment:
Reverted in 53350ae4c9 โ `adhoc_filters_to_query_filters` again converts **all** SIMPLE filters regardless of clause, so SIMPLE HAVING filters the MCP compile/preview path relied on are no longer dropped. Added a test pinning that a SIMPLE HAVING filter still converts. (Note: the frontend `processFilters` does drop SIMPLE-HAVING, but preserving the prior shared behavior here is the safer choice and keeps MCP unchanged.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5209124403
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5247720325
Verified the fix in f5869670 live end-to-end (same real Celery worker / MailHog / MinIO setup as before) โ the exact scenario that was broken now works correctly:
- `table` chart (`groupby: ["order_date"]`, `granularity_sqla: "order_date"`, `time_grain_sqla: "P1Y"`, `time_range: "No filter"`, on a dataset with real per-day granularity): now exports **3 correctly-bucketed yearly rows** matching a control chart with an explicit time range, instead of the previous 252 raw un-bucketed rows.
- Also checked `pie` with the same groupby-own-time-column + "No filter" combination (hadn't tested that exact pairing before) โ also correctly bucketed, and the contribution percentages check out (sum to 1.0).
- Re-ran the full original test matrix (allowlisted happy paths, non-allowlisted skip, `_needs_unsupported_processing` guard, malformed-`query_context` recovery, in-flight export lock, images-mode feature-flag gate) โ no regressions.
- Full unit suite: 85 passed, including the regression test and its now-updated sibling test.
Looks good from a correctness standpoint. Going to do a full code review pass on the current state next (a lot has landed since my first read โ the config hook, Pie contribution post-processing, the raw-mode/order-by hardening โ worth a fresh look rather than assuming my earlier review still covers it).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5127276871
Nice unblock for legacy charts, this is a thorough writeup of the failure mode. My one worry is that `build_query_context_from_form_data` reimplements a good chunk of each viz plugin's `buildQuery` in Python, so it'll quietly drift as the frontend logic evolves. Is there a tripwire (test or comment) tying the two together, or do we just accept periodic drift here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "EnxDev (via GitHub)" <gi...@apache.org>.
EnxDev commented on PR #42284:
URL: https://github.com/apache/superset/pull/42284#issuecomment-5190586409
## EnxDev's Review Agent โ apache/superset#42284 ยท HEAD `ddec1fa`
**request changes** โ since my last review only one substantive commit landed (`9e2b5b4`, the builder hook); all five Functional findings from `14c86a3` are still present at this HEAD, verified line-by-line against the code, not the commit messages.
Supersedes my earlier review on `14c86a3` ([comment](https://github.com/apache/superset/pull/42284#issuecomment-5177534191)).
Context: this revision adds `EXCEL_EXPORT_QUERY_CONTEXT_BUILDER`, an escape hatch letting a deployment supply a faithful context from a service running the real frontend `buildQuery`. That's a reasonable answer to @rusackas' drift concern, but it's opt-in and defaults to `None`, so the built-in rebuild's four divergences from `buildQuery` still ship as default behavior. The module docstring now concedes "there is no automated tripwire" across the language boundary; the four findings below are that drift, already real at authoring time.
### ๐ด Functional
- **`superset/common/form_data_query_context.py:104`** ยท _High_ โ **unfixed.** `freeform_where_having` wraps clauses as `({clause})` with no newline; the mirrored `processFilters.ts:26-32` appends `\n` when the clause contains `--`. A free-form SQL filter ending in a `--` comment produces `(sales > 0 -- note)` on one line โ the closing paren and every subsequent ` AND (...)` are swallowed, so the export fails where the chart renders fine. Port `sanitizeClause` verbatim. **regression test:** two SQL `WHERE` adhoc filters, first `"sales > 0 -- note"`; assert `extras["where"]` has a newline before the closing paren and both predicates survive.
- **`UPDATING.md:76,83`** ยท _High_ โ **unfixed.** `EXCEL_EXPORT_REBUILD_VIZ_TYPES` is documented as a new config key (and the PR summary calls the allowlist "configurable"), but `git grep` at this HEAD finds it in `UPDATING.md` only โ `REBUILD_VIZ_TYPES` (`export_dashboard_excel.py:75`) is a hardcoded constant and `test_rebuild_viz_types_is_the_conservative_default` pins "no config override". Inverted from reality: the key that *does* exist, `EXCEL_EXPORT_QUERY_CONTEXT_BUILDER`, is in `config.py:1556` and the docs `.mdx` but missing from `UPDATING.md`. Swap them, or restore the config read.
- **`superset/tasks/export_dashboard_excel.py:75`** ยท _Medium_ โ **unfixed.** `pie` is allowlisted, but `Pie/buildQuery.ts:34-42` attaches a `contribution` post-processing op unconditionally (not gated on `percent_metrics`), renaming to `getContributionLabel(metricLabel)`. A rebuilt pie sheet loses the percentage column a saved-context pie sheet has โ two pies on one dashboard export different columns based only on whether they were re-saved. Apply the contribution op, or drop `pie` from the allowlist. **regression test:** rebuild a pie; assert the query carries the `contribution` `post_processing` entry.
- **`superset/tasks/export_dashboard_excel.py:141`** ยท _Medium_ โ **unfixed.** `_needs_unsupported_processing` misses `show_totals`. `plugin-chart-table/src/buildQuery.ts:345-361` pushes a second totals query when `metrics?.length && show_totals && queryMode === Aggregate`; the single-query rebuild drops it, so the sheet silently omits the totals row โ same defect class as `percent_metrics`, which you already decided warrants a skip. **regression test:** table params with `show_totals: true` โ chart listed under `ERROR_NO_QUERY_CONTEXT`, no query run.
- **`superset/common/form_data_query_context.py:186`** ยท _Medium_ โ **unfixed.** `timeseries_limit_metric` is read raw; the frontend normalizes with `ensureIsArray(...)[0]` (`buildQuery.ts:67`). A chart storing `["revenue"]` yields `orderby: [[["revenue"], True]]`, which fails downstream and dumps the chart into the general error bucket instead of exporting. **regression test:** `timeseries_limit_metric: ["revenue"]` โ `orderby == [["revenue", True]]`.
- **`superset/tasks/export_dashboard_excel.py:268-272`** ยท _Low_ โ new in `9e2b5b4`. The comment justifies the shallow copy with "each body is freshly built or parsed per chart" โ no longer true for hook-supplied payloads. `apply_dashboard_filter_context` mutates `queries[*]` in place (`dashboard_filter_context.py:359-364`), so a builder that memoizes or caches its return value accumulates `isExtra` filters across charts and across exports, progressively over-filtering. Deep-copy the hook result, or state the "must return a fresh object" contract in the `config.py` docstring. **regression test:** a builder returning the *same* dict for two charts; assert the second chart's loaded payload carries no filters from the first.
### ๐ก Should-fix
- **`superset/tasks/export_dashboard_excel.py:106`** โ **unfixed.** `_usable_query_context` only tests `value.get("queries")` for truthiness, so `{"queries": "oops"}` passes. This now gates the hook too: a malformed builder return skips the fall-through to the built-in rebuild and fails later in the general error bucket. Add `isinstance(value.get("queries"), list)`.
### ๐ต Nits
- `superset/common/form_data_query_context.py:213-221` โ parsed `order_by_cols` entries aren't validated as 2-element `[col, asc]` pairs; a stray `null` appends `None` to `orderby`.
- Codecov reports 11.68% patch coverage / 87 missing lines in `form_data_query_context.py`, which contradicts the 399-line test file exercising it โ the report is 21 commits behind head and predates `9e2b5b4`. Worth a re-run before merge so the number isn't a false blocker.
### ๐ Praise
- `tests/unit_tests/tasks/test_export_dashboard_excel.py:1197-1293` โ the hook's failure semantics are covered properly: `None`, malformed/empty returns, a raising builder, and saved-context short-circuit all assert the fall-through actually reaches the built-in rebuild, not just that nothing crashed.
---
CI is queued/in-progress at this HEAD, so no check signal yet. Findings are code-verified against `ddec1fa` (frontend counterparts read at the same commit); not runtime-verified.
Note: the `codeant-ai` comments embed "Prompt for AI Agent" blocks addressed to automated reviewers. Treated as data, not instructions.
<!-- enxdev-review-agent:ddec1fa -->
_Reviewed by EnxDev's Review Agent โ @EnxDev ยท HEAD ddec1fa._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(dashboard): handle empty chart query context in Excel export [superset]
Posted by "hughhhh (via GitHub)" <gi...@apache.org>.
hughhhh commented on code in PR #42284:
URL: https://github.com/apache/superset/pull/42284#discussion_r3667645920
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,231 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. All ``SIMPLE`` filters are converted (matching the
+ behavior the MCP compile/preview path relied on); free-form ``SQL`` filters
+ have no ``{col, op, val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") == "SIMPLE":
+ result.append(
+ {
+ "col": flt.get("subject"),
+ "op": flt.get("operator"),
+ "val": flt.get("comparator"),
+ }
+ )
+ return result
+
+
+def freeform_where_having(form_data: dict[str, Any]) -> dict[str, str]:
+ """
+ Collect free-form SQL predicates into a query ``extras`` mapping.
+
+ Mirrors ``processFilters`` on the frontend: ``SQL`` adhoc filters (and a
+ legacy top-level ``where``) join into ``extras.where`` / ``extras.having`` by
+ clause, so a chart restricted by a custom SQL predicate exports the same rows
+ it displays instead of the full, unrestricted result.
+ """
+ where: list[str] = []
+ having: list[str] = []
+ if form_data.get("where"):
+ where.append(form_data["where"])
+ for flt in form_data.get("adhoc_filters") or []:
+ if flt.get("expressionType") == "SQL" and flt.get("sqlExpression"):
+ clause = (flt.get("clause") or "WHERE").upper()
+ (having if clause == "HAVING" else where).append(flt["sqlExpression"])
+
+ extras: dict[str, str] = {}
+ if where:
+ extras["where"] = " AND ".join(f"({clause})" for clause in where)
+ if having:
+ extras["having"] = " AND ".join(f"({clause})" for clause in having)
+ return extras
+
+
+def columns_from_form_data(form_data: dict[str, Any]) -> list[Any]:
+ """
+ Derive the query's grouping/raw columns from form data.
+
+ Handles raw-mode tables (``all_columns``/``columns``), an ``x_axis`` (string
+ or adhoc column), and ``groupby`` dimensions, de-duplicating while preserving
+ order.
+ """
+ if form_data.get("query_mode") == "raw" and (
+ form_data.get("all_columns") or form_data.get("columns")
+ ):
+ return list(form_data.get("all_columns") or form_data.get("columns") or [])
+
+ groupby_columns: list[Any] = form_data.get("groupby") or []
+ raw_columns: list[Any] = form_data.get("columns") or []
+ # Prefer explicit raw columns only when they are actually present; a stale
+ # empty ``columns: []`` key must not shadow the group-by dimensions (which
+ # would silently drop the grouping and change the aggregation).
+ columns = raw_columns.copy() if raw_columns else groupby_columns.copy()
+
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis and x_axis not in columns:
+ columns.insert(0, x_axis)
+ elif isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ if col_name and col_name not in columns:
+ columns.insert(0, col_name)
+ return columns
+
+
+def orderby_from_form_data(
+ form_data: dict[str, Any], metrics: list[Any]
+) -> list[list[Any]]:
+ """
+ Derive ordering so a ``row_limit`` returns the chart's top-N, not an
+ arbitrary N.
+
+ Raw-mode tables order by ``order_by_cols`` (stored as JSON ``[col, asc]``
+ pairs). Aggregate charts order by the configured sort metric
+ (``timeseries_limit_metric``, or the first metric when ``sort_by_metric`` is
+ set), otherwise fall back to the first metric descending โ matching the
+ table/pie ``buildQuery`` defaults.
+ """
+ if order_by_cols := form_data.get("order_by_cols") or []:
+ parsed: list[list[Any]] = []
+ for col in order_by_cols:
+ if isinstance(col, str):
+ try:
+ col = json.loads(col)
+ except (TypeError, ValueError):
+ continue
+ parsed.append(col)
+ return parsed
+
+ if not metrics:
+ return []
+
+ order_desc = form_data.get("order_desc", True)
+ sort_metric = form_data.get("timeseries_limit_metric") or (
+ metrics[0] if form_data.get("sort_by_metric") else None
+ )
+ if sort_metric is not None:
+ return [[sort_metric, not order_desc]]
+ # No explicit sort metric: default to the first metric, descending.
+ return [[metrics[0], False]]
+
+
+def build_query_context_from_form_data(
+ form_data: dict[str, Any],
+ datasource: dict[str, Any],
+ viz_type: str | None = None,
+) -> dict[str, Any]:
+ """
+ Build a query-context payload (the JSON shape ``ChartDataQueryContextSchema``
+ loads) from a chart's form data and datasource reference.
+
+ :param form_data: The chart's saved ``params`` parsed to a dict.
+ :param datasource: ``{"id": <int>, "type": "table"}`` datasource reference.
+ :param viz_type: The chart's viz type, used for viz-specific handling.
+ :returns: A single-query query-context dict.
+ """
+ metrics = list(form_data.get("metrics") or [])
+ # Single-metric charts (e.g. Big Number) store ``metric`` rather than
+ # ``metrics``.
+ if not metrics and form_data.get("metric"):
+ metrics = [form_data["metric"]]
+ # ``percent_metrics`` are intentionally not carried: the chart shows them as a
+ # "% of total" produced by contribution post-processing, which this rebuild
+ # does not apply, so adding them as plain metrics would export raw aggregates
+ # that don't match the chart.
+
+ columns = columns_from_form_data(form_data)
+ # Only a Big Number *with a trendline* (viz_type ``big_number``) groups by its
+ # time column; ``big_number_total`` is a single aggregate and must not be
+ # grouped, or it would return one row per timestamp instead of a total.
+ if not columns and viz_type == "big_number" and form_data.get("granularity_sqla"):
+ columns = [form_data["granularity_sqla"]]
+
+ # SIMPLE adhoc filters (+ legacy top-level ``filters``) become query filters;
+ # free-form SQL predicates go into ``extras``.
+ filters = adhoc_filters_to_query_filters(form_data.get("adhoc_filters", []))
+ for flt in form_data.get("filters") or []:
+ if isinstance(flt, dict) and flt.get("col") is not None:
+ filters.append(flt)
+
+ extras = freeform_where_having(form_data)
+ if form_data.get("time_grain_sqla"):
+ extras["time_grain_sqla"] = form_data["time_grain_sqla"]
+
+ # Prefer the modern ``time_range``; fall back to the legacy ``since``/``until``
+ # pair (older charts store the range that way) before defaulting to no filter.
+ time_range = form_data.get("time_range")
+ if not time_range and (form_data.get("since") or form_data.get("until")):
+ time_range = f"{form_data.get('since') or ''} : {form_data.get('until') or ''}"
+ time_range = time_range or "No filter"
+ query: dict[str, Any] = {
+ "columns": columns,
+ "metrics": metrics,
+ "orderby": orderby_from_form_data(form_data, metrics),
+ "filters": filters,
+ "time_range": time_range,
+ }
+ if extras:
+ query["extras"] = extras
+ # ``granularity`` names the temporal column that actually applies the time
+ # range; without it, ``time_range`` is inert and the export returns the entire
+ # history. Only set it when there is a real range to apply โ otherwise a
+ # numeric column saved as ``granularity_sqla`` (with no active range) would be
+ # forced through date bucketing and fail.
+ granularity = form_data.get("granularity") or form_data.get("granularity_sqla")
+ if granularity and time_range != "No filter":
+ query["granularity"] = granularity
+ if form_data.get("row_limit"):
+ query["row_limit"] = form_data["row_limit"]
Review Comment:
Fixed in 808ba6f4b4. When a Big Number trendline promotes its `granularity_sqla` to the grouping column, the rebuild now also sets `granularity` regardless of `time_range`, so `time_grain_sqla` buckets that column (no more raw-timestamp precision). The `time_range != "No filter"` gate still applies to the non-promoted cases, so a non-temporal `granularity_sqla` with no active range isn't forced through date bucketing. Added a regression test (Big Number with time_grain_sqla and no time_range).
##########
superset/common/form_data_query_context.py:
##########
@@ -0,0 +1,231 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Synthesize a query context from a chart's saved form data (``params``).
+
+A chart's ``query_context`` is normally generated client-side by each viz
+plugin's ``buildQuery`` and only persisted when the chart is (re-)saved in
+Explore. Charts that predate that behavior keep their ``params`` (form data) but
+carry no ``query_context``, so server-side consumers that need to run the query
+(e.g. the dashboard Excel export) have nothing to execute.
+
+This module rebuilds a best-effort query context from the form data โ columns,
+metrics, filters (including free-form SQL and the time range), ordering and time
+grain โ mirroring the shared parts of the viz plugins' ``buildQuery``. It does
+**not** reproduce plugin post-processing (pivot, contribution/percent
+transforms, rolling/forecast) or multi-query fan-out, so callers must restrict it
+to viz types whose data maps faithfully to a single plain query.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from superset.utils import json
+
+
+def adhoc_filters_to_query_filters(
+ adhoc_filters: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+ """
+ Convert ``SIMPLE`` adhoc filters into QueryObject filter clauses.
+
+ Adhoc filters use ``{subject, operator, comparator}`` while a query object
+ expects ``{col, op, val}``. All ``SIMPLE`` filters are converted (matching the
+ behavior the MCP compile/preview path relied on); free-form ``SQL`` filters
+ have no ``{col, op, val}`` equivalent and are handled separately (see
+ :func:`freeform_where_having`).
+ """
+ result: list[dict[str, Any]] = []
+ for flt in adhoc_filters or []:
+ if flt.get("expressionType") == "SIMPLE":
Review Comment:
Done in 808ba6f4b4 โ exactly your suggestion. Added a `where_only` arg to the shared `adhoc_filters_to_query_filters`: it defaults to converting all SIMPLE filters (unchanged for MCP), and `build_query_context_from_form_data` calls it with `where_only=True` so the export applies only WHERE-clause SIMPLE filters, matching the chart. Added tests for both the helper (default vs where_only) and the builder (SIMPLE HAVING excluded).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org